Hardware Components of a Ai Server

Build and Setup Your Own Deep Learning Server From Scratch

Earlier this year, I completed the "Practical Deep Learning — Part 1" course by Jeremy Howard's. It west as a businesslike form that teaches you lot how to practise various deep learning techniques using AWS. AWS was a way to get upwardly and running really apace but the cost adds up apace. I had a p2 instance for about $0.9/hour and let's not forget all the extra subconscious toll that comes with AWS. I was racking up $50–$100/month. It was affecting my productivity because I was trying to limit my use of AWS to reduce cost. I wanted to pursue my more than in deep learning and decided to take the plunge and build my own deep learning server. This stuff changes very chop-chop, so I will non exist surprise if the content is outdated by the time you read this. However, I hope this volition give you some insights on how to build your ain server.

6 Steps to Building Your Own Deep Learning Server

1. Select Components
2. Hardware Assembly
3. Install Operating System
four. Install Graphics Card and Commuter
5. Setup Deep Learning Surround
6. Setup Remote Access

1. Select Components

Information technology's useful to do a bunch of inquiry (i.e. reading blogs) to become an idea on what parts you need to buy.

Utilise pcpartpicker.com before you brand your purchases. The site helps y'all do a "compatibility check" to make sure all your components are uniform with each other. My function listing is here.

In general you need the following:

CPU — Intel — Core i5–6600K 3.5GHz Quad-Core Processor $289.50
RAM — G.Skill — Ripjaws V Serial 32GB (2 ten 16GB) DDR4–2133 Memory $330
GPU — EVGA — GeForce GTX 1070 8GB SC Gaming ACX 3.0 Video Card $589
SSD — Samsung — 850 EVO-Serial 500GB 2.5" Solid State Drive $200
Motherboard — MSI — Z270-A PRO ATX LGA1151 Motherboard $140
CPU Libation — Cooler Master — Hyper 212 EVO 82.nine CFM Sleeve Begetting CPU Cooler $30
Power Supply — EVGA — SuperNOVA NEX 650W 80+ Gilded Certified Fully-Modular ATX Power Supply $120
Case — Corsair — 200R ATX Mid Belfry Case $75
Total: under $1800 CAD

Primal Takeaways:

• GPU is the about of import component! If you are on a tight budget, make sure you spend your money on a good graphics carte. NVIDIA has many years of experience building graphics card. Do some additional research and you'll run into what I hateful. At the time of writing, GTX1080Ti was the latest best option available, however a GTX1070 will do just fine.
• Recall virtually components that enables you to upgrade in the future. Consider paying a petty extra to go a motherboard that can back up multiple 16e PCIe slots. This enables you to install an boosted GPU if yous want to get a performance boost later. Similarly, brand certain there's enough RAM slots for upgrades every bit well.
• When in uncertainty go for parts that has the most customer reviews. I notice that if there's a lot of reviews, it ways the role has been tested and y'all're more probable to find help or how-to-guides online.
• Buy from trusted sellers! pcpartpicker.com will give you pricing from various sellers. I noticed at that place was a reseller that offered components at a slightly lower cost. It'south e'er nice to save a couple bucks whenever possible. I was about to purchase a few items from this *less well-known* reseller until I looked at their reviews. Thank goodness I did because their reviews were horrible! I opted on the safer side and bought my components from my stores I trust such as Amazon and newegg.
• Gratuitous Aircraft & Handling: Consider buying your parts from places that offer costless shipping & treatment such as Amazon. You can save yourself some money.

Other helpful resources:

• Building your ain deep learning box
• Not bad Deep Learning Box Assembly Setup and Benchmarks
• Deep Learning Hardware Guide
• Setting upwardly a deep learning car in a lazy all the same quick way
• Build Personal Deep Learning Rig

ii. Hardware Associates

I oasis't touched computer hardware *stuff* since I was in loftier-school so I was a little nervous at the start. If you're actually not comfortable putting the hardware together, you can caput to Canada Computers, or the equivalent, and they can assembly it together for you for nigh $eighty. For me, I didn't want to reject on this great learning opportunity.

Before doing annihilation, I recommend looking through the instruction manuals and how-to videos to familiarize yourself with the various components. Yous've already spent then much $$$, you might as well take a picayune time to brand sure yous know what you lot're doing!

Here are some helpful videos for the components that I used:

Key Takeaways:

• Watch all the videos before starting. Words and manuals tin only communicate so much. You'll feel much more confident after watching someone else install it. There's oft a few gotchas that manuals don't point out.
• Brand sure everything is plugged in properly. I was afraid of breaking my parts and so I was really gentle with everything. The commencement time I plugged in the power cord and turned on the power, nothing happened — no lights, no audio. I idea I must accept broken something and wasted $g. I turned it off and checked all the plugs to brand sure everything was plugged in properly. Load and behold everything worked afterwards.
• Don't install your GPU nonetheless! Wait till you install your Bone. I fabricated this mistake and thought my motherboard was broken because I got a blackness screen on my monitor. Just exit your video card on a side for at present. See details below — I defended a department only for information technology.

3. Install Operating System

I chose to install ubuntu 16.04 with a USB. You tin technically install Windows likewise just I didn't see the need for it at the moment.

iii.one Create a Bootable USB Drive

1. Get a 2GB or larger USB.
ii. Download the ISO file from Ubuntu. I used 16.04
3. Follow these instructions to create a bootable USB stick.

3.two Install Ubuntu

Reboot your arrangement and the Ubuntu setup screen should pop upward. Follow the instructions on the screen and it should be pretty straight-forrad. Hither's a useful tutorial.

Some people may not automatically get the Ubuntu setup screen. If that'south the case for you, you lot just need to reboot into the BIOS (for me information technology'south pressing F11 when the figurer starts) and configure your boot priority to load the USB drive first earlier the hard bulldoze (SSD).

3.iii Getting Up-to-date

Open your last and run the post-obit commands:

          sudo apt-get update
sudo apt-become upgrade
sudo apt-become install build-essential cmake thousand++ gfortran git pkg-config python-dev software-backdrop-mutual wget
sudo apt-get autoremove
sudo rm -rf /var/lib/apt/lists/*        

4. Install Graphics Carte du jour and its Driver

NVIDIA graphics cards are tricky to install. It's a bit of a craven and egg considering yous tin't use the graphics card until you lot install the driver, just you tin't really install the driver unless the motherboard detects that your graphics card is installed. My motherboard also automatically boots the graphics card first if information technology detects that there's a card in the PCI slot BUT because I don't accept the drivers installed even so, I just finish up getting a blank screen. So what do you do? After doing a bunch of research, I found the following steps work. Promise it will save you some trouble.

4.1 Change the BIOS Settings

Motherboards should have an integrated graphics device so you can hook up your monitor to the motherboard direct. By at present, you should take your monitor hooked upwardly to the motherboard brandish device. Since we're going to install a graphics card, this is where yous need to tell the BIOS which graphics device to initiate kickoff. There are 2 settings in the BIOS:

• PEG — PCI Express Graphics: motherboard will kicking the graphics card if at that place's one detected on the PCI slot. If at that place isn't one, then it volition kick the congenital-in one from the motherboard.
• IGD — Integrated Graphics Device: volition always use the motherboard'south built-in card

By default the BIOS will have the setting set to PEG. In normal circumstances once you have the NVIDIA driver installed, this makes sense considering you lot desire to leverage the card. Notwithstanding, it causes problem when you have to install the NVIDIA commuter.

Alter the BIOS config by:

1. restart the computer and boot into the BIOS (F11).
2. Goto: Settings -> Integrated Graphics Configuration -> Initiate Graphic Adapter
3. Change settings from PEG to IGD
4. Save and turn off your computer

4.2 Physically Insert The Graphics Card

Now that the arrangement is turned off, insert the graphics card into the PCI slot. Sometimes you lot have to push harder than you think. Make sure it's fully in.

4.three Install The Driver

At present that the graphics bill of fare is continued to the motherboard, turn on your machine to install the NVIDIA driver.

1. Notice your graphics card model

          lspci | grep -i nvidia        

2. Determine the latest version of NVIDIA commuter available for your graphics card

• Visit the graphics drivers PPA page and see the versions of drivers bachelor. Every bit of August 2017, 384 is the latest version.
• Visit the NVIDIA driver downloads site to determine the version that is compatible with your card and Os.

3. Add and update the graphics-driver PPA

          sudo add-apt-repository ppa:graphics-drivers
sudo apt-get update        

4. Install NVIDIA commuter. Enter the version number that is compatible to your card. For instance, I installed version 384

          sudo apt-go install nvidia-384        

5. Reboot your organisation and verify your installation

          lsmod | grep nvidia        

If you accept issues and demand to start again fresh, y'all can purge everything by using the following command. I hear that people who are upgrading graphics menu ofttimes run into drivers issue.

          sudo apt-get purge nvidia*        

half-dozen. Change your BIOS graphics device priority to PEG over again. (See step two)

5. Setup Deep Learning Environs

Earlier yous start installing anything, or blindly follow instructions from a blog, take some fourth dimension to understand what library version you need. These things modify so fast that most blogs volition be outdated by the time yous read them. Likewise, don't blindly download and install the latest version either. For case, Tensorflow 1.3 supports up-to python iii.half-dozen (as of August 1st 2017) just Theano only supports up-to python 3.5. Anaconda's latest version runs on python iii.six simply you lot can create a conda environment with python 3.five. The lawmaking I used in part i of my deep learning grade was written with pythonn2.7 but part ii of the course uses python 3.5 — for that reason I created 2 conda environments.

I ended up using the following list of libraries and its corresponding version.

CUDA (v8) —a parallel computing platform by leveraging the power of GPU

cuDNN (v6)— CUDA Deep Neural Network library that sits on-pinnacle of CUDA.

Anaconda (v3.six)— A good package and environment manager that comes with a lot of data scientific calculating tools such as Numpy, Pands and Matplotlib. The other do good of Anaconda is it makes it easy to create custom python environments.

Tensorflow (v1.3)— Google's machine learning framework

Theano (v0.9) — An alternative car learning framework.

Keras — college level neural network library that runs on top of Tensorflow, Theano or other automobile learning framework.

5.one Install CUDA

• Goto the NVIDIA website and find download the CUDA Toolkit.
• Run the following code to install CUDA. Change the version that correspond to the packet that's compatible to your system.

          wget http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_8.0.44-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1604_8.0.44-1_amd64.deb
sudo apt-get update
sudo apt-get install cuda        

• Add the path to your environment

          echo 'consign PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
source ~/.bashrc        

• bank check that you have installed the right version

          nvcc -V        

• bank check out details about your video carte du jour

          nvidia-smi        

v.2 Install cuDNN

• Goto the NVIDIA website to download the library. You have to sign-upwardly in social club to download. The site says it may take up to a few days to corroborate but the plough-around fourth dimension for me was instant.
• Extract and copy the files over to where CUDA is installed (usually /usr/local/cuda)

          cd ~/<binder you downloaded the file>
            tar xvf cudnn*.tgz
            cd cuda
            sudo cp */*.h /usr/local/cuda/include/
            sudo cp */libcudnn* /usr/local/cuda/lib64/
            sudo chmod a+r /usr/local/cuda/lib64/libcudnn*        

5.3 Install Anaconda

• Goto the website to download the installer and follow the instructions. The latest Tensorflow version (at the time of writing it is v1.iii) needs Python 3.5. So download the python 3.six version and you tin create a conda environment with a specific python version.
• Create a virtual environment with the following command. More than details on how to manage the surround can be constitute hither. Yous may want to create one environment for Theano and one for Tensorflow since they oftentimes support different versions of libraries.

          conda create -n tensorflowenv anaconda python=3.5        

• Actuate the conda environment you lot but created

          source activate tensorflowenv        

5.4 Install TensorFlow

• Install the TensorFlow library. In that location's a unlike bundle depending on your OS, Python version and CPU vs GPU support. Have a look here to decide the TensorFlow binary URL you should use.

          pip install --ignore-installed --upgrade <tensorFlowBinaryURL>                  

For case, y'all would use the following command if you are running Python iii.v on Ubuntu with GPU support.

          pip install --ignore-installed --upgrade \              
            https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.3.0-cp35-cp35m-linux_x86_64.whl        

For more details about TensorFlow installation, encounter here.

5.5 Install Theano

You can discover installation details here. However, information technology's pretty much only one command.

          sudo pip install Theano        

v.half dozen Install Keras

• Install Keras with the post-obit command

          sudo pip install keras        

• Depending on which backend you lot utilize (TensorFlow or Theano) you accept to configure it accordingly.

          vi ~/.keras/keras.json        

The configuration file looks something like this. You just accept to alter "backend" to "tensorflow" or "theano" depending on what backend y'all're using.

          {            
            "image_data_format": "channels_last",            
            "epsilon": 1e-07,            
            "floatx": "float32",            
            "backend": "tensorflow"            
}        

More details about Keras installation: 1, ii

5.7 Other useful resource:

• NVIDIA GTX 1080 on Ubuntu xvi.04 for Deep Learning
• Setting upwardly a Deep Learning Machine from Scratch

6. Setup Remote Access

This step is optional, but if you plan to work remotely from your laptop, here are some options.

6.i Teamviewer

Teamviewer is basically a screen share software. It'southward a little clunky at times, but information technology's a adept option for remote command.

6.two SSH

Most developers will want to SSH into the box via a terminal and don't really demand a graphical user interface. You tin can use OpenSSH.

You can use the following commands to install and check its condition. By default, the OpenSSH server will showtime running on first boot.

          sudo apt-get install openssh-server
sudo service ssh status        

6.three Remote Jupyter Notebook

I apply Jupyter notebook quite a bit. If you want to run information technology off of your laptop'due south browser, this is a pretty absurd fob. Bank check out a thorough tutorial here. In summary, but run the following code on your laptop and server.

          $laptop: ssh -l <username>@<yourNewServerIP>
$server: jupyter notebook --no-browser --port=8888
$laptop: ssh -NL 8888:localhost:8888 <username>@<yourNewServerIP>        

Then you tin goto http://localhost:8888 on your laptop'southward browser and remotely view/edit your Jupyter notebooks.

That'due south it! I'd love to hear your setup experience!

Share this post