Deep learning workstations need server-grade CPUs, a ton of RAM and at least four high-end GPUs… Right?
This assumption makes most deep learning workstations extremely expensive: around five to twenty-five thousand euros if you want to buy a pre-configured and fully assembled machine. They are obviously built for raw performance regardless of the costs, but is this really necessary?
What if you designed a cheaper workstation that is maximised for value and upgradability instead? A workstation with a smart choice of components in order to achieve a good performance per cost now, but that will also be able to utilise the more powerful, next generation of GPUs in the future...
To achieve this goal a lot of research and a solid understanding of all the components that go into a modern computer were necessary. It's fairly easy to buy the best of everything, pay a fortune and call it a day. Choosing the right components to find the balance that allowed me to build the best value-for-money machine was way more challenging. Oh, and I also wanted it to look nice.
Along the way I encountered more compatibility restrictions than I imagined, which got even more complicated with dynamic prices and availability of the components - not to mention size, performance, noise, design and support considerations. More on these in the details of this build.
But before the deep dive begins I would like to note that most product links in this blog post are Amazon affiliate links. This means that the price doesn't change for you, but I will get a small commission fee for orders made through those links.
CPU: AMD Ryzen 7 3700X
CPU Cooler: Noctua NH-U12S (optional, CPU comes with cooler)
Mainboard: ASUS PRO WS X570-ACE ATX-Workstation
RAM: Corsair Vengeance LPX 32GB (2x16GB) DDR4 3200MHz (2x for 64GB)
Corsair MP600 1TB Gen 4 PCIe x4, NVMe M.2 SSD
WiFi Card: Cudy AX3000 WiFi 6 PCIe-Adapter, Bluetooth 5.0 (optional)
Case: NZXT H510
Best value: 1x or 2x Asus RTX 2070 SUPER Turbo
Best performance: 1x or 2x Titan RTX
Power supply: Corsair RM850x, adapt for GPU needs
The first decision I had to make was whether I was going to use an intel or AMD CPU for my build. After a little research the decision was easy: AMD simply provided a better value, especially with its latest 3rd Generation Ryzen processors.
Specifically, I chose the eight core AMD Ryzen 7 3700X. It comes with PCIe 4.0 support which, if you are not familiar with it, is the bus that connects all components on the motherboard to the CPU. The new 4.0 version is twice as fast as the current 3.0 standard, which will be essential to utilise the full potential of upgrades in the future. The Ryzen 7 3700X also has a lower power consumption compared to equivalent intel processors and is a great deal over all in my opinion.
By the way: The next best processor, the AMD Ryzen 7 3800X, is the exact same silicon chip as the 3700X, but operates at a higher frequency, which brings performance gains at the cost of a higher power consumption. To do this AMD selects the best chips of a production batch to become 3800Xs and brands the others as 3700X. That means with a bit of luck a 3700X is a perfectly fine 3800X in disguise and just didn't make it to meet the demand of both processors. I didn't experiment with overclocking yet, so I can't tell if my CPU is what is known as a "gold chip". But I think it's an interesting insight and at the time I bought mine it didn't justify the price increase for me.
Why not more cores? Well, I think 8 cores should be enough to start with. In GPU intensive applications like training a neural network, the CPU is not as important as long as it doesn't limit the GPU. But if you want more cores, AMD recently released this 16 core beast: AMD Ryzen 9 3950x. Everything above that needs server grade components that would increase the overall build price substantially.
Another benefit of the AMD Ryzen processors is that they come with the AMD Wraith Prism cooler included, which performs really well for a stock cooler.
Interesting fact I didn't know: No RGB cable doesn't mean no RGB. The cooler shows all colours of the rainbow by default and has no way (yet) to disable that on Linux. For now I don't mind, but it somewhat disturbs the black-stealth look I was trying to achieve. In the future I might upgrade to an Noctua NH-U12S, a brand known for silent operation and great performance.
Every component is connected to the mainboard which makes it an essential part. Its features determine the upgradability of the system in the future.
For my build I chose the ASUS PRO WS X570-ACE ATX-Workstation Mainboard . It is designed for dual-GPU work loads and professional use, which becomes apparent by the missing RGB support (standard for high-end gaming mainboards) and the focus on pro features like ECC Memory support, a durable and passively cooled CPU power supply and two PCIe 4.0 x16 expansion slots for the GPUs. For a full list of features see the ASUS Product page. (Note: Max RAM is 128GB, not 64GB like it says on the overview page)
The X570 in the name refers to the chipset on the motherboard which is responsible for key features, like the support of PCIe 4.0 and the USB standard. It's the latest chipset and was released together with the 3rd Gen Ryzen processors, which would also work with older chipsets, but not to their full potential. The X570 is somewhat special in that it needs active cooling, a small fan on the mainboard. I have read that a lot of people online are concerned about noise levels of that little fan, but so far I couldn't hear anything.
There are four considerations when it comes to RAM: Size, speed, latency and of course price. For my build I chose the Corsair Vengeance LPX 32GB (2x16GB) DDR4 3200MHz C16 XMP 2.0 RAM. Just using two 16GB sticks out of the four slots the mainboard offers, gives me an easy way to double my RAM to 64GB by simply buying another set later on.
My conclusion after watching a lot of videos about the best RAM for the 3rd Gen Ryzen processors is that faster is only better in some situations. The base RAM clock speed supported by the CPU is 3200MHz - at this frequency everything synchronises perfectly. But it is also possible to go higher, with 3600MHz and beyond bringing small performance gains. On the other hand I also read that fast RAM is more gaming specific requirement and won't have a noticeable impact on deep learning workloads. In the end I opted for the 32GB with 3200MHz and fairly low latency because of the price and availability in Austria. Depending on your location this might be different. This Video helped me a lot to choose the right RAM. Make sure the mainboard supports the chosen memory kit as well!
A development I completely missed is how common M.2 SSD drives in desktop computers are these days. They are small, extremely fast and some even need passive cooling. The Corsair MP600 1TB Gen 4 PCIe x4, NVMe M.2 SSD is also the first component that already takes advantage of the PCIe 4.0 bus. In addition to this super quick 1TB storage, which you should use to boot and run applications from. I also added a used 4TB hard-disk I already owned to store large datasets and backups.
The mainboard has no integrated WiFi, but this is easily fixable with a WiFi expansion card. I got mine from an old PC I disassembled, but I would recommend a card like this one: Cudy AX3000 WiFi 6 PCIe-Adapter, Bluetooth 5.0. It supports WiFi 6 (802.11ax) which is the fastest standard available and is based on an intel chip rather than Realtek. The internet opinion on this topic is that intel has better driver support and causes little to no pain compared to Realtek. Make sure that the card is compatible with your operating system!
I love the minimalist design of the NZXT H510. It has a very good build quality and great options for cable management. The only thing I would critique is that the thumb screws were so tight, I needed a tool to loosen them and that the top case fan was rattling at first. Lucky this was easily fixed by tightening the screws there a little bit more.
Unfortunately, I only noticed when assembling that the top USB-C 3.2 Gen 2 Port is not connectable to the mainboard. While the mainboard supports USB 3.2 Gen 2 (4x Type A and 1x Type C at the back) it only has two USB 3.2 Gen 1 connections - but no Gen 2.
By the way: Who came up with those confusing names? You already started numbering, just use it!
Anyways, another interesting design choice is that all fans are installed to blow outwards by default. This means that air is sucked in at every opening, which usually leads to more dust build-up over time compared to other fan configurations. To deal with this most intakes are dust filtered and interestingly enough in this case review video they found that negative air pressure is an effective strategy for this case design.
We've come to the most important and likely most expensive part of this build - the GPUs. The best price to computation value in the consumer section at the time of writing has the NVIDIA RTX 2070 Super. It's the 2019 predecessor to the RTX 2070, but with a similar performance to the RTX 2080 at a lower price. It is also the cheapest card to support NV-Link with the disadvantage that it only has 8GB of VRAM.
The memory size is actually one of the most critical specs to keep in mind when it comes to deep learning work loads! In my opinion it is much more important than GPU performance, because a slower computation won't hurt the workflow as much as a model or data not fitting into memory at all. This is especially true because the number of parameters of state-of-the-art deep learning models has recently exploded. A good example for that are all transformer based models in NLP, which can truly get massive.
Unfortunately cards with bigger memory, for example the RTX 2080 Ti (11GB) and the Titan RTX (24GB) are ridiculously more expensive. But if you need a lot of VRAM, the Ryzen 7 3700X should have enough power to support two Titan RTX cards. In this case I would suggest upgrading the RAM to 64GB, if not even more.
But even on a budget two RTX 2070 Super cards can give you 16GB of VRAM for the price of a single RTX 2080 Ti, which only has 11GB. On the other hand, it is also rumoured that new cards with bigger memory will be announced at some point in 2020. My current strategy is to use the single GPU I have right now and wait for the new generation which is likely to have more VRAM at a similar price.
Regardless of the card you are going to pick, the training of a neural network is a very long and intensive task. This is why I would recommend picking up a blower style card like the Asus RTX 2070 SUPER Turbo or the Asus RTX 2080 Ti Turbo to avoid turning your case into a hot box, which would hurt performance as well as all of your components. A blower type card exhausts hot air directly out the back, instead of circulating it in the case. This is especially important if you run a dual GPU setup, as both cards are close to each other. Gamers usually dislike blower style cards but they are the right choice for a workstation in my opinion.
The power supply is a very important component, especially with power hungry high-end GPUs. It is vital for the stability of the system to select a power supply with enough headroom, so it is a good idea to also factor in planned updates. To make this task easier there is this awesome Power Supply Calculator tool.
If you are not familiar with PC assembly, I can highly recommend this first person view PC build video by Linus Tech Tips to guide you through the process. It took me about 3 hours and doing everything step by step with the video was not very complicated in my opinion.
Keep in mind that static electricity is not a myth and it can actually damage your PC! If you stick with the AMD Wraith Prism cooler, make sure not to damage the already applied thermal paste at the bottom. And yes, the mounting lever has to be pressed down with a lot of force. This is normal.
Happy workstation building! :)