There are essentially two types of accelerator cards for deep neural networks: training and inference. The former should speak for itself, accelerating the training of a deep neural network before it’s deployed in the field. Inference, however, is the process of providing an input to the deep neural network and having it extract data based on that input. That includes translating speech in real-time and localizing faces in images.
According to Nvidia, the new Tesla P4 and Tesla P40 accelerator cards are designed for inferencing and include specialized inference instructions based on 8-bit operations, making them 45 times faster in response time than an Intel Xeon E5-2690v4 processor. They also provide a 4x improvement over the company’s previous generation of “Maxwell” Tesla cards, the M40 and M4.
The company said this week during its GTC Beijing 2016 conference that the Tesla P4 sports a small form-factor that’s ideal for data centers. It’s 40x more energy efficient than CPUs that are used for inferencing, and a single Tesla P4 server can replace 13 CPU-only servers built for video inferencing workloads. Meanwhile, the Tesla P40 is ideal for deep learning workloads, with a server containing eight of these accelerators able to replace more than 140 CPU-based servers.
Compared to the previous Tesla M40, the new P40 packs more CUDA cores, higher clock speeds, a faster memory clock, a higher single precision of 12 TFLOPS, and a higher number of transistors at 12 billion. However, the power requirement (thermal envelope) stays the same, thus Nvidia has managed to boost the performance-per-watt level without forcing the card to require more power. The same holds true with the slower Tesla P4 model too when compared to the older Tesla M4 card.
“With the Tesla P100 and now Tesla P4 and P40, NVIDIA offers the only end-to-end deep learning platform for the data center, unlocking the enormous power of AI for a broad range of industries,” said Ian Buck, general manager of accelerated computing at Nvidia. “They slash training time from days to hours. They enable insight to be extracted instantly. And they produce real-time responses for consumers from AI-powered services.”
Nvidia revealed the Tesla P100 during its local GTC 2016 conference five months ago. This card is ideal for accelerating neural network training, delivering a performance increase of more than 12 times compared to the previous generation Maxwell-based solution. Again, neural networks need to be trained first before they’re deployed into the field, and the new Tesla card speeds up the process, cutting AI training down from weeks to days.
In addition to the two new Tesla cards, Nvidia also launched TensorRT, a library for “optimizing deep learning models for production deployment.” The company also introduced the Nvidia DeepStream SDK for simultaneously decoding and analyzing up to 93 HD video streams. However, here’s a brief list of hardware details for Nvidia’s two new Tesla cards that are now avaialble:
| Tesla P40 | Tesla P4 | |
| GPU | GP102 | GP104 |
| CUDA Cores | 3,840 | 2,560 |
| Base Clock | 1,303MHz | 810MHz |
| Boost Clock | 1,531MHz | 1,063MHz |
| GDDR5 Memory Clock | 7.2Gbps | 6Gbps |
| Memory Bus Width | 384-bit | 256-bit |
| GDDR5 Amount | 24GB | 8GB |
| Single Precision | 12 TFLOPS | 5.5 TFLOPS |
| TDP | 250 watts | 50 to 75 watts |
