At the 2022 Taipei International Computer Show (Computex), Nvidia announced the launch ofLiquid-cooled A100 PCIe GPU, to meet customer demand for high-performance carbon-neutral data centers. This is the first of its kind in a mainstream server GPU.

At the same time, NVIDIA announced that a number of leading manufacturers have adopted the world’s first batch ofSystem design based on NVIDIA’s self-developed data center CPUand more than 30 global technology partners released the first edge AI and embedded computing systems based on NVIDIA Jetson AGX Orin at Computex.

At present, NVIDIA is developing comprehensively around the three pillars of data center chips: CPU, GPU, and DPU, to assist its partners in building a new wave of data center transformation and building modern AI factories. Among them, the CPU manages the operation of the entire system, the GPU is responsible for providing core computing power, and the DPU is responsible for handling secure network communications and providing network built-in computing power to optimize overall performance.

Brian Kelleher, Nvidia’s senior vice president of hardware engineering, revealed that Nvidia has set a two-year update cadence for each chip architecture.One year will be dedicated to the x86 platform and one year will be dedicated to the Arm platformregardless of customer and market preference, the NVIDIA architecture and platform will support both x86 and Arm.

Ian Buck, vice president of NVIDIA’s accelerated computing business, said that if all the world’s AI, high-performance computing, and data analysis workloads are run on GPU servers, NVIDIA estimates that more than 12 trillion watt-hours of electricity can be saved each year, equivalent to Take 2 million cars off the road every year.

1. Liquid-cooled GPU: same performance, less power consumption

Liquid cooling technology was born in the mainframe era and matured in the AI ​​era. Today, Liquid Cooling Technology is widely used in high-speed supercomputers around the world in the form of direct-to-chip cooling. NVIDIA GPUs are already 20 times more energy efficient than CPUs in AI inference and high-performance computing, and accelerated computing will naturally use liquid cooling technology.

Nvidia estimates that switching all of the world’s CPU servers running AI and high-performance computing to GPU-accelerated systems could save up to 11 terawatt-hours of energy annually. The amount of energy saved could cover more than 1.5 million homes for 1 year.

Today, Nvidia released the first to adoptData Center PCIe GPUs with Direct Chip Cooling Technology.This liquid-cooled GPU, which reduces power consumption while maintaining performance, is in trial phase and is expected to be released soon.this summerOfficially released.

Equinix, a global service provider that manages more than 240 data centers, has validated the A100 80GB PCIe liquid-cooled GPU in its data centers as part of the company’s comprehensive approach to sustainable cooling and heat capture.

In separate tests, Equinix and Nvidia both found that data center workloads with liquid cooling can be on par with air-cooled facilities,At the same time, the energy consumption is reduced by about 30%. Nvidia estimates that the PUE for liquid-cooled data centers could reach 1.15, well below the 1.6 for air-cooled data centers.

Liquid-cooled data centers can double the amount of computing in the same space. This is because the A100 GPU uses only one PCIe slot, while the air-cooled A100 GPU uses two PCIe slots.

Later this year, at least a dozen system makers, including Asus, H3C, Inspur, Ningchang, Supermicro, and Hyperfusion, will use liquid-cooled GPUs in their products.

It is reported that NVIDIA plans to launch a version of the A100 PCIe card next year with the H100 Tensor Core GPU based on the NVIDIA Hopper architecture. In the near term, NVIDIA plans to apply liquid cooling technology to its own high-performance data center GPUs and NVIDIA HGX platforms.

2. Dozens of NVIDIA Grace CPU-based servers will ship next year

Grace is NVIDIA’s first data center CPU built for AI workloads. The chip is expected to ship next year and will be available in two form factors.

The Grace-Hopper on the left side of the above figure is a single super-chip module designed to accelerate large-scale AI, high-performance computing, cloud and hyperscale workloads. It implements a chip-level direct connection between the Grace CPU and the Hopper GPU. The GPUs communicate via NVLink-C2C, an interconnect technology with bandwidths up to 900GB/s.

According to Brian Kelleher, Grace will transfer data to Hopper 15 times faster than any other CPU and increase Hopper’s working data size to 2TB.

At the same time, Nvidia also offers the Grace super chip that interconnects two Grace CPU chips through NVLink-C2C.The Grace super chip features 144 high-performance Armv9 CPU cores, memory bandwidth up to 1TB/s, and 2x the energy efficiency of existing servers. The entire module, including 1TB of memory, consumes only 500W.

In addition to NVLink-C2C, NVIDIA also supports UCIe, the still-evolving chiplet standard released earlier this year.

Today, NVIDIA is announcing 4 Grace reference designs for standard data center workloads:

• 1. CGX for cloud games;

• 2. OVX for digital twins and Omniverse;

• 3. HGX suitable for high performance computing and supercomputing;

• 4. HGX for AI training, inference and high performance computing.

Immediately after, Nvidia announced the HGX Grace and HGX Grace Hopper systems, which will provide Grace Hopper and Grace CPU super-chip modules and their corresponding PCB reference designs. Both are designed as OEM 2U high-density server chassis for reference and modification by NVIDIA partners.

Dozens of server models of Grace systems from ASUS, Foxconn Industrial Internet, GIGABYTE, QCT, Supermicro and Wiwynn are expected to begin shipping in the first half of 2023.

3. The first Jetson AGX Orin servers and devices released

The NVIDIA Isaac robotics platform has four pillars: one is to create AI; two is to simulate the operation of a robot in a virtual world and then try it out in the real world; three is to build a physical robot; and four is to manage the entire lifecycle of a deployed robot fleet .

In building and deploying real-world physical robots, NVIDIA Jetson has become the AI ​​platform for edge and robotics, with more than 1 million developers, more than 150 partners, and more than 6,000 companies using Jetson for volume production.

Jetson AGX Orin features NVIDIA Ampere Tensor Core GPUs, 12 Arm Cortex-A78AE CPUs, next-generation deep learning and vision accelerators, high-speed interfaces, faster memory bandwidth, multi-modal sensors, delivering 275 teraflops of performance , equivalent to a “handheld server”.

With the same pin compatibility and form factor, it has 8x more processing power than its predecessor, the NVIDIA AGX Xavier.

The Jetson AGX Orin developer kit has been available globally through resellers since March, and production modules will be available in July starting at $399. The Orin NX module measures just 70mm x 45mm and will be available in September.

For edge AI and embedded computing applications, more than 30 global NVIDIA partners such as AAEON, ADLINK, and Advantech released the first batch of NVIDIA Jetson AGX Orin-based production systems at Computex, covering servers, edge devices, industrial PCs, on-board boards, AI software and other categories.

The products will be available in fanned and fanless configurations with a variety of connectivity and interface options, and will incorporate specifications for critical economic sectors such as robotics, manufacturing, retail, transportation, smart cities, healthcare, or ruggedized applications.

To accelerate the development of AMRs, Nvidia is also introducing Isaac Nova Orin, an advanced computing and sensor reference design for AMRs.

Nova Orin consists of 2 Jetson AGX Orin, supports 2 stereo cameras, 4 wide angle cameras, 2 2D lidar, 1 3D lidar, 8 ultrasonic sensors, etc. The reference architecture will be later this year roll out.

The Jetson platform also has full NVIDIA software support. To meet the needs of specific use cases, the NVIDIA software platforms have been added: NVIDIA Isaac Sim on Omniverse for robotics, GPU-accelerated SDK Riva for building voice AI applications, AI multi-sensor processing, video, audio and image understanding DeepStream, a streaming analytics toolkit, and Metropolis, an application framework that integrates visual data and AI to improve industry operational efficiency and security, a developer toolset, and an ecosystem of partners.