Recently, NVIDIA announced the launch of Dlss 2.0, the first major update to its proprietary AI-based upscaling technology used in the RTX GPUs. While it’s a big step up from DLSS 1.0, the important part is that it leverages temporal motion vectors and jitter offsets. That’s similar to how PlayStation 4’s Checkerboard Rendering works. The core distinction here is that the former utilizes a complex neural network to fill in the gaps and make the images sharper. So, where exactly do the two upscaling technologies stand with respect to picture quality?

While we don’t have an apples to apples comparison due to obvious reasons, we can explain the similarities and differences between DLSS 2.0 and CBR. After that, we’ll substitute the latter with a temporal upscaling alternative, and put it side by side against DLSS. After all, it is a refined version of the same.

PS4 Checkerboard Rendering

Let’s have a look at how PS4’s checkerboard rendering upscales images:

This is line-interpolation, the most popular interpolation technique used in upscaling algorithms. Here, alternate columns of pixels are rendered, effectively rendering the image at half-resolution. As you can see, the resultant interpolated image is blurry and plagued by aliasing. Now, have a look at checkerboard interpolation:

CBR renders the diagonal pixels in a frame, with alternate frames consisting of a single checkerboard-pattern samples.

Just a simple switch in the interpolation technique yields vastly superior image quality. There’s less aliasing, more detail and a clearer image overall.

The drawback of checkerboard rendering is that it can lead to artifacting, as you can see below:

There are certain filters that can be implemented to tackle this, but they often tend to reduce the sharpness, further reducing the LOD of the upscaled image. So you’re basically trading one thing for another: sharpness or smoothness.

Then there’s the matter of filling in the blanks in the individual frames. While Sony’s official slides made it look very simple: plugging the n and n+1 frame, it’s a lot more complicated:

The missing pixels P and Q are calculated using a series of calculations:

Each odd pixel uses data from its preceding even frame, and that obtained pixel color is blended with the neighboring color samples of A, B, E and F pixels. To ensure maximum accuracy, a couple of more weights are also included. You can see the entire algorithm here:

All this takes around 1.4ms. In comparison rendering a frame at native resolution takes as much 10ms.

NVIDIA DLSS 2.0: Improved Quality and Temporal Vectors

Enter NVIDIA DLSS 2.0

DLSS 2.0 introduces some major changes in the upscaling algorithm. You can read the nitty-gritty details in the above link, but here’s the thin and thick of it: Unlike DLSS 1.0, it uses a single neural network for every game and resolution, and adds temporal vectors to improve the sharpness and accuracy.

The main advantage of NVIDIA’s DLSS 2.0 technology is its powerful neural network that has already analyzed and upscaled tens of thousands of high-resolution images (64x SS). Combining an intelligent upscaling algorithm that’s ever-improving with temporal data across various frames results in images that are as good or even better than the native target.

The basic low-resolution shaded geometry with jitter offsets and motion vectors from previous frames (same as CBR) are consumed. The resulting high-resolution image is reconstructed by the neural network, with guided data from neighboring frames including motion vectors and the jitter offsets.

This hybrid upscaling algorithm helps offset the drawbacks of temporal filters (loss of detail and clarity) and AI-based scaling algorithms (hallucinations or invented details).

You can see the results below:

Images natively rendered at 540p using DLSS 2.0 actually look better than 1080p shots, and are a major step up from the exiting TAAU upscaling algorithms such as checkerboard rendering.

The most notable differences are with respect to the sharpness and clarity. While temporally rendered images loose a significant chunk of texture detail, DLSS 2.0 not only manages to preserve it, but also offsets the blur induced by TAA.

Control Gets NVIDIA DLSS 2.0; Offers Better Visual Quality than Native

How many developers actually adopt DLSS 2.0 is what will actually matter in the end. With AMD working on its own AI-based upscaling tech for Navi 2x and next-gen consoles, one thing is for certain. The role of neural networks in games for both improving the image quality and gameplay will drastically increase in the coming years.

The post NVIDIA DLSS 2.0 vs PS4’s Checkerboard Rendering: Comparing RTX Upscaling with Console Technology appeared first on Hardware Times.