Nvidia Canvas: Painting With AI Has Never Been More Fun

Nvidia Canvas is a digital art creation tool that uses a combination of deep learning and computer vision to let you create digital artwork by simply drawing or painting on a canvas. Here is the technology behind it, how it is used, and its merits and drawbacks.

The technology

Canvas is built on generative adversarial networks, a class of deep learning model trained to generate new data resembling a given training set. Here the training set is a large collection of digital art images. The GAN learns to generate new images similar to that set, so your rough strokes become finished work.

Alongside the GAN, Canvas uses computer vision to interpret what you draw. Image segmentation algorithms divide the canvas into regions, then deep learning models generate new imagery based on your marks. That is what produces highly detailed and realistic results from crude input.

It also employs style transfer, letting you apply the style of one image to another. Take a piece of artwork and recast it in the style of a different one to produce something new.

Together, GANs, segmentation and style transfer let you create detailed digital artwork without any technical skill, which makes it genuinely powerful for artists and designers.

Use cases

The main use is creating digital artwork: paintings, drawings, animations and 3D models. It also serves digital media for film and video production, game development and other creative fields.

Merits

The main benefit is the ability to produce highly detailed and realistic results. It is also very easy to use and requires no prior knowledge of digital art or programming.

Demerits

At the time of writing it was in beta and available to selected users only. It may not suit all types of digital art, and it is not as versatile as traditional digital art tools.

More at Nvidia Studio.