Rembrandt Harmenszoon van Rijn (15 July 1606 – 4 October 1669) was a Dutch painter. He is considered an extraordinary artist and his paintings, seeking auction prices of up to $33m and being exhibited in the world’s most prestigious museums, are a some of the most important parts of Dutch and European art history. ING Group is a dutch multinational banking and financial services corporation headquartered in Amsterdam. In 2016, the traditional bank house wanted to create a new way to express its innovative spirit and company culture by sponsoring a project using the latest artificial intelligence technology and combine it with Dutch heritage. The bank roped in the global marketing agency J. Walter Thompson (JWT) to create a 3D printed Rembrandt painting, more than 300 years after his death, leveraging deep learning algorithms and facial recognition software.
Execution of the project & Technology leveraged
ING and Walter Thompson teamed up with Microsoft to deliver this new-age project. A team of 20 data scientists, developers, AI and 3D printing experts worked on the project for 18 months. The team started with a thorough study of the entire collection of Rembrandt’s +300 paintings through high-resolution scans. This resulted in a database of +150GB of data. After that, the team leveraged neural network algorithms to increase the resolution of paintings and improve the image quality for paintings that had been damaged before. The algorithm then examined each and every picture and classified the painting across different parameters e.g. outfit of person in painting, gender of person in painting, gestures, facial expressions, eye color etc. – a total of 67 different features was recorded (if you want to learn more about the actual image recognition algorithm that was used – it is called a Convolutional Neural Network and you can read up more here; it is also used by companies such as Google and Facebook in their image processing (e.g. automated tagging of faces)).
Along with facial features, the data-set was further enriched by information around brush-strokes, canvas quality and height-maps to identify the density of paint on a canvas and the way these densities and thicknesses varied. The team used 3D scans and x-rays to identify the height of different features in a painting (see below an illustration for the process).
After the painting was generated by the algorithm, it was printed through a Canon 3D printer with 13 layers of special paint (see below a video on the 3D printing technique applied by the project team to create the Next Rembrandt).
The final painting, as shown below, was exhibited all over the world through 2016. The campaign, led by JWT, won over 60 advertising awards including the prestigious Cannes Lions. According to Creative Director Bas Korsten, the campaign generated an earned media value of more than $12m with 1.8 billion media impressions, 2,000 newspaper and magazine articles and front pages of multiple newspapers and magazines. Beyond the commercial value, though, the campaign is a true example of the power of most recent artificial intelligence algorithms, imaging technology as well as 3D printing technology. It blurs the boundaries between arts and science in an entirely new way and created a new piece of Dutch heritage empowered by convolutional neural networks.
Challenges and future opportunities
One of the key critiques of this innovation is its sustainability. The Next Rembrandt campaign was successful on its own in providing publicity to ING, Microsoft as well as the Dutch heritage. However, it is yet to be seen whether the agency, JWT, will be able to deploy the technology and insights developed in other projects. It is also not clear who owns the IP on the methodology that was used to create the painting. One way to commercially exploit this technology could be to recreate paintings of other well-known artists and auction or sell them in the open market. Another commercial opportunity could be to create other forms of media using convolutional neural networks e.g. songs of musicians who passed away, poems and novels of famous writers, paintings of other artists. It is without doubt that the Next Rembrandt and the associated coverage makes for a great case study, however, it is not clear whether or not adjacent opportunities will arise and the technology will be used in the future.