Using paint, a blank canvas, and brushes will always have a charm, no matter the century we live in. Technology, however, has opened up a large space for artists to experiment with different software, virtual art tools, and imaging techniques. The sky’s the limit when it comes to art and technology which makes the intersection between the two particularly interesting.
The use of 3D effects in visual arts is among the advanced techniques for the creative expression of ideas. Projection mapping is a technique that uses various irregularly shaped objects such as buildings or interiors for a canvas for video projection. In order to turn objects into display surfaces, studios use special software to spatially map two or three-dimensional figures on the virtual program. By doing so, projection specialists and programmers can create a digital twin of the real environment and use it to calculate how to best project images or videos.
One of the projection mapping pioneers in Europe is the Bulgarian MP-STUDIO which specializes in 3D projection mapping and visual effects. Founded by Marin Petkov in 2008, the company has completed more than 500 projects and has won multiple awards and international recognitions. Some of their biggest recognitions are winning the 2015 and 2018 Festivals of Lights in Berlin. Today, MP-STUDIO is the main partner of the Festivals of Lights brand and Petkov is the Art director of the show.
At the beginning of July this year, MP-STUDIO created a 360-degree 3D projection mapping of the painting of the two greatest Dutch painters, Rembrandt and Vincent Van Gogh. The project “Vincent Meets Rembrandt: The Untold Story” is located in the Noorderkerk church in Amsterdam and will stay there for 5 years for visitors to experience. Our team met with Petkov to find out more about how technologies impact the way we do art, and how MP-STUDIO mixes creativity and talent for painting with machine learning and mathematical projections.
Pioneering in projection mapping
While studying Informatics and Multimedia in Germany, Petkov got inspired to switch his focus to visual effects when he watched the action box-office movie Blade. “Back then the technology that was used in art was limited to visual effects and motion graphics, projection mapping did not exist yet,” Petkov shares. After graduating, he came back to Bulgaria with the vision to work on innovation in visual arts, but found out that the local market was not mature enough. Therefore, in the beginning, he concentrated on doing projects abroad, and participating in visual arts competitions.
“It was my goal to first complete some project and make a name abroad as I knew that the market in Bulgaria was not ready and needed some time. After we did our first shows in Bulgaria, projection mapping became an integral part of making events more interesting and attractive. Not just on buildings but also for interiors and stage performances. As we were among the projection mapping pioneers in Europe, we were able to build a network of partners who opened a lot of opportunities for us in the international markets.
Between brushes and algorithms
Petkov explains that the process of 3D projection mapping on buildings and interiors begins with scanning and photogrammetry. During this initial phase, the team of MP-STUDIO uses multiple photos to make a 3D model of the building and gather all the data they need to make a digital twin of the environment. When developing the digital twin of the buildings, it is of great importance to account for all small irregularities and map them on the digital software. This process ensures that the projected image or video will fit with the irregularities of the objects.
Next, in the production phase, MP-STUDIO uses ML, AI, and custom algorithms. Petkov gives an example of the work they did around the Van Gogh project. “We used various technologies to replicate and complete some of the paintings and make them suitable for 360 degrees projection. We had to develop the continuation of the masterpieces without changing them in any way. In order to be as accurate as possible in our projections, we had to examine every detail of the paintings, which meant that we paid a lot of attention to the brushes used in the paintings. A really interesting experience for the team was exploring how Van Gogh played with different techniques to visualize light the Starry Night painting,” Petkov shares.
Read more: Leading with empathy to fight social injustice with Iva Gumnishka from Humans in The Loop
To make the image fit the irregular canvas on which it is projected either scans the building or uses photogrammetrical functions. Additionally, they calculate how many projectors will they need for the projection mapping, and how the natural lightning affects the projected image. Storytelling is also a key element of the experience of their shows and that is why MP-STUDIO often adds voiceover to accelerate the effect.
A one-stop-shop for visual art and tech
From the design of the creative and writing the screenplay, to the development of interactive installations and custom applications, MP-STUDIO covers the full spectrum of visual art technologies. The most popular service that the company offers 3D building projection mapping which can be done on cultural and historic sites, as well as sports and cultural events and for corporate brand advertising. Additionally, MP-STUDIO does interactive projection shows and specializes in 4D experiences by incorporating additional elements and technologies such as laser and video mapping, light mapping, and water projection.
Last year when the pandemic hit, MP-STUDIO developed a platform for virtual events which generates a 3D environment in real-time and allows the audience to interact with the speakers. So far they have done over 40 events on the platform, including the annual strategic conference of Coca-Cola HBC Bulgaria, and the DigitalK conference of LAUNCHub Ventures and Economedia, and continue developing it with new functionalities.
