Museum metaverses promise to redefine the cultural experience

Pre­ser­ving cultu­ral heri­tage, rai­sing the pro­file of art col­lec­tions and making immer­sive museum expe­riences acces­sible to as many people as pos­sible in a dedi­ca­ted meta­verse… As the win­ner in 2023 of wave 3 of the PIA4 (Plan d’in­ves­tis­se­ment d’a­ve­nir) call for pro­jects ‘Digi­ti­sa­tion of heri­tage and archi­tec­ture’, the Méta­vers du patri­moine pro­ject is tack­ling these ambi­tious objec­tives. Led by Man­za­lab, the consor­tium also includes Ecole Poly­tech­nique, Télé­com Sud­Pa­ris, Mus’X and Forum des Images. The idea is to digi­tise works of art and then insert them into col­la­bo­ra­tive and inter­ac­tive vir­tual worlds. Ulti­ma­te­ly, the aim is to offer museums the pos­si­bi­li­ty and the right tools to digi­tise their arte­facts and rede­fine cultu­ral experiences.

Europe, and France in par­ti­cu­lar, has a rich and well-pre­ser­ved museum heri­tage. Digi­ti­sing works of art ensures their pre­ser­va­tion and makes them more acces­sible. This is what Titus Zaha­ria and Marius Pre­da, pro­fes­sor and lec­tu­rer res­pec­ti­ve­ly at Télé­com Sud­Pa­ris, are see­king to achieve under the umbrel­la of the “Méta­vers du patri­moine” pro­ject. The aim is to create 3D repre­sen­ta­tions of objects so that they can be visua­li­sed in a vir­tual envi­ron­ment cal­led a meta­verse. “The idea is to use digi­ti­sed works of art and vir­tual worlds to sup­port an expe­rience,” explains Marius Preda.

The fusion of art and tech­no­lo­gy is ope­ning up new pros­pects for museums. While some fear a dema­te­ria­li­sa­tion of the artis­tic expe­rience, accor­ding to the experts, it is above all an oppor­tu­ni­ty to broa­den the audience and gene­rate new forms of enga­ge­ment. Vir­tual worlds know no spa­tial boun­da­ries. In the meta­verse, for example, an exhi­bi­tion that is usual­ly res­tric­ted by the sur­face area allo­ca­ted to it could trans­cend the phy­si­cal bar­riers of the real world. The crea­tion of inno­va­tive, immer­sive expe­riences could also arouse new inter­est among visi­tors. As Marius Pre­da points out, “I don’t get the impres­sion that the youn­ger gene­ra­tion is keen to visit museums in the tra­di­tio­nal way. I see this as an oppor­tu­ni­ty to reach them, as they are alrea­dy very used to consu­ming digi­tal content.” Last­ly, the museum meta­verse pro­ject pro­motes acces­si­bi­li­ty. Without the need to tra­vel, this new way of expe­rien­cing exhi­bi­tions could become a major edu­ca­tio­nal and cultu­ral lever.

Pho­to­gram­me­try is the cor­ners­tone of this pro­cess. Syno­ny­mous with mea­su­re­ment by pho­to­gra­phy, this tech­nique is based on a very old prin­ciple : the cal­cu­la­tion of dis­tances by tri­an­gu­la­tion. This method, which enables large dis­tances to be mea­su­red accu­ra­te­ly, was used to map ter­ri­to­ries in the 17th cen­tu­ry¹. By ima­gi­ning a tri­angle on a piece of land, we can deter­mine the length of the other two sides from the angles. Once the length of the first tri­angle is known, ano­ther tri­angle with a side in com­mon with the first can be cal­cu­la­ted. To digi­tise objects, scien­tists use a simi­lar pro­cess : seve­ral came­ras take images of the object from dif­ferent angles. Using the 2D images of the work, the scien­tists are able to cal­cu­late posi­tions in three dimen­sions. They find com­mon points bet­ween the images cap­tu­red and cal­cu­late the 3D posi­tion of each pixel. They then put these mea­su­re­ments toge­ther to recons­truct the three-dimen­sio­nal object.

Cap­tu­ring the com­plexi­ty of works of art while res­pec­ting their artis­tic essence is a chal­lenge. First­ly, this tech­nique can only represent the sur­face of the object, not what it contains. What’s more, the diver­si­ty of art forms requires constant adap­ta­tion, with each crea­tion requi­ring a spe­ci­fic approach. The mate­rial of the work is one of the main obs­tacles. “If it is capable of reflec­ting light, then it can be digi­ti­sed”, says Marius Pre­da. This being the case, trans­lucent or trans­pa­rent objects can­not be digi­ti­sed using this tech­nique. Ano­ther fac­tor is the size of the object. The smal­ler the object, the easier it is to cap­ture. When the object is too large, scien­tists use drones to cap­ture the images. This method is used, for example, to digi­tise archi­tec­tu­ral works.

The digi­ti­sa­tion of works of art opens up new artis­tic pos­si­bi­li­ties. “Once digi­ti­sed, there are an infi­nite num­ber of ways of repre­sen­ting the object. You can play with mul­tiple para­me­ters such as trans­pa­ren­cy, colour or even shape,” explains the lec­tu­rer. But this vast field of pos­si­bi­li­ties raises ethi­cal ques­tions, par­ti­cu­lar­ly when it comes to the modi­fi­ca­tion or repre­sen­ta­tion of ancient cultu­ral objects that could offend cer­tain communities.

“You have to bear in mind that having works digi­ti­sed in 3D is not enough to make a meta­verse, there are other aspects that need to be taken into account,” insists Titus Zaha­ria. In the end, although digi­ti­sing works of art involves a num­ber of high­ly tech­ni­cal pro­cesses, there are no major tech­no­lo­gi­cal bar­riers to gami­fi­ca­tion, cap­tu­ring, making avai­lable and crea­ting expe­riences. The cur­rent chal­lenge is above all to dis­co­ver the right uses. Titus Zaha­ria asks, “At what point does a user think they are visi­ting a vir­tual rea­li­ty museum?” 

Accor­ding to the resear­chers, these vir­tual museums will be col­lec­tive expe­riences. Marius Pre­da adds : “I don’t believe in expe­rien­cing things alone at home. I believe more in online group visits, like playing video games.” To achieve this, tech­no­lo­gies need to be demo­cra­ti­sed and deve­lo­ped to ensure that they are user-friend­ly. For example, cur­rent vir­tual rea­li­ty head­sets are hea­vy, uncom­for­table and expen­sive. Without “good” devices for acces­sing meta­vers, the user expe­rience will be compromised.

So, while tech­no­lo­gy opens new doors, inno­va­tion is nothing without ‘uses’. This French pro­ject is being run with and for museums. Many museums and cultu­ral ins­ti­tu­tions are alrea­dy repre­sen­ted, either as part­ners in the pro­ject, such as Mus’X and the Forum des Images, or as mem­bers of the stee­ring com­mit­tee, such as the Châ­teau de Ver­sailles, the Musée du Quai Bran­ly, the Musée des armées, the Biblio­thèque natio­nale de France, the Cité des sciences et de l’in­dus­trie, the Musée d’ar­chéo­lo­gie natio­nale, the Musée de la Marine, the Palais des beaux-arts de Lille and the Châ­teau de Cham­bord. Ulti­ma­te­ly, the aim is to enable museum staff to car­ry out 3D recor­ding inde­pen­dent­ly, using a plat­form that the resear­chers are deve­lo­ping. This plat­form will have to unders­tand the needs of the ins­ti­tu­tions and adapt to their busi­ness models. If these cultu­ral meta­verse pro­jects are to be demo­cra­ti­sed, indi­vi­dual museums will need to be able to create their own content, with a high degree of cus­to­mi­sa­tion and scrip­ting of the user experience.

Loraine Odot