Museum metaverses promise to redefine the cultural experience

Pre­serving cul­tur­al her­it­age, rais­ing the pro­file of art col­lec­tions and mak­ing immers­ive museum exper­i­ences access­ible to as many people as pos­sible in a ded­ic­ated meta­verse… As the win­ner in 2023 of wave 3 of the PIA4 (Plan d’in­ves­tisse­ment d’avenir) call for pro­jects ‘Digit­isa­tion of her­it­age and archi­tec­ture’, the Métavers du pat­rimoine pro­ject is tack­ling these ambi­tious object­ives. Led by Man­za­lab, the con­sor­ti­um also includes Ecole Poly­tech­nique, Télé­com Sud­Par­is, Mus’X and For­um des Images. The idea is to digit­ise works of art and then insert them into col­lab­or­at­ive and inter­act­ive vir­tu­al worlds. Ulti­mately, the aim is to offer museums the pos­sib­il­ity and the right tools to digit­ise their arte­facts and redefine cul­tur­al experiences.

Europe, and France in par­tic­u­lar, has a rich and well-pre­served museum her­it­age. Digit­ising works of art ensures their pre­ser­va­tion and makes them more access­ible. This is what Tit­us Zaharia and Mari­us Preda, pro­fess­or and lec­turer respect­ively at Télé­com Sud­Par­is, are seek­ing to achieve under the umbrella of the “Métavers du pat­rimoine” pro­ject. The aim is to cre­ate 3D rep­res­ent­a­tions of objects so that they can be visu­al­ised in a vir­tu­al envir­on­ment called a meta­verse. “The idea is to use digit­ised works of art and vir­tu­al worlds to sup­port an exper­i­ence,” explains Mari­us Preda.

The fusion of art and tech­no­logy is open­ing up new pro­spects for museums. While some fear a dema­ter­i­al­isa­tion of the artist­ic exper­i­ence, accord­ing to the experts, it is above all an oppor­tun­ity to broaden the audi­ence and gen­er­ate new forms of engage­ment. Vir­tu­al worlds know no spa­tial bound­ar­ies. In the meta­verse, for example, an exhib­i­tion that is usu­ally restric­ted by the sur­face area alloc­ated to it could tran­scend the phys­ic­al bar­ri­ers of the real world. The cre­ation of innov­at­ive, immers­ive exper­i­ences could also arouse new interest among vis­it­ors. As Mari­us Preda points out, “I don’t get the impres­sion that the young­er gen­er­a­tion is keen to vis­it museums in the tra­di­tion­al way. I see this as an oppor­tun­ity to reach them, as they are already very used to con­sum­ing digit­al con­tent.” Lastly, the museum meta­verse pro­ject pro­motes access­ib­il­ity. Without the need to travel, this new way of exper­i­en­cing exhib­i­tions could become a major edu­ca­tion­al and cul­tur­al lever.

Pho­to­gram­metry is the corner­stone of this pro­cess. Syn­onym­ous with meas­ure­ment by pho­to­graphy, this tech­nique is based on a very old prin­ciple: the cal­cu­la­tion of dis­tances by tri­an­gu­la­tion. This meth­od, which enables large dis­tances to be meas­ured accur­ately, was used to map ter­rit­or­ies in the 17th cen­tury¹. By ima­gin­ing a tri­angle on a piece of land, we can determ­ine the length of the oth­er two sides from the angles. Once the length of the first tri­angle is known, anoth­er tri­angle with a side in com­mon with the first can be cal­cu­lated. To digit­ise objects, sci­ent­ists use a sim­il­ar pro­cess: sev­er­al cam­er­as take images of the object from dif­fer­ent angles. Using the 2D images of the work, the sci­ent­ists are able to cal­cu­late pos­i­tions in three dimen­sions. They find com­mon points between the images cap­tured and cal­cu­late the 3D pos­i­tion of each pixel. They then put these meas­ure­ments togeth­er to recon­struct the three-dimen­sion­al object.

Cap­tur­ing the com­plex­ity of works of art while respect­ing their artist­ic essence is a chal­lenge. Firstly, this tech­nique can only rep­res­ent the sur­face of the object, not what it con­tains. What’s more, the diversity of art forms requires con­stant adapt­a­tion, with each cre­ation requir­ing a spe­cif­ic approach. The mater­i­al of the work is one of the main obstacles. “If it is cap­able of reflect­ing light, then it can be digit­ised”, says Mari­us Preda. This being the case, trans­lu­cent or trans­par­ent objects can­not be digit­ised using this tech­nique. Anoth­er factor is the size of the object. The smal­ler the object, the easi­er it is to cap­ture. When the object is too large, sci­ent­ists use drones to cap­ture the images. This meth­od is used, for example, to digit­ise archi­tec­tur­al works.

The digit­isa­tion of works of art opens up new artist­ic pos­sib­il­it­ies. “Once digit­ised, there are an infin­ite num­ber of ways of rep­res­ent­ing the object. You can play with mul­tiple para­met­ers such as trans­par­ency, col­our or even shape,” explains the lec­turer. But this vast field of pos­sib­il­it­ies raises eth­ic­al ques­tions, par­tic­u­larly when it comes to the modi­fic­a­tion or rep­res­ent­a­tion of ancient cul­tur­al objects that could offend cer­tain communities.

“You have to bear in mind that hav­ing works digit­ised in 3D is not enough to make a meta­verse, there are oth­er aspects that need to be taken into account,” insists Tit­us Zaharia. In the end, although digit­ising works of art involves a num­ber of highly tech­nic­al pro­cesses, there are no major tech­no­lo­gic­al bar­ri­ers to gami­fic­a­tion, cap­tur­ing, mak­ing avail­able and cre­at­ing exper­i­ences. The cur­rent chal­lenge is above all to dis­cov­er the right uses. Tit­us Zaharia asks, “At what point does a user think they are vis­it­ing a vir­tu­al real­ity museum?” 

Accord­ing to the research­ers, these vir­tu­al museums will be col­lect­ive exper­i­ences. Mari­us Preda adds: “I don’t believe in exper­i­en­cing things alone at home. I believe more in online group vis­its, like play­ing video games.” To achieve this, tech­no­lo­gies need to be demo­crat­ised and developed to ensure that they are user-friendly. For example, cur­rent vir­tu­al real­ity head­sets are heavy, uncom­fort­able and expens­ive. Without “good” devices for access­ing metavers, the user exper­i­ence will be compromised.

So, while tech­no­logy opens new doors, innov­a­tion is noth­ing without ‘uses’. This French pro­ject is being run with and for museums. Many museums and cul­tur­al insti­tu­tions are already rep­res­en­ted, either as part­ners in the pro­ject, such as Mus’X and the For­um des Images, or as mem­bers of the steer­ing com­mit­tee, such as the Château de Ver­sailles, the Musée du Quai Branly, the Musée des armées, the Bib­lio­thèque nationale de France, the Cité des sci­ences et de l’in­dus­trie, the Musée d’archéo­lo­gie nationale, the Musée de la Mar­ine, the Pal­ais des beaux-arts de Lille and the Château de Cham­bord. Ulti­mately, the aim is to enable museum staff to carry out 3D record­ing inde­pend­ently, using a plat­form that the research­ers are devel­op­ing. This plat­form will have to under­stand the needs of the insti­tu­tions and adapt to their busi­ness mod­els. If these cul­tur­al meta­verse pro­jects are to be demo­crat­ised, indi­vidu­al museums will need to be able to cre­ate their own con­tent, with a high degree of cus­tom­isa­tion and script­ing of the user experience.

Loraine Odot