Video games and research: a two-way relationship?

For the past two years, our aim at the Sci­ence and Video Games research and teach­ing chair has been to facil­it­ate exchanges between three main play­ers: research­ers from all fields of sci­ence, experts from the video game industry and, finally, our stu­dents. The former has known how to use video games to pop­ular­ise sci­ence since the early days of this form of enter­tain­ment. For example, the game Ten­nis for two – the ancest­or of Pong – was developed in 1958 at Brookhaven, a research labor­at­ory near New York. The research­ers’ idea was to demon­strate the cap­ab­il­it­ies of the com­puter to the pub­lic at an open day.

Also, the first gamers were sci­ent­ists from all over the world based in research centres; a situ­ation that las­ted for years before video games became pop­u­lar in the early 1970s. This chair, which relies on a hand­ful of gam­ing pro­fes­sion­als in res­id­ence at Poly­tech­nique, for example, pushes col­lect­ive sci­entif­ic pro­jects that allow some fifty second-year stu­dents to explore the prob­lems asso­ci­ated with video games.

Sci­ence and video games have always had an unbreak­able bond. Their com­mon fron­ti­ers are two­fold: exper­i­ence and tech­no­logy. Sci­ence, like video games, is all about exper­i­ence; for both, the notions of tri­al and error are cent­ral, essen­tial. Moreover, both sci­ence and video games make use of tech­no­lo­gies, often innov­at­ive ones. Examples include vir­tu­al real­ity (VR), aug­men­ted real­ity (AR), net­work tech­no­logy, arti­fi­cial intel­li­gence (AI), com­puter graph­ics, 3D mod­el­ling and more. In 2016, aug­men­ted real­ity was glob­ally pop­ular­ised by a game, Poké­mon Go, which achieved half a bil­lion down­loads only three months after its release. New digit­al tech­no­lo­gies have been used very quickly to pro­duce or enrich games. The lat­ter some­times even serve as illus­trat­ors, if not pop­ular­isers, of these same technologies.

In my opin­ion, video games com­bine at least four import­ant ele­ments that explain this strong link. Firstly, it has an audi­ence, the largest in terms of enter­tain­ment on the plan­et, with over $100bn annu­al rev­en­ue. In fact, almost every­one plays today, thanks to mobile phones. The second aspect is the power of rep­res­ent­a­tion of the mov­ing image, which can also be found in anim­ated films. The third import­ant fea­ture is game mech­an­ics. These mech­an­ics can be inspired by the sci­entif­ic approach, allow­ing people to under­stand an envir­on­ment through experience.

Finally, the fourth asset is the strength of inter­activ­ity in learn­ing. One of the best ways to learn is to put know­ledge into prac­tice rather than to be sub­jec­ted to it. For example, as a teen­ager, I felt that I under­stood some­thing about the his­tory of human civil­isa­tion by play­ing the game Civil­iz­a­tion. It was an ideal first step towards a more ser­i­ous course or con­tent. So, games are fant­ast­ic tools for learn­ing or at least for illus­trat­ing science.

But can video games help research? On the face of it, this enter­tain­ment industry has driv­en devel­op­ment in tech­no­logy of graph­ics com­put­ing, and to a less­er extent, arti­fi­cial intel­li­gence. Com­puter graph­ics because with the advent of 3D, the gam­ing industry has seized on every advance in this tech­no­logy, often before the anim­a­tion industry. A French game like Alone in the Dark, in 1992, was a pion­eer in this field.

AI has seen pro­gress thanks to video games, too. You may remem­ber when Deep Blue, the IBM super­com­puter, man­aged to beat Garry Kas­parov at chess in 1997. Is it con­sidered a video game or not? It’s a ques­tion of defin­i­tion. Today, video games at least advance AI because they rep­res­ent a new chal­lenge. In 2019, AlphaStar, an AI sys­tem designed by Deep­Mind – a sub­si­di­ary of Google – ranked among the top 0.2% of play­ers in the world on the game Star­craft 2. The best human play­ers are able to per­form about 300 actions per minute with infin­ite com­bin­at­or­ics; still a major chal­lenge for an AI.

But in some cases, the play­ers’ intel­li­gence can be used to solve prob­lems that AI can­not. This is the case of the game Foldit, which has been try­ing for sev­er­al years to provide a col­lect­ive answer to unsolved pro­tein fold­ing prob­lems in bio­logy. After solv­ing the struc­ture of an enzyme called M‑PMV in 3D in 2011, the plat­form was used in the fight against Cov­id-19 to make the 200,000 play­ers on the site ‘work’ togeth­er. But just recently, Deep­Mind announced that they had bent this prob­lem thanks to AI.

In fact, while I doubt that there will be much more pro­gress of this kind for the hard sci­ences, there is still a lot of research to be done on the play­er, his brain, his social atti­tudes, etc. When the sub­ject becomes human­ity, there is still, in my opin­ion, a lot to be done using video games and what they can provide in terms of data. The cog­nit­ive, social, and eco­nom­ic sci­ences should take advant­age of this in the years to come, via crowd­sourcing for example.

Finally, we can ask the oppos­ite ques­tion. Can sci­ence inspire game exper­i­ences? This is one of the aspects I believe in the most. There have been thought exper­i­ments in games for a few years now. A play­er with a hel­met on his head is able to play a kind of Space Invaders without a con­trol­ler, just by think­ing. This is an example where a sci­ence, cog­nit­ive sci­ence in this case, would help to pro­duce a new tech­no­logy, in this case the hel­met, which can cap­ture the player’s thoughts and trans­late them into com­mands. If research makes advances on this issue, it will lead to new innov­at­ive games. From these games, we will be able to col­lect the data pro­duced by the play­ers. And ask, for example, how cog­nit­ive pro­cesses and the brain itself work. Once again, sci­ence will be re-fuelled by video games.