ChatGPT, Midjourney: everything you need to know about generative AI

It’s all the talk these days, and Chat­G­PT is arriv­ing in our soci­et­ies like a ver­it­able revolu­tion. So, it is hardly sur­pris­ing that, giv­en the wide-ran­ging applic­a­tions of these tools, their arrival is fuel­ling so much debate. But do we really know how this AI works?

A gen­er­at­ive AI can gen­er­ate writ­ten, visu­al, or aud­ible con­tent by ingest­ing con­tent. By giv­ing it indic­a­tions as input, the AI can cre­ate as out­put any con­tent that cor­res­ponds to the indic­a­tions inges­ted. “Here, we’re look­ing to gen­er­ate ori­gin­al con­tent,” explains Éric Mou­lines, pro­fess­or of stat­ic learn­ing at École Poly­tech­nique (IP Par­is). “This ori­gin­al con­tent will be obtained by gen­er­al­ising the inform­a­tion seen dur­ing learning”.

There are cur­rently two main types of AI mod­el. GPTs (Gen­er­at­ive Pre-trained Trans­formers), such as Chat­G­PT, and Dif­fu­sion Mod­els. “By giv­ing it text as input, the AI will be able to under­stand the con­text through a mech­an­ism called atten­tion,” adds Hatim Bourfone, AI research engin­eer at IDRIS (CNRS). “Its out­put will there­fore be a list of all the words in the dic­tion­ary that it knows [learned dur­ing train­ing phase] on which it will have placed a prob­ab­il­ity”. Depend­ing on the data­base it has trained on, the tool can be pro­grammed for vari­ous functions.

Bloom, for example, the AI developed by the team of which Hatim Bourfoune is a mem­ber at IDRIS, is a tool that helps research­ers to express them­selves in sev­er­al lan­guages. “The primary aim of the Bloom mod­el,” adds Pierre Cor­nette, also a mem­ber of the IDRIS team, “is to learn a lan­guage. To do this, we give them a whole bunch of texts to ingest, ask­ing them to pre­dict the next word in the giv­en text, and we cor­rect them if they get it wrong”. 

“The first gen­er­at­ive AI mod­els are not even 10 years old,” explains Éric Mou­lines. “The first revolu­tion in this field was the arrival of trans­formers – a tech­no­logy per­fect­ing this atten­tion mech­an­ism – in 2017. Four years later, we already have com­mer­cial products. So, there has been con­sid­er­able accel­er­a­tion, much faster than on any oth­er Deep Learn­ing mod­el.” Mod­els like Chat­G­PT are there­fore still very new, and there are still many things that can, or must, be improved.

The ques­tion of the reli­ab­il­ity of the answers giv­en is still not cer­tain: “Chat­G­PT is not famil­i­ar with the notion of reli­ab­il­ity”, admits the pro­fess­or. “This type of AI is incap­able of assess­ing the vera­city of the answers it gives.” This leaves room for an eas­ily observ­able phe­nomen­on known as ‘hal­lu­cin­a­tions’. “It is pos­sible [for Chat­G­PT] to gen­er­ate con­tent that seems plaus­ible, but is rig­or­ously false,” he adds. “It uses com­pletely prob­ab­il­ist­ic reas­on­ing to gen­er­ate sequences of words. Depend­ing on the con­text, it will gen­er­ate strings of words that seem the most likely.”

Apart from its abil­ity to invent book titles, oth­er lim­it­a­tions should be borne in mind when using it. By apply­ing Deep Learn­ing meth­ods, these AIs go through a train­ing phase dur­ing which they ingest a quant­ity of exist­ing texts. In this way, they will incor­por­ate the biases of this data­base into their learn­ing. Geo­pol­it­ic­al ques­tions are a good example of this. “If you ask it geo­pol­it­ic­al ques­tions, Chat­G­PT will essen­tially reflect the West­ern world,” says Éric Mou­lines. “If we show the answers giv­en to a Chinese per­son, he will cer­tainly not agree with what is said about the sov­er­eignty of such and such a coun­try over a giv­en territory.”

Each mod­el will there­fore be able to gen­er­ate con­tent accord­ing to the data­base it has been trained on. This is per­haps where the magic of this tech­no­logy lies, because, know­ing this, a myri­ad of applic­a­tions can be cre­ated. “A good ana­logy for this tech­no­logy would be that of an engine,” says Pierre Cor­nette. “You could have a very power­ful engine, but it can be used for either a tract­or or a racing car.” For example, Chat­G­PT is a racing car, and its engine is GPT‑4. “The advant­age is that the tech­no­lo­gies are con­cen­trated in what is the engine,” he con­tin­ues, “and you don’t need to under­stand how it works to use the race car.”

Bloom is an example of anoth­er use for this type of mod­el: “A year ago, Bloom was one of the only mod­els that was com­pletely open to research,” insists Hatim Bourfoune. In oth­er words, any­one could down­load the mod­el and use it for their own research. Trained with a data­base of vari­ous sci­entif­ic art­icles in many lan­guages, this mod­el can be extremely use­ful for sci­entif­ic research. Pierre Cor­nette adds: “There is also anoth­er Big­code pro­ject, run by the same people, which pro­motes a mod­el spe­cial­ising in com­puter code. We ask it for a func­tion, simply describ­ing its action, and it can write it for us in the desired language.”

The pop­ular­ity of Chat­G­PT shows just how import­ant it is for the gen­er­al pub­lic. Bing has also integ­rated it into its search engine with a view to com­pet­ing with Google. This integ­ra­tion makes it pos­sible to counter one of the lim­it­a­tions of this tech­no­logy: the reli­ab­il­ity of the answers giv­en. By giv­ing the sources used to com­pile its response, the search engine enables us to under­stand and veri­fy them bet­ter. Even more recently, Adobe has integ­rated a gen­er­at­ive AI mod­el into vari­ous soft­ware applic­a­tions in its suite (such as Pho­toshop and Illus­trat­or), reveal­ing yet anoth­er impress­ive applic­a­tion of this technology.

All this can only mean an excit­ing future for this innov­a­tion. How­ever, the range of applic­a­tions raises ques­tions about its pos­sible uses. “As with all tools, there can be mali­cious uses,” admits Hatim Bourfoune. “That’s why com­pan­ies like OpenAI put up dif­fer­ent secur­ity bar­ri­ers.” Today, many of the ques­tions put to Chat­G­PT remain unanswered, because the AI believes that they viol­ate its con­tent policy.

Even so, this tech­no­logy is still in its infancy. “That’s the prin­ciple of research – we’re still at ground zero,” says Éric Mou­lines. “It’s amaz­ing that it even works.” There are still many loop­holes to be filled, par­tic­u­larly from a leg­al point of view. As explained, the con­tent gen­er­ated by these tools will be built using an exist­ing data­base. The AI will there­fore “copy” exist­ing texts or works without cit­ing their ori­gin­al author. “This poses a major prob­lem,” he con­tin­ues, “because the rights hold­ers of the con­tent used to gen­er­ate these new images [or texts] are not respected.”

Des­pite its vari­ous lim­it­a­tions, the poten­tial remains enorm­ous: “What excites me … is that the pro­gress to be made is enorm­ous,” adds the pro­fess­or. “But the trend and the deriv­at­ives are enorm­ous. It’s hap­pen­ing very quickly and there’s very excit­ing com­pet­i­tion in these sub­jects.” Speak­ing of deriv­at­ives, Bloom illus­trates this per­fectly. Use­ful for research, it is also a lin­guist­ic tool that could make it pos­sible to save dead lan­guages, but also to trans­late sci­entif­ic texts into less­er-spoken lan­guages to facil­it­ate the dis­sem­in­a­tion of research.

How­ever, its “excit­ing” future may be hampered by its con­sid­er­able car­bon impact. “These mod­els require a lot of memory, because they need to store a huge amount of data,” explains Éric Mou­lines. “Today, we estim­ate that OpenAI con­sumes as much memory as the grid in a coun­try like Bel­gi­um.” This is the prob­lem that will surely be the most com­plic­ated to solve.

Pablo Andres