How and why should we explain the decisions made by artificial intelligence (AI) algorithms?
The need for explainability is not new! The question was already being asked as far back as ancient times, even if back then it was from a philosophical point of view. It was later posed in a formal way at the end of the 19th century, particularly since the work of Charles Peirce. This American philosopher and theorist introduced abductive reasoning, i.e. the search for explanations. Many of the methods used in symbolic AI, which are based on knowledge modelling with approaches such as logic, symbolic learning, etc., are said to be ‘inherently explainable’, because the sequence of reasoning that leads to a decision is identified. But this is only partially true, because if the problem posed becomes too large, with a large number of logical formulas, very complex decision trees, and very numerous association rules, explanation becomes difficult.
The question of explainability is all the more relevant today as the second paradigm of AI, statistical approaches to AI, has been at the forefront in recent years. While symbolic AI is based on rules and reproduces human reasoning, statistical approaches to AI rely on statistical learning methods, in particular artificial neural networks that are trained on large volumes of data. These approaches are part of what is known as machine learning (ML), including deep learning (DL). It is very difficult to extract and express the rules of what neural networks do, which begin with the data.
How can an AI decision be explained?
First of all, it is necessary to define what to explain, for whom, how and why. The choice of explainability tools or methods depends on the answer to these questions. For neural networks, it is possible to answer them at the level of the data used, at the level of the operation of the network itself or at the level of the result produced. For the operation, one may ask whether it is necessary to explain. Take aspirin for example, for a long time it was used without anyone knowing how it worked. When it the way it worked was finally understood, it was used to develop new things, without changing the way aspirin itself was used. In the same way, you can drive a car without understanding the engine but with a level of knowledge that is sufficient to use a car well.
At the level of the result, the explanation may need to go through intermediate steps to explain the final result. For example, I work with radiologists on measuring the thickness of the corpus callosum in premature babies. The radiologists wanted to know where the results came from, which region was recognised in the image, where the measurements were made, to understand what contributed to the decision and explain the final result. These steps were necessary for them to have confidence in the tool.
An algorithm is expected to be neutral, but nothing is ever neutral! The doctor triggers an imaging test for his patient because he is looking for something that he can identify in this image, he has an intention. This introduces biases, which are not statistical, but cognitive, of framing, confirmation, complacency, etc. These same biases are found in the face of images that have been taken. We are faced with these same biases when it comes to the results produced by an algorithm. Furthermore, we should not forget that we trust the algorithm more if it shows us what we are looking for. Another factor to consider is the cost of an error, which can be very different depending on whether or not anything has been detected. Radiologists generally prefer to have a higher number of false positives (since other examinations will always confirm or invalidate what has been detected) than false negatives. It is when the algorithm does not detect anything that it must not be mistaken, even if the doctors always verify the results visually.
Explainability therefore varies according to the user and how an algorithm is used?
Explanation is a process of conversation, of communication. We adapt the level of explanation according to the person we are talking to. To stay within the medical framework, let’s take the example of an image showing a tumour. The doctor will explain this image and the tumour differently depending on whether he is talking to his staff, to students, to an audience in a conference or to his patient. This is why doctors do not want the results from algorithms to be made part of the patient’s records before having had a chance to check them themselves.
We also need to ask ourselves why we want to explain. Is it to justify, to control the functioning of an algorithm, to discover scientific knowledge, a phenomenon? The objectives vary and will therefore require different tools. The stakes also differ, there are issues of trust, ethics, responsibility, and possibly economic issues.
Why is the need for explicability stronger at the moment?
This is mainly due to the increasing use of deep neural networks, which have millions of parameters and are extremely complex. There is a lot of reliance on data in the hope that increasing the volumes used will help improve the results. This being said, there is a lot of domain knowledge that could be used. This is what hybrid AI proposes to do, combining several approaches to AI. It combines knowledge and data, symbolic AI and neural networks, logic and learning. Personally, I’m a big believer in this. But whatever the approach, the role of the human being remains paramount, and the decisions made by an algorithm will always have to be justified.
