06/09/2019

The Mathematics of Artificial Intelligence

Imagem: Freepik

Reproduction from the IMPA Science & Mathematics blog, from O Globo, coordinated by Claudio Landim.

André Carlos Ponce de Leon Ferreira de Carvalho – Vice-Director of the Institute of Mathematical and Computer Sciences of the University of São Paulo (ICMC-USP)

Not a day goes by without media outlets bringing us news about yet another application of Artificial Intelligence (AI) affecting our daily lives. Many of these news items bring surprises, hopes, and fears. In general, the benefits and risks are presented clearly. What is not clear is that behind the major applications of AI, there is always a good deal of mathematics. Adapting the phrase attributed to Isaac Newton, "If I have seen further it is by standing on the shoulders of giants," AI has only reached where it is today because it relies on various sub-areas of mathematics.

In this text, I will focus on an application where AI has been successful: Data Science, currently focusing on its mathematical aspects. Reflecting the growing demand for professionals in this area, as has happened in similar situations in different fields, the number of professionals from diverse backgrounds seeking to work as Data Scientists is increasing.

A similar process occurred in the field of Computer Science, when demand exceeded supply, and there were not enough training courses with sufficient places to meet market needs. In several sub-areas of Computer Science, this demand did not require a background in mathematics. In Data Science, the situation is different.

In this field, the required training is different; a strong understanding of mathematics is essential. Although many of these professionals come from areas that provide a good mathematical background, the mathematics they learn may differ from that needed to work in Data Science.

This is because many of the techniques used in Data Science have a strong mathematical foundation. To use them correctly and efficiently, it's important to know how they work internally. Without understanding why and how these techniques work, their choice and use end up occurring through simple intuition or based on previous experience. In these cases, the chance of things going wrong is very high. "Going wrong" is not limited to not working as expected: depending on the application area, it can cause economic and social damage. As an example, imagine a computing tool that predicts medical diagnoses based on Data Science. An incorrect diagnosis can lead to serious health problems, including death. In some countries, medical diagnostic models based on Data Science are already legally permitted. To minimize damage, it is required that the models generated for decision-making be easily interpretable.

Data Science techniques are generally applied to datasets represented by matrices. Each row of these matrices corresponds to an object in the set. In the previous example, a dataset could be a set of patient records from a medical clinic. In the matrix representing this set, each row contains the clinical data of a patient. When the number of columns in the matrix is very large compared to the number of rows, we may have problems modeling the data. This is known as the "curse of dimensionality." This can be minimized by Analytic Geometry, which offers several mathematical tools to analyze and reduce the dimensionality of the data.

A crucial step in Data Science is data exploration, which identifies problems in the data and properties that can be useful in subsequent steps. Many of the relevant properties are obtained through the application of statistical techniques. Statistics are also important for planning a valid and effective way to collect data, and the correct sequence of steps to extract relevant information from the data. Furthermore, data is generally produced according to a probability distribution. Statistical techniques also allow us to estimate the probability distribution that generated the data, which is important for various analysis techniques, and to define how statistically significant the results of an experiment are. Many of the generated models should indicate the probability of something occurring. A good understanding of probability is essential for anyone who wants to work not only in Data Science, but also in AI.

To read the full text, visit the newspaper's website.