07/08/2018

Machine learning can reveal the workings of the mind.

Computer scientist Sanjeev Arora believes that more efficient pattern discovery algorithms in unstructured data can help uncover how the mind works. In his plenary session on Tuesday (7), he said that machine learning techniques open up space for "a new kind of science".

These algorithms learn from experience and interaction with data. The larger the volume of data, the more patterns they discover. In doing so, they mimic how human intelligence works, in an increasingly sophisticated way. According to him, the use of machine learning can help reveal unknown aspects of how the mind works.

While in the past it was necessary to teach the rules of grammar to an algorithm that translates texts, the increasing availability of digital data for analysis and processing power allows algorithms to discover the elements that are linked to each other using predictions made from the frequency with which, for example, one word appears after another in a large set of texts. In a way, this is similar to how babies learn to speak.

The example Arora used to demonstrate the mathematics of machine learning was how to determine, from vocabulary, whether a movie review available on Amazon is positive or negative, predicting the numerical rating to create a kind of "law of movie review ratings".

The frequency of words in positive and negative reviews allows one to calculate a correlation between each word and each "sentiment" involved in the analysis. It's difficult to do this through computation, but the human mind already does it instinctively. "In the first review you read, you already know what the sentiment is," he says.

However, algorithms only understand the result passively, since there is no mathematical model of the language.

When applying machine learning to a complex dataset like natural language, you need to create and optimize a loss function from a test set (usually 80% of your data) and test its effectiveness with new data (usually the remaining 20%) before you have an algorithm that applies to the world at large.

The reduction of loss in training data is achieved through the gradient method, which seeks the minima of a function. At each step, it improves the data. Through backpropagation, a result is generated that is "non-linear enough to express many things, but linear enough to be computable".

In some cases, unsupervised learning is used, where the computer calculates the probabilities involved in the data set on its own. This is a "black box" algorithm, making it difficult to detail the decision-making process and lacking transparency about which variables influence the prediction. One of Arora's research topics is the analysis of black boxes, through small steps.

His full presentation can be downloaded at this link . For those interested in the topic, Arora recommends watching Michael Jordan's plenary session on the 8th (Wednesday).