The algorithm should itself learn the notion of intuitive physics

Fast prediction algorithms

Author
Professor Marco Zaffalon
Istituto Dalle Molle di Studi sull’Intelligenza Artificiale

Interview with the principal investigator of this NRP75 project.

What was the aim of your project “Hungry to learn: Ultrafast algorithms devour Big Data”?

The usefulness of an algorithm in analysing data is limited by the speed required to process it. Much research in machine learning is thus geared towards combining power and speed, but this has yet to be achieved with Big Data. This project focused on a new approach to create algorithms that deliver both maximum power and speed. They enable the promise of Big Data to be fulfilled.

Results?

The project advanced the state-of-the-art in two families of methods for Gaussian processes regression with Big Data: sparse inducing points methods and local Gaussian processes methods.

Regarding the first family, sparse inducing points methods, the project achieved significant advances in training such approximation methods in the mini-batch setting with stochastic gradient descent techniques. In particular a recursive method for training sparse Gaussian processes was developed that, by exploiting Kalman filter-like updates, reduced the number of parameters to be estimated and increased overall performance. By following a similar line of research another method, based on information filter updates and independence assumptions, provided up to four times computational speed-ups compared to state-of-the-art.

The project also developed a correlated method for local Gaussian processes approximations. This method allows for a precise control on the level of sparsity and locality of the approximation. It is thus easier to tune to specific applications. Moreover, by properly accounting for correlations in a product of expert setting, the proposed method achieves better performance than state-of-the-art approximations.

What are the main messages of the project?

  • It is possible to use Gaussian processes on Big Data and they bring the advantage of prediction uncertainties.
  • There is a unique framework that includes locality and sparsity. This framework can be tuned to the specific problem at hand.
  • Sparse inducing points approximations have an important role in machine learning related applications. Choosing the right training method could be even more important than choosing the right sparse inducing points approximation.

Does your project have any scientific implications?

Our project showed that it is possible to use sound Gaussian processes approximations on Big Data. This means that also Big Data predictions come with an uncertainty quantification, i.e. we know the quality of the model’s predictions.

Big Data is a very vague term. Can you explain to us what Big Data means to you?

In this project “Big Data” took the meaning of datasets so large that challenge the possibility of learning a specific model, Gaussian processes. For large datasets, Gaussian processes bring the important advantage of having a sound quantification of the uncertainty on the predictions. This is a key aspect which was often disregarded in Big Data applications. Nonetheless, for decision making, “knowing the unknown” (i.e. knowing the uncertainty on the prediction) is very important. In our project we focused on improving the methods that allow Gaussian processes to scale to large datasets. Our contributions open the way to the use of a sound Bayesian probabilistic method on large datasets. On a broader societal perspective, the sound quantification of the uncertainties, along with accurate predictions, should improve the decisions that are taken based on such predictions.

About the project

Related links