Machine learning models: robustness and generalisability
Author
Prof. Volkan Cevher
EPFL
Interview with the principal investigator of this NRP 75 project.
What was the aim of your project?
We investigated three interrelated research trusts: (1) an accurate and scalable prediction framework with neural networks and Langevin dynamics, (2) a flexible and robust decision framework via reinforcement learning, and (3) generalisation and certification of neural networks.
Results of the project?
We developed an optimisation framework to estimate the Lipschitz constant of neural networks, which is key in their generalisation as well as their verification. The approach is based on finding a polynomial certificate via Krivine certificates as well as Lasserre hierarchies. We have also studied regularisation of neural networks with the 1-path-norm and developed computational tools for obtaining numerical solutions. In addition, new tools were derived that quantify learning generalisation.
We considered machine teaching problems where a teacher provides examples to accelerate learning. We studied the settings where the teacher is robust to the diversity of the students in terms of their learning rates. We also extended this problem to the inverse reinforcement learning setting and observed that learning progress can be sped up drastically.
Finally, we developed a new mixed Nash equilibrium framework for robust reinforcement learning. The key idea is to consider a lifted version of the robust reinforcement learning problem and then use the Langevin dynamics developed to sample from the solution distributions. This approach not only increases robustness but also generalisation performance in general.
What are the main messages of the project?
- In the face of Big Data and neural network models, algorithms are subject to grander challenges beyond scalability in order to develop certifiably correct, unbiased, and fair machine learning models.
- We need unified optimisation and representation foundations in how we capture functions via non-linear representations, such as neural networks, how we set-up our learning objectives that govern our fundamental goals, and how we optimise these goals to obtain numerical solutions in scalable fashion.
- Real progress on machine learning requires a coordinated effort based on theoretical and algorithmic foundations that balances, for a given statistical risk, the competing roles of data and representation size, computation, and robustness.
Does your project have any scientific implications?
Thanks to neural networks, faster computation, and massive datasets, machine learning is under increasing pressure to provide automated solutions to even harder real-world tasks with beyond human performance with ever faster response times due to potentially huge technological and societal benefits. Unsurprisingly, the neural network learning formulations present a fundamental challenge to the back-end learning algorithms despite their scalability, in particular due to the existence traps in the non-convex optimisation landscape, such as saddle points, that can prevent algorithms to obtain “good” solutions.
Our research has demonstrated that the non-convex optimisation dogma is false by showing that scalable stochastic optimisation algorithms can avoid traps and rapidly obtain locally optimal solutions. Coupled with the progress in representation learning, such as over-parameterised neural networks, such local solutions can be globally optimal. Unfortunately, we have also proved that the central min-max optimisation problems in machine learning, such as generative adversarial networks and distributionally robust machine learning, contain spurious attractors that do not include any stationary points of the original learning formulation. Indeed, algorithms are subject to a grander challenge, including unavoidable convergence failures, which explain the stagnation in their progress despite the impressive earlier demonstrations.
Does your project have any policy recommendations?
We do need a joint technical workgroup on machine learning theory and optimisation algorithms within Switzerland, if we will increasingly rely on automated decision making from Big Data moving forward. We need new theory and methods for reliable and sustainable way of certifying decisions, making sure that they are unbiased and are fair. This requires an end-to-end expertise in how we generate data, how we set up our learning formulations and models, and how we optimise these formulations.