Aim

Co-design AI learning algorithms and learning theoretic guarantees so to provide deep learning systems with strong empirical performance as well as theoretical guarantees.

Objectives

1. Develop parameter-light learning algorithms based on contemporary foundation models.
2. Develop learning theoretic guarantees for model performance during inference.
3. Extend these learning algorithms and guarantees to the non-IID case where the deployment environment is not constrained to be statistically identical to the training environment.
4. Extend these capabilities to different modalities (imagery, text, control) and tasks (eg: structured output, beyond recognition)

Description

AI methods typically either have theoretical guarantees but poor empirical performance, or strong empirical performance with no guarantees. This is especially the case for real-world non-IID (“domain generalisation”) scenarios, and structured output tasks. The advent of large-scale neural foundation models (FM) has improved empirical model performance, but appears at first glance to make theoretical guarantees even harder. However, recent approaches to downstream learning based on FMs and with recent learning theoretic techniques suggest that careful co-design of algorithms and theory may enable breakthroughs in guaranteeable AI performance. This could have substantial implications for the ability to actually deploy AI in the defence and security sectors.

References

[1] Amortized Invariance Learning for Contrastive Self-Supervision, Chavhan et al, International Conference on Learning Representations (ICLR 2023).

[2] On the Limitations of General Purpose Domain Generalisation Methods, Gouk et al, arXiv, 2024