Aim

To demonstrate a hybrid-AI-based, intelligent vision pipeline that can recognise objects from their properties in a robust and generalisable way.

Objectives

1. Stage 1: Build an efficient “front-end” that can recognise the rough location and ID of an array of objects (including sub-objects).
2. Stage 2: Build a “back-end” that allows the semantic composition of objects recognised from stage 1 into “compound objects”. For example, the system in stage 1 might recognise a couple of “eyes”, and a “mouth” object in the image, but fail to recognise the full “face” due to an occlusion over the “nose” area. The back-end should be able to achieve this instead.
3. Stage 3: Demonstrate the pipeline with a “natural” dataset (as opposed to the toy datasets used to prove the concept in stages 1 & 2).
4. Stage 4 (stretch goal): Propose an efficient hardware implementation for the pipeline.

Description

The project seeks to build a hybrid-AI pipeline combining the strengths of machine-learning (ML) and symbolic-AI (SY). The ML part will form the “front-end” tasked to recognise “what is where”. It may use frequency-domain pre-processing and infer localisation either exactly (like in image segmentation) or approximately (bounding box or similar) as convenient. The SY part will explicitly connect the right combinations of “sub-objects” into a database, from which a “parent object” can be deduced even if it is not recognised as a whole unit (see “stage 2” above). This system should then work as a unit to provide robust object recognition in a wide range of scenarios. Industry-provided datasets can be used to prove the concepts and more “natural” data-sets can be used to take the concept further. Finally, all should be hardware-implementable within reasonable budget.