Artificial intelligence (AI) is embedded in our daily lives. When we use social media, or our smartphones, our self-driving vehicles rely on them, in healthcare, and even when we use a search engine, we’re using the power of AI. All these technologies have made our lives easier and are constantly evolving.

Professor Ajmal Mian, at the UWA School of Physics, Maths and Computing is known internationally for his research in 3D computer vision, AI and machine learning. He and his team remain at the forefront of discovering novel algorithms and better model learning and training techniques and collaborate with multiple disciplines to find innovative solutions to meet their needs. Their algorithms are published on GitHub, an open-source software development platform which allows users to apply the research to their own field and gives the team freedom to be creative in their discoveries.

Professor Mian is a strong advocate of leading the AI boom and for the establishment of an AI centre in Australia to grow our Postgraduate talent as well as attract great minds from elsewhere.

“Artificial intelligence is not going to take the jobs of humans, artificial intelligence is going to enable humans to do their jobs better.” Professor Ajmal Mian, UWA

A smarter environment

Aquaculture

As a free diver, Professor Mian was inspired to work with marine scientists on projects that would conserve the underwater environment.

In 2012, as part of an Australian Research Council (ARC) project, Professor Mian along with a team of scientists and industry stakeholders developed commercial grade automated software, which included multiple object tracking algorithms to measure the length of Southern Bluefin Tuna. The novel software offered improved accuracy on previous methods, with a less than 1% error rate, and a six fold reduction in operator time above the gold standard, leading to reduced costs in measurement and frequency distributions of populations of fishes. SeaGIS Pty Ltd was provided with the software to incorporate into their existing aquaculture platform EventMeasure.

Their subsequent work with technology companies Mapizy Pty Ltd and DCube Tech, and collaborator the Australian Institute of Marine Science used AI technology underwater to estimate fish abundance and size. Supported by a Business Research Innovation Initiative sector grant.

Supporting marine scientists to monitor fish using AI

“As a free-diver, I know the rate at which the fish population is depleting, and how fast the corals are vanishing. If we can measure these rates, we can create awareness and drive policies to help conserve the environment.’ Professor Ajmal Mian, UWA

Agriculture

In 2015, the team collaborated with the Western Australian No-tillage Farmers Association to examine the effect of frost exposure on flowering wheat plants. Using hyperspectral imaging data, they detected an increased susceptibility to aphids which would lead to reduced grain yield for producers. This led to further research into airborne remote sensing of frost-induced stress and the possible secondary effects on crop susceptibility to pests.

With a growing interest in the field, Professor Mian joined the ARC Research Hub for Driving Farming Productivity and Disease Prevention as a Chief Investigator in 2019, which uses big data and AI to improve farm automation, automate crop quality control and monitor crop and animal growing conditions.

The Department of Agriculture, Fisheries and Forestry approached the team in 2023 to better understand the value of forest trees. Specifically, they were keen to investigate the trunk structure: width above ground level; and the presence of bends or straight segments. This is valuable for effective resource management and planning, and for environmental considerations and regulatory compliance.

Farming

For sheep producers, the economic cost includes the commercial value of the flock, as well as the cost to grow sheep. It is critical to know which breeds are present as they can differ in value, though this can be challenging for inexperienced farmers. Whilst DNA testing is commonly used, it is not practical for real-time assessment.

In 2019, Professor Mian and his team used computer vision to efficiently classify the breed of sheep with an average accuracy of 95.8%. Working in collaboration with sheep producers to meet their specific needs has resulted in better identification of breeds and a more accurate estimation of meat yield and cost management, and improved consumer trust in the product.

Examples of classified sheep images with predicted labels

Defending against malicious attacks

Deep learning is the AI workhorse for many applications from self-driving cars to surveillance and security. It uses artificial neural networks to help computers learn to identify objects and perform complex tasks quickly and more efficiently than a human.

Within deep learning (and deep fakes), adversarial perturbation, or ‘noise’ that can’t be seen by humans, makes it vulnerable to malicious attacks that can manipulate data or embed viruses (backdoors) to gain control over the AI system. In 2022, Professor Mian and his team began work on a project to defend Artificial Intelligence against deception attacks.

The ARC project, and associated projects funded by the US Department of Defense examine the security of AI systems for defence applications by addressing its vulnerabilities to attacks. The aims are to detect attacks, identify the source, estimate the attackers’ capabilities, and cleanse the model, making them more robust and trustworthy. The team will provide a trained model that will be able to identify how much bias there is and any backdoor activity, providing surety to companies who would otherwise not have the resources to train their own models to detect malicious attacks and backdoor activity.

Perturbations not seen by the human eye

“We’re trying to find techniques where we can defend them or at the least we create this awareness within people that these are models that can be easily fooled by others explicitly.” Professor Ajmal Mian, UWA

The team have also used that research to better understand the process of generating deep fakes and to advocate that systems must be developed responsibly and without harmful bias. Their Backdoor Attack on text-to-image Generative Models (BAGM) framework embedded models with backdoors inflicting them with bias, to make them generate fake content which would manipulate user sentiments towards a product, with alarming realism. The next step being to train their models to better detect such bias and defend models and users.

AI in the urban jungle

By 2003, Professor Mian had already begun solving the challenge of viewing objects from all angles to create a 3D, 360-degree object. In 2006, he added object recognition and segmentation in clustered scenes, which considers the position of objects and poses of people, along with scene analysis. This made it possible to segment the environment and introduced the ability to track people from scene to scene. The work is highly applicable to urban and landscape planning authorities, for city security and for crowd management. This work has been highly cited by the research community for its contribution to the advancement of scientific knowledge.

In 2012, neural networks like AlexNet became famous by training on more than a million images for object classification. It solved many of the existing issues around deep learning, except one: multiple object tracking.

Professor Mian and his team were the first to propose a deep learning based solution to multiple object tracking which is commonly used for surveillance, to track multiple people in crowded settings, and for autonomous driving. In 2019, they developed Deep Affinity Network (DAN) which harnessed the power of deep learning for data association in tracking, by modelling object appearances in different frames, and in an end-to-end environment. It meant that objects that disappeared between frames could be easily identified and tracked. In 2022, they developed Simultaneous Multiple Object detection and Pose Estimation Network (SMOPE-Net) which is able to detect multiple objects at the same time, and understands the position of each.

Their research in 2021 used LiDAR technology to develop high-definition 3D point cloud mapping of the cityscape of Perth CBD, Western Australia to present a technique that was met local autonomous driving requirements and offered scalability. The work was supported by the ARC and led to the development of numerous self-localization algorithms published in ICRA 2023 (International Conference on Robotics and Automation), IROS 2023 (International Conference on Intelligent Robots and Systems) and DICTA 2021 (International Conference on Digital Image Computing Techniques and Applications). The technology can also be used for urban and landscape design, surveillance, and traffic management.

Multiple object tracking can track people in clustered scenes

“At the core of all these projects is artificial intelligence, machine learning and computer vision.” Professor Ajmal Mian, UWA

A deep learning workbench for athletes

In 2016, the team were approached by sport scientists at the UWA School of Human Sciences who were keen to improve how they record and advise runners on achieving optimum performance with minimal knee stress.

Traditionally, data was captured by videoing athletes from different angles whilst they ran through a sports lab, and ensuring their stride touched a force plate embedded into the ground by concrete.

With ARC funding, the team’s novel approach was to design a machine learning algorithm that removed the reliance on immovable and expensive force plates. By 2019 they had a proven approach that did not use a force plate, could recognise actions in real time irrespective of changes in camera viewpoint, lighting and clothing textures, and had an accuracy of 98%. The open source approach is now commonly used globally and is effective in reducing laboratory costs, and for injury prevention. The outcomes were published internationally and made the cover of the prestigious IEEE Biomedical Engineering Journal in 2019.

The team was supported by the Australian Institute of Sport in 2020 to develop a ‘deep learning workbench‘, a graphical interface which would make the process of data collection and analysis significantly easier. The team’s approach has the potential to trigger a revolution in the accuracy and validity of wearable sensors from community fitness to professional sport.

The team’s novel approach was featured on the cover of the IEEE Journal

There’s a code to every face

Professor Mian’s 3D facial analysis research in 2004 formed the basis of his future research to overcome the limitations around illumination, poses and subjects wearing makeup and created a fast and accurate way to identify individuals. By 2009, he was able to accurately differentiate between expression deformations and interpersonal disparities and hence recognise faces under any facial expression.

His work on 3D facial analysis is well regarded and has broad application.

“A significant part of my research was on 3D facial recognition. We started with a humble custom built 3D scanner in our lab, and eventually scaled to millions of 3D faces from around the world to train AI models that would learn to distinguish even between identical twins.” Professor Ajmal Mian, UWA

Autism

From 2015 – 2019 the team collaborated with the Telethon Kids Institute and UWA’s School of Psychological Sciences to use AI to identify autism in children. The project was supported by the National Health and Medical Research Council (NHMRC) and data from the multigenerational observational Raine Study. Their novel findings included that: children with autism have asymmetry and more facial masculinity; their parents have higher facial asymmetry; facial sex characteristics can be measured; and that facial masculinity in adulthood is highly correlated with higher testosterone levels in the umbilical cord at birth. They provided the first direct evidence of a link between prenatal testosterone exposure and human facial structure and brought clinicians closer to early diagnosis and treatment for autism. They expanded their work in 2023 to investigate movement and gesture patterns specific to autism, which could hold significant potential for advancing our understanding of autism related behaviours.

Creation of the ‘gender score’ for each face

Sleep Apnoea

Their project from 2016 – 2019 on predicting obstructive sleep apnoea (OSA) using 3D craniofacial photography resulted in techniques that could detect OSA with 88% accuracy. The software developed is used in medical research at Sir Charles Gairdner Hospital and by the UWA Centre for Sleep Science in estimating the correct CPAP mask for OSA patients. The study was funded by the NHMRC and used data from the Raine Study.

Outlining distance of facial features

“I want to make this world a better place to improve the human lifestyle, living standards and to protect our environment.’ Professor Ajmal Mian, UWA

Orthodontic surgery

Since 2018, Professor Mian and his team have worked with Senior Lecturer Mithran Goonewardene, the UWA School of Dentistry to apply facial analysis to orthodontic needs. The team has developed a technique, with a 2-millimetre accuracy that can guide the maxillofacial surgeon on where to move the bones during reconstructive surgery to ensure the best possible outcome for a patient with facial deformity. Orthodontists can use the technique during the consultation and post-surgery analysis. The team have also developed CAD-CAM customised surgical cutting guides and fixation plates to provide improved predictability of end results and better accuracy when performing mandibular repositioning surgery.

“I want to make this world a better place to improve the human lifestyle, living standards and to protect our environment.’ Professor Ajmal Mian, UWA