Analyzing facial features has long been a vital step in diagnosing genetic syndromes. In recent years, AI-driven technologies have transformed this process, making it more efficient and accurate. Leading this innovation is Face2Gene, an advanced AI platform that leverages machine learning to assist clinicians in identifying genetic disorders. To explore the development of this groundbreaking tool, we spoke with Aviram Bar-Haim, Strategic AI Advisor at FDNA, who played a key role in its creation.
The Early Days of Face2Gene
Face2Gene was born from a vision to leverage AI in diagnosing genetic syndromes through facial analysis. The inspiration initially stemmed from conditions such as Down syndrome and Noonan syndrome, where distinct facial features make identification more straightforward. The challenge – and ultimate goal – was to extend this capability to a wider range of syndromes using deep learning, enabling more precise and efficient diagnoses
“In the early days, our team drew inspiration from facial recognition advancements in AI,” Aviram explains. “Between 2015 and 2017, facial recognition technology surpassed human capabilities, with models like DeepFace and FaceNet leading the field. We wanted to pivot this technology from identifying individuals to recognizing facial patterns associated with genetic syndromes.”
Building the AI Model
The development process involved several stages. Initially, researchers collected data and experimented with classical machine learning models, such as support vector machines (SVMs). However, these early models lacked the sophistication required for accurate syndrome classification. We clearly needed to reach a richer phenotypic representation of the facial images.
“We started with many different models, each attempting to classify a syndrome from an image,” Aviram recalls. “But soon, we realized that training a single, unified model to recognize multiple syndromes at once, produced better results.”
A breakthrough occurred when the team harnessed large-scale datasets to train deep learning models. By analyzing thousands of images, the AI system learned to extract relevant facial features and classify genetic syndromes with greater accuracy. This revolutionary technology became known as DeepGestalt.
“DeepGestalt transforms facial images into numerical vectors, which represent key facial features,” Aviram explains. “These vectors are then used to predict syndromes with increasing accuracy as more data is incorporated into the training process.”
The Role of Data in AI Training
One of the biggest challenges in developing Face2Gene was obtaining high-quality, diverse training data. Genetic syndromes vary widely, and many are rare, making it difficult to collect enough images for robust model training.
“We needed many thousands of images to train the model effectively,” Aviram explains. “At first, we relied on public databases along with published research papers. We also partnered with clinicians and geneticists to expand our dataset, and had to design ways in which we could use the images while these were fully de-identified.”
As more clinicians adopted Face2Gene, the system continuously improved through machine learning. Each new diagnosis provided valuable feedback, refining the algorithm model’s accuracy over time.
Overcoming Limitations and Expanding to New Syndromes
Despite its success and good performance, Face2Gene’s technology faced inherent limitations, particularly when it comes to ultra-rare disorders. To address this, FDNA developed patient-matching initiatives, connecting facial images of individuals with similar phenotypic traits to enhance the understanding for rare conditions. “Most genetic syndromes fall into what we call the ‘long tail’ – conditions with very few documented cases,” Aviram explains.” To overcome this, we developed a patient-matching approach that groups similar cases, enhancing our ability to identify rare syndromes. We called this technology GestaltMatcher.”
The Evolution from DeepGestalt to GestaltMatcher
Aviram explains that the DeepGestalt technology employs deep learning to analyze facial phenotypes and classify them into predefined syndromes. While highly effective, it has an inherent limitation: it can only identify conditions already present in its database. If a syndrome is absent from the system, DeepGestalt cannot recognize or categorize it.
GestaltMatcher addresses this constraint by introducing an innovative approach. Rather than relying solely on classification, it converts facial images into numerical vectors that represent distinct phenotypic features. “These vectors are then compared to detect similarities between patients, even if their syndrome is not yet classified in the database. This method enables the delineation of new syndromes by clustering patients with shared phenotypic characteristics.”
Aviram continues by saying that GestaltMatcher has the potential to revolutionize genetic diagnostics. “Its ability to analyze and compare phenotypic data in a mathematical space opens the door to improved precision in diagnosing rare genetic disorders, enhanced research opportunities for uncovering new gene-to-syndrome correlations and facilitates collaborations between geneticists and companies to develop targeted therapies for undiagnosed patients.”
By continuously refining the technology and expanding its dataset, FDNA aims to further improve GestaltMatcher’s accuracy and usability in both clinical and research settings
How GestaltMatcher Works
The process begins with an image of a patient’s face, which is passed through multiple layers of a neural network. Each layer applies a series of filters, progressively extracting and refining relevant features. The final output is a numerical vector that captures the unique phenotypic characteristics of the patient.
Unlike DeepGestalt, which classifies images based on a predefined list of syndromes, GestaltMatcher measures the similarity between these vectors. By calculating the distance between vectors, the system can group patients with similar facial phenotypes, even when their condition has never been previously documented.
This newer approach allows researchers to:
- Identify undiagnosed patients with similar phenotypic features.
- Cluster patients based on facial similarities rather than a pre-existing diagnosis.
- Support the delineation and description of new genetic syndromes and possibly molecular pathways, through phenotype-based clustering.
“The development of the GestaltMatcher algorithm has only been possible using DeepGestalt as a basis. We demonstrated in the 2022 Nature Genetics publication that without its rich phenotypic representation, the GestaltMatcher results would be far less accurate,” highlights Aviram
The Future of AI in Genetic Diagnosis
Face2Gene has already transformed genetic diagnostics, yet its potential remains far from fully realized. As AI models become more sophisticated and datasets expand, both the accuracy and scope of genetic diagnosis will reach unprecedented levels.
“We are only beginning to unlock AI’s true potential in genetics,” Aviram concludes. “By continuously refining our algorithmic models and integrating diverse sources for phenotypic descriptions, from medical notes using Large Language Models, to various imaging modalities beyond facial analysis, and fostering global collaborations within the genetics community, we aim to accelerate rare disease diagnoses, enhance precision, and make genetic insights universally accessible.”
Face2Gene stands as a testament to the transformative power of AI in medicine, offering hope to families and clinicians searching for answers in the complex world of genetic disorders.
