VAE Latent Space Maps Enzymes, Finds New FDMOs for Biocatalysis

TL;DR: University of Michigan researchers built a variational autoencoder that maps enzyme sequences into a continuous two-dimensional space to guide sampling for new biocatalytic reactions.

A continuous map for choosing which enzymes to test

A study published September 1, 2026, in volume 123, issue 36, article e2608891123 described an optimized variational autoencoder, or VAE, based latent space model for visualizing and exploring enzyme sequence-function landscapes, according to Leveraging latent space models for enzyme discovery and sampling | PNAS. The authors were Chang-Hwa Chiang, Daniel Ong, Alison R. H. Narayan and Charles L. Brooks III.

The work was edited by Nikolay V. Dokholyan of the University of Virginia in Charlottesville, VA, received March 11, 2026, and accepted July 29, 2026 by Editorial Board Member Eugene V. Koonin. The article is identified by https://doi.org/10.1073/pnas.2608891123.

The team trained models on simulated datasets and on real enzyme families, such as cyclases and flavin-dependent monooxygenases, termed FDMOs. The authors reported that the optimized latent space effectively preserves phylogenetic relationships and enables high-resolution clustering for functionally distinct enzymes.

The same report stated that the models outperform traditional sequence similarity networks, termed SSNs, in capturing local and global relationships in a continuous two-dimensional space. Application of the approach enabled the discovery of multiple uncharacterized FDMOs for oxidative dearomatization and decarboxylative hydroxylation. The findings were presented as evidence that low-dimensional latent spaces can serve as valuable tools for enzyme discovery, allowing for interpolation and extrapolation to guide novel enzyme sampling for biocatalytic reactions, as described in Leveraging latent space models for enzyme discovery and sampling.

Why sequence-function relationships matter

Understanding how protein sequences relate to their functions is critical for discovering enzymes with applications in biotechnology and medicine. That statement of significance framed the entire project.

Traditional tools, such as phylogenetic trees and sequence similarity networks, are commonly used for this purpose. The paper reported that those tools struggle to handle large protein datasets and fail to capture important information about higher-order and complex interresidue interactions in protein sequences.

In response, the authors developed a VAE-based model to overcome these limitations by mapping protein sequences into a continuous, interpretable space. This approach enables more precise clustering of enzymes and identifies functional relationships that traditional tools miss.

By applying this model, the team reported the discovery of uncharacterized enzymes with distinct functions. The authors presented that result as demonstrating potential to accelerate enzyme discovery for diverse biocatalytic applications.

The scale that motivates new tools

The increasing number of protein sequences in databases such as UniProt, Pfam, InterPro and the National Center for Biotechnology Information, termed NCBI, provides scientists with access to an astonishing number of enzyme sequences for novel functions in the lab.

The paper reported approximately 218 million sequence entries in the UniProt Knowledgebase, termed UniProtKB, in January 2026. It also reported that only 0.22% of protein sequences have been manually reviewed and annotated by experts in UniProtKB.

The authors noted that this percentage will become lower as the number of sequences grows exponentially. They described this pattern as a widening gap between sequence and function information.

That gap was presented as underscoring the need for tools to explore the protein sequence space and to facilitate understanding of sequence-function relationships based on available data.

Where phylogenetic trees help and where they stall

Phylogenetic analysis is an essential tool for investigating protein sequence-function relationships, commonly used by bioinformaticians and evolutionary biologists. Phylogenetic trees are constructed from a set of aligned homologous sequences obtained through multiple sequence alignment, termed MSA, often with a chosen substitution model.

Phylogeny can then be used to infer sequence-function relationships. While useful for studying evolutionary relationships, phylogenetic trees become computationally intensive and difficult to visualize for larger protein datasets.

The paper noted that visualizing trees can be challenging. It also stated that high-quality phylogenetic analysis requires extensive curation and is computationally demanding especially for larger protein datasets.

Those limits were described as exacerbated for large enzyme families. The authors positioned the VAE approach as a complement that preserves phylogenetic relationships while remaining visualizable in low dimensions.

How sequence similarity networks are built

Sequence similarity networks, as an alternative, have been developed to display pairwise sequence similarity relationships among homologous proteins. SSN generation is typically performed on the EFI-EST website, beginning with pairwise sequence alignment, followed by selection of a user-defined alignment score, termed AS, and visualization in Cytoscape.

Each SSN comprises multiple clusters, with nodes representing enzymes and edges indicating relatedness between the nodes within each cluster. The number of clusters is determined by the user-determined AS for isofunctional pairs of sequences.

A higher threshold results in more putatively isofunctional clusters, while a lower threshold causes all enzyme nodes to merge into one giant cluster. SSNs are renowned for their user-friendliness and easy integration with downstream tools like genome neighborhood analysis on EFI-GNT.

However, the authors stated that SSNs have several intrinsic limitations. They reported that SSNs are sensitive to user-defined thresholds for sequence clustering and easily fail to capture more distant relationships between clusters.

The alignment limitation shared by older tools

Traditional tools like phylogenetic trees and SSNs are widely adopted for sampling enzymes for novel transformations. Their utility suffers from inherent limitations, which are exacerbated for large enzyme families.

Additionally, both tools are alignment based and cannot capture higher-order interactions between residues. That shared property was central to the rationale for a model that learns from sequences without relying solely on alignment.

The exponential growth in available protein sequence data has broadened enzyme discovery opportunities but simultaneously highlighted a significant gap between sequence and function information. The authors addressed these limitations by optimizing a VAE-based latent space model to visualize and explore enzyme sequence-function landscapes.

The model maps sequences into a continuous, interpretable space rather than sorting them into threshold-dependent clusters or branching diagrams alone. The authors reported more precise clustering and identification of functional relationships that traditional tools miss.

What the VAE latent space does differently

In the study, the optimized latent space was trained and evaluated on simulated datasets and real enzyme families. The use of simulated datasets provided a controlled setting for checking whether known relationships were retained.

Training included real enzyme families, such as cyclases and FDMOs. The authors demonstrated that the optimized latent space effectively preserves phylogenetic relationships.

The same evaluation showed high-resolution clustering for functionally distinct enzymes. The continuous two-dimensional space was reported to capture both local and global relationships.

The models were reported to further outperform traditional SSNs in capturing those local and global relationships. The comparison was framed around visualization and exploration of sequence-function landscapes, not only assignment to a single cluster.

Testing on cyclases and FDMOs

By training models on simulated datasets and real enzyme families, such as cyclases and flavin-dependent monooxygenases, the team tested whether the approach generalized beyond synthetic controls.

The paper demonstrated effective preservation of phylogenetic relationships in the optimized latent space. It also demonstrated high-resolution clustering for functionally distinct enzymes within those families.

The FDMO experiments provided the applied illustration. The models enabled the discovery of multiple uncharacterized FDMOs for oxidative dearomatization and decarboxylative hydroxylation.

That application was presented as illustrating the use of low-dimensional latent spaces for enzyme discovery. The authors stated that the spaces allow for interpolation and extrapolation to guide novel enzyme sampling for biocatalytic reactions.

From map to sampling decisions

The reported workflow moves from visualization to exploration to sampling. Visualization places enzyme sequences in a continuous two-dimensional space where proximity reflects learned relationships.

Exploration examines local neighborhoods and broader global organization in that space. Local relationships distinguish closely related enzymes, while global relationships retain more distant connections that threshold-based clustering can miss.

Interpolation and extrapolation were described as the mechanism for guiding novel enzyme sampling. Interpolation points between characterized regions, while extrapolation points beyond them, to prioritize candidates for biocatalytic reactions.

The authors concluded that low-dimensional latent spaces can serve as valuable tools for enzyme discovery on that basis. The discovery of multiple uncharacterized FDMOs for oxidative dearomatization and decarboxylative hydroxylation was offered as the concrete illustration.

Who did the work and how to reproduce it

C.-H.C., A.R.H.N. and C.L.B. designed research, while C.-H.C. and D.O. performed research, according to the author contributions statement. C.-H.C., D.O., A.R.H.N. and C.L.B. analyzed data, and C.-H.C., A.R.H.N. and C.L.B. wrote the paper.

The corresponding authors were identified as Alison R. H. Narayan and Charles L. Brooks III of the University of Michigan, with contact addresses listed in the article. The authors declare no competing interest.

Python scripts, Jupyter notebooks and computational results to reproduce the modeling analyses are at https://github.com/BrooksResearchGroup-UM/latent_space_paper. The supporting information includes Appendix 01 as a PDF download listed at 13.39 MB.

The team thanked the team led by Dr. Jan-Fang Cheng at the Joint Genome Institute, termed JGI, for providing the genes in the FDMO library. They also thanked David Braun for maintaining computational clusters and solving computing-related problems.

Funding and publication record

This research was supported by funds from the University of Michigan Life Sciences Institute, the University of Michigan Department of Chemistry, the NIH R35 GM124880 to A.R.H.N., and R35 GM130587 and S10OD034346 to C.L.B.

The publication history lists receipt March 11, 2026, acceptance July 29, 2026, and publication September 1, 2026. The editorial handling involved Nikolay V. Dokholyan and Editorial Board Member Eugene V. Koonin.

The article appears as September 1, 2026, 123 (36) e2608891123 with DOI https://doi.org/10.1073/pnas.2608891123. Its significance section emphasizes applications in biotechnology and medicine through improved understanding of sequence-function relationships.

For readers tracking methods, the key distinctions remain the continuous interpretable space, preservation of phylogenetic relationships, high-resolution clustering, and outperformance of traditional SSNs in capturing local and global relationships. For readers tracking outcomes, the key result remains the discovery of multiple uncharacterized FDMOs for oxidative dearomatization and decarboxylative hydroxylation to illustrate application to biocatalytic sampling.