How the SaProt Protein Language Model Breaks the "Sequence-Only" Limit

Fig. 1 SaProt structure-aware protein language model: from sequence and structure toward function prediction

When a language model can not only read the amino-acid sequence but also "see" the local 3D fold around every residue, what happens to protein function prediction? SaProt offers a reproducible answer.

Why does a protein language model need to "see" structure?

The rise of protein language models (PLMs) rests on a simple yet powerful fact: the amino-acid sequence determines the 3D fold, which in turn determines function. Over the past few years, PLMs such as the ESM family, pretrained in an unsupervised way on hundreds of millions of natural sequences, learned a "sequence → implicit biological rule" mapping and performed well on structure prediction, function annotation, and variant-effect assessment.

But a "sequence-only" model has a natural blind spot: the same amino-acid sequence can adopt clearly different local conformations under different conditions or as part of different domains, and similar functions often arise from similar 3D structures rather than identical sequences. Sequence tokens cannot explicitly carry the spatial environment of a residue; the model must infer structure implicitly.

So the question becomes: can we feed explicit structural information directly to the language model instead of making it infer structure from sequence alone?

That is exactly where SaProt starts — converting 3D structure into discrete tokens that a Transformer can consume directly, aligning them with amino-acid tokens to form a "structure-aware vocabulary."

How does SaProt's "structure-aware vocabulary" work?

Fig. 2 Structure-aware vocabulary: Cartesian product of amino-acid and 3Di structure tokens

Core mechanism: a vocabulary that encodes both sequence and structure into tokens

A classic PLM vocabulary is 20 amino acids plus special tokens. SaProt's approach:

1. Residue token: keeps the amino-acid type (A, V, L, D…), representing "which residue this is";

2. Structure token: uses Foldseek's "3Di" local-structure alphabet (about 20 3Di states) to encode "what local 3D fold this region takes," representing "how this residue sits in its local structure";

3. Symbolic combination: multiply "residue × structure" to obtain a structure-aware vocabulary — 21 (20 amino acids + mask) × 21 (20 structure states + mask) ≈ 441 tokens. Each position becomes a paired symbol "amino-acid letter + 3D-state letter," so the model sees sequence and structure simultaneously in one attention space.

Pretraining and architecture

With this vocabulary, SaProt uses BERT-style masked language modeling, pretrained on roughly 40 million protein sequences and structures (mainly AlphaFoldDB and PDB). Architecturally it reuses ESM-2 (about 650M parameters), but because the vocabulary grows, its embedding layer is enlarged to host 441 tokens. The team releases 35M / 650M / 1.3B checkpoints and an additional PDB-tuned 650M version, so users can choose by task scale and compute budget.

Figure note: The image above is an abstract illustration of the "amino-acid token × 3Di structure token → structure-aware vocabulary" fusion; it is not to scale and is a schematic, not a residue-level annotation of any specific protein.

Boundaries (important)

Because "structure awareness" depends on 3D structure input, SaProt's boundaries follow:

· Structure required: every target protein/mutant needs a 3D structure first (experimental or predicted, e.g., AlphaFold2);

· Sensitive to structure quality: low-confidence regions (e.g., low pLDDT) degrade the reliability of their 3Di tokens and can affect downstream judgment;

· Not a "structure substitute": it fuses local-conformation information, not full de novo structure modeling, and cannot replace complete 3D structure prediction plus experimental validation.

These boundaries are precisely why "model + experiment + engineered platform" synergy matters.

Where does SaProt perform better?

The SaProt paper systematically evaluates a broad set of downstream tasks, with overall performance exceeding mature baselines such as ESM-1b and ESM-2. This spans both protein-level and residue-level tasks:

 Notably, SaProt is not only better on "structure-centric" tasks; it also leads overall on purely sequence-based evaluations (such as zero-shot variant-effect, stability, and function prediction), showing that explicit structure information can feed back into sequence-semantic modeling.

Real results must be measured: The above are model-level results from the paper and public benchmarks. Applied to a specific protein and process, they must be validated with experimental data and the target system; do not transfer them to an untested protein.

How to use SaProt in protein engineering?

To get value from SaProt, a closed loop usually applies: "structure preparation → scoring/embedding → experimental validation → iteration":

1. Prepare the structure: use an experimental structure or an AlphaFold2 prediction for the target protein;

2. Generate a structure-aware sequence: encode the structure into 3Di structure tokens with Foldseek and concatenate with the amino-acid sequence;

3. Infer and score: run masked-pretraining/fine-tuning or extract embeddings for variant-effect scoring, stability ranking, function prediction;

4. Experimental validation: wet-test top candidates (enzyme activity, thermal stability, expression, etc.);

5. Close the loop: feed results back to refine scoring preferences.

Fig. 3 SaProt applications in protein engineering and the dry-wet closed loop

Community practice confirms this path: per public reports, SaProt / ColabSaProt has been wet-lab validated on more than 10 proteins (enzyme mutations, fluorescent-protein modification and fluorescence prediction, among others) and has ranked first on the authoritative ProteinGym leaderboard.

Engineering note: SaProt is an open-source discriminative/representation model (MIT license; available on GitHub and HuggingFace). To scale it in real R&D, however, you still need to turn "structure acquisition, scoring, experiments, data feedback" into an automated dry-wet closed loop, controlling structure quality and batch consistency.

What MatwingsVenus® brings: from "model" to "dry-wet closed loop"

MatwingsVenus® does not add value by rebuilding a model like SaProt; it adds value by turning model predictions into verifiable, iterable R&D outcomes:

· wet-lab service: provides real measurements (enzyme activity, thermal stability, expression) for SaProt's high-scoring candidates, closing the loop from "prediction" to "measurement" rather than staying at numeric scores;

· expert-center: works with protein engineering and bioinformatics experts to deliver an end-to-end plan from structure preparation, model selection, to experimental design;

· dry-wet-closed-loop-platform: strings "structure-aware scoring → experimental validation → data feedback" into an automated loop, supporting batch sequence design, variant ranking, and next-generation candidate generation.

In other words: SaProt provides a "smarter discriminator"; MatwingsVenus® provides the engineered system that validates it experimentally and keeps it self-correcting. Together they turn a structure-aware model's potential into real R&D results.

FAQ

What is SaProt?

SaProt (Structure-aware Protein Language Model) is a structure-aware protein language model proposed by the Westlake University team. It encodes 3D structure explicitly into the model's token space through a dual vocabulary of "amino-acid residue + 3Di local structure."

How is SaProt different from ESM-2?

Both are Transformer-based; SaProt reuses the ESM-2 architecture (about 650M parameters) but expands the vocabulary from 20 amino acids to a structure-aware vocabulary (~441 tokens) that includes 3Di structure tokens, so its embedding layer is larger and it explicitly fuses structural information.

What input does SaProt need?

It needs the target protein's amino-acid sequence + 3D structure (experimental or predicted, e.g., AlphaFold2). The structure is encoded by Foldseek into 3Di structure tokens and concatenated with sequence tokens.

What can SaProt predict?

It supports protein function and structure annotation, stability / fitness / variant-effect prediction, protein–protein interaction, clinical variant interpretation, and can serve as a discriminative base for sequence design and directed-evolution assessment.

Where are SaProt's boundaries?

It depends on reliable 3D structure input; 3Di tokens degrade on low-confidence structures. It encodes local-conformation information and cannot replace full 3D structure modeling or experimental validation, so it must be used together with experiments and an engineered platform.

Which SaProt size should I use?

Official checkpoints are 35M / 650M / 1.3B, plus a PDB-tuned 650M variant. Use 35M for small proteins/lightweight tasks to save compute; use 650M–1.3B for higher ceilings; use the PDB-tuned version when stronger structure alignment is needed. Choose by task and compute budget.

Is SaProt open source and commercially usable?

Yes. SaProt is open-sourced under the MIT license; code is on GitHub (westlake-repl/SaProt) and pretrained weights are on HuggingFace, supporting community and commercial reuse and retraining.

Summary

SaProt's core insight is: turn structure from a background that a language model must infer, into tokens the model can explicitly consume. Through a structure-aware vocabulary of "amino-acid residue × 3Di local structure," it pretrains on about 40 million protein structures and sequences and leads classic sequence models overall across 10 tasks, making it a representative open-source base for joint sequence–structure–function modeling.

But like all models, SaProt has boundaries — structure quality, input dependence, and the fact that "prediction is not measurement." Turning its capability into R&D outcomes requires model + wet-lab validation + dry-wet closed-loop working together. MatwingsVenus® empowers protein engineering precisely by "making the structure-aware model experimentally validated and continuously self-correcting," turning AI prediction into deliverable R&D progress.

Compiled from public papers and open-source materials; performance descriptions of SaProt are model-level / public-benchmark results, not measured commitments for any specific protein. Please validate on your target system.