exp: what contacts-v1 computes (representations, neurons, circuit)

Kind: evals

Question. How does contacts-v1-exp277-m2-p06-full-epoch-1.5B turn a
shuffled bag of <pN> <AA> statements into <contact> <pX> <pY>
statements? No training decision rides on this; the deliverable is a set
of findings about the model, each either a representation with a control
or a mechanism with a causal test on the production readout. Full plan: the first
comment below.

Starting map (from exp341, exp262, exp224, exp218). Layers 0-2 are
essential (ablation costs 0.41-0.51 R-precision) and hold previous-token
heads; no contact signal is readable at the query before boundary 12;
layer 14's MLP and attention in layers 18-20 are costly (0.25-0.43); late
MLPs do most of the final work; the contact Jacobian is local; exposure
to a structure in training adds 0.04-0.085 R-precision.

Three analyses.

Representations by layer. Fourier and circulant analysis of the
2,000 <pN> embeddings with a wrap-crossing behavioural test; linear
probes per boundary for chain-neighbour identity, DSSP, burial, Pfam
domain, a bilinear contact probe, and a distance probe tested by
triangle inequality and MDS-to-TM-score; random-init and shuffled-label
controls; replicated on exp232.
Neuron audit. Top and bottom examples for all 196,608 MLP neurons
on 2,000 training documents, scored against a structural label table at
thresholds fixed in advance; model-written jobs with a held-out
discrimination test and a random-init baseline; strong-vs-weak
ablation; 100 clamp tests; 12 attribution graphs on raw neurons verified
by ablation. Three TopK SAEs as a bounded comparison (label table,
near-copy-of-a-neuron rate, contact-map prediction, clamping), continued
only if a latent finds something no neuron does.
Contact-decision circuit. A census of all 768 heads (statement
binding, chain assembly, partner lookup, Rao readout, emitted-contact
attention, sinks) with ablation and path patching; activation patching
to locate where and how the partner is decided; the gain from
already-emitted contacts under teacher forcing; the duplicate-avoidance
mechanism; pair symmetry.

Data. 2,000 training-corpus AFDB documents with structures for labels;
eval-val, eval-denovo and legacy-554 for accuracy-linked splits; synthetic
documents varying offset, shuffle and index wrap. No eval-test reads.

Compute. About 55 L40S-hours planned, budgeted at 80 with reruns, plus
API calls for about 6,600 neuron explanations; about 100 GB of in-region
S3 at peak.

Not included (follow-ups): across-model CKA and memorization traces
on exp341's panel, checkpoint trajectories, the sequence-section
categorical Jacobian (exp341 A1), crosscoders, training-data attribution.

Relation to open issues. This is #341's deferred list (position-token
geometry, attention-head circuits and patching, the chain-reconstruction
probe, SAEs) as its own experiment. #157 (positional embeddings for
residue tokens) is a training change that analysis 1.1 informs; #262
Phase 0 is the head census this extends to all 768 heads; #339 and #211
ask about rollout outputs, not internals.

Builds on. exp341 (harness, lens, ablations, Jacobians, panel), exp262
Phase 0 (head probes), exp321 (emitted-contact prior), exp224, exp218,
exp247, exp201. Linked references are in research_plan.md §12.

What contacts-v1 computes: research plan

Status: draft, revised 2026-10-26 (three analyses, references added). The
directory is named expXXX_ until an issue exists; rename to
exp<N>_evals_contacts_v1_mechanisms when it does. Nothing here has run.

1. Purpose

exp341 used interpretability to settle two training decisions (family
weighting; depth vs width) and reported each result against a rule fixed in
advance. This plan has no training decision attached. Its object is the
model itself: contacts-v1-exp277-m2-p06-full-epoch-1.5B, a 24-layer Qwen3
decoder with a 2048-wide residual stream, 8192-wide SwiGLU MLPs (196,608 MLP
neurons in all), 32 query heads per layer (768 heads in all), and a 2,845
token vocabulary of which 2,000 are residue-index tokens <p0>..<p1999>.
It reads a shuffled bag of <pN> <AA> statements and emits
<contact> <pX> <pY> statements. The question is how.

Three analyses, no more:

Representations by layer. What is decodable from the residual
stream, where, including whether the model holds an internal geometry.
The neuron audit. What the MLP neurons respond to, whether they
have one job each, and whether the ones with a job do it. SAEs enter
here as a bounded comparison.
The contact-decision circuit. Which heads and components turn a
<contact> <p_i> query into a partner, and what bookkeeping the
structure section requires.

A result counts as a finding when it meets one of two bars:

A representation with a control. Something is decodable from the
residual stream at a named layer, the same decoder scores near chance on
a random-init model and on shuffled labels (Hewitt and Liang's selectivity),
and the decoded quantity is one a structural biologist would name
(secondary structure, burial, sequence separation, pairwise distance).
A mechanism with a causal test. A named set of components (heads,
neurons, a residual direction) is proposed to do a job, and removing or
clamping those components changes the production readout
(pairwise_score_matrix, 4 document seeds, R-precision) in the predicted
direction and on the predicted pairs, by more than the same number of
random components does.

Descriptions that meet neither bar (an SAE latent with a plausible label and
no intervention; a probe with no control) are recorded as leads, not
findings.

2. The framing questions

2.1 SAEs?

Inside analysis 2, bounded, after the neuron baseline exists. The 2025
negative results (DeepMind's deprioritisation; Kantamneni et al. on sparse
probing; Heap et al. showing the SAE plus auto-interp pipeline scores the
same on a random-init model) mean an SAE-only result will not be believed
without a probe baseline and a causal check, and once those exist the SAE
is the optional part. Goodfire's "Neural networks are already clean"
(Cammarata, 2026) reports that on ConvNeXt's last layer 88% of SAE activity
is near-copies of single neurons, that SAEs helped most in the smallest
model, and that attribution graphs on raw MLP neurons of Qwen3-32B
reproduce Anthropic's transcoder graphs with no reconstruction loss.
Whether that holds for a protein model is a question with a cheap answer
once the audit exists: three SAEs, one evaluation table, and a rule (2.4 in
analysis 2) for whether SAE work continues.

2.2 Straightforward analysis in the Goodfire style?

Yes, as analysis 2. The style is: top and bottom activating examples per
neuron on the training distribution, both poles (SwiGLU products go
negative), strong activations weighted over weak ones, a label table of
cheap annotations scored against every neuron, a model-written explanation
for the clean ones with a held-out test, then an ablation that tests the
explanation on the readout. Superposition is the last hypothesis, not the
null. contacts-v1 suits this because every token has a known grammatical
role and every residue has a known structure, so the label table is richer
and cheaper than for a text model.

2.3 Training data with structures, or predictions?

Both, with a rule for which is used where:

Mechanism analyses condition on the model's own output. A circuit
explains why the model emitted <p_j> after <contact> <p_i>, whether or
not (i, j) is a true contact. Attribution, head and neuron ablations and
patching therefore target the model's top-ranked pairs from the pairwise
readout, and ground truth enters afterwards to split the results into
correct and incorrect predictions. exp341 A2 already found one such split
with no difference (locality 0.002 apart), a hint that the computation is
the same either way.
Representation analyses use ground truth from structures. Probes for
secondary structure, burial, pairwise distance and contact need labels,
and the labels come from the structure the document was built from.
The neuron audit runs on the training distribution. Neurons were
shaped by the corpus, so their top examples come from it. The corpus
documents are AFDB and ESM Atlas predictions with per-residue pLDDT, and
the AFDB structures (mmCIF in timodonnell/afdb-24M's cif_content)
supply DSSP, coordinates, and, through the UniProt accession, Pfam
domains.
Accuracy-linked claims use held-out proteins. eval-val (97),
eval-denovo (19), legacy-554 (ground truth with continuous degree) and
exp341's 2,000-structure panel (1,000 removed structures exp277 never saw,
1,000 matched kept structures it trained on; ground truth from each
structure's own untruncated document). eval-test is not read.

Teacher-forcing a full training document (sequence section plus its
contact statements) is the only way to observe the model in the structure
section, where emitted contacts condition later ones and where
duplicate-avoidance and symmetry bookkeeping happen (analysis 3). The
pairwise readout only ever shows the model a prefix and one two-token tail.

3. What is already known

The starting map, from runs in this repo:

Layer range	Evidence	Reading
0-2	Ablating attention or MLP of layer 0 or 1 costs 0.49-0.51 R-precision; layer 2 attention 0.41 (exp341 B1). exp262 Phase 0: layer 1 holds heads that are nearly pure previous-token heads (L1H26 and two others), and 10 of 768 heads put over 0.30 of their mass at offset 1	Statement parsing: binding `<pN>` to its `<AA>`, and the first steps of chain assembly from the shuffled bag
3-13	Ablation costs 0.005-0.20; layers 11-13 under 0.02. No lens signal at the contact query through boundary 11 (R-precision 0.004-0.028)	Whatever is built here is not readable at the query by the unembedding. The sequence-section pairwise probe that would read it never ran
14-15	Layer 14 MLP costs 0.39-0.43; layer 15 attention 0.20. Logit lens rises from 0.059 at boundary 12 to 0.390 at 15	Onset of contact information at the query
18-22	Attention in 18, 19, 20 costs 0.25-0.32; MLPs in 18, 20, 21, 22 cost 0.33-0.53; attention in 21 and 22 costs 0.01 and 0.002	Partner selection: three attention layers that gather, then MLP-heavy refinement. The lens dips at boundary 21 and recovers by 24
23	MLP 0.10-0.18, growing with length	Final sharpening, heavier on long proteins

Other facts the plan leans on:

The contact Jacobian is local (exp341 A2): mutating residue i or j moves
P(i, j) 6 times the chain average, and residues that touch i or j in the
structure move it barely more than random residues (structural share
0.028 against a null of 0.023). Exposure to a structure in training
raises locality, not lowers it.
Training on a structure adds 0.04-0.085 R-precision over never seeing it,
at every count of surviving relatives (exp341 A4).
The sequence section is an any-order amino-acid model worth about
ESM2-35M on ProteinGym (exp218).
About 77% of validation loss is permutation entropy (exp201 Phase 0, as
cited in exp262): the order of statements is noise the model pays for.
A circular permutation of DsbA costs 0.093 R-precision, and 0.121 on the
pairs whose separation the permutation changed (exp224): the model uses
sequence separation, not only residue content.

The three analyses fill the gaps in this table: what layers 3-13 build,
how the residue index is represented so that separation can be computed
from two tokens, what the layer-18-20 attention gathers, and which neurons
do the late MLP work.

4. Shared setup

4.1 Model and controls

Primary: contacts-v1-exp277-m2-p06-full-epoch-1.5B (step 266344), HF
export at hf://buckets/open-athena/MarinFold/checkpoints/contacts-v1-exp277-m2-p06-full-epoch-1.5B/hf/step-266344.
Random-init control: the same config, seeded. Every probe and every
auto-labelled neuron is scored on it too.
Replication model for analysis 1: contacts-v1-exp232-m2-p06-train-1.5B
(step 363000), trained from scratch on a different corpus with the same
architecture; already staged in exp341. A probe curve that holds on both
is a property of contacts-v1 models; one that holds on exp277 alone is
reported as such.
RoPE hazard from exp341: every load goes through load_backend so
repair_rope is applied (theta 500000, Llama3 scaling); exp341's harness
check 1 verified it.

4.2 Protein sets

Set	Size	Ground truth	Used by
S-train	2,000 AFDB documents sampled from the decontaminated corpus the model trained on (the exp341 `panel_candidates.py` path reads them by column projection from the public bucket), stratified by length (4 bins to 1,000 residues)	the document's own contacts; mmCIF from `afdb-24M` for DSSP, CA coordinates, pLDDT; Pfam domains from InterPro by accession	1, 2, 3
S-eval	eval-val 97, eval-denovo 19, legacy-554 (continuous degree)	experimental	every accuracy-linked split; probe test sets
S-synthetic	documents built to vary one thing: chains placed so the index crosses the wrap at 1999 to 0; the same protein at 8 offsets; the same protein at 8 shuffles with the offset fixed; a document with one residue mutated	none needed	1, 3

Document token roles, recorded for every token of every document so the
label table and the head census can condition on them: seq-pos,
seq-aa, nterm-marker, nterm-pos, cterm-marker, cterm-pos,
begin-statements, contact-verb, contact-arg1, contact-arg2, end.

4.3 Label table

One row per (document, token), built once on CPU and joined to every
activation:

Residue-level (for seq-pos, seq-aa, contact-arg1, contact-arg2):
amino acid; hydrophobicity class; DSSP 3-state (pydssp on the mmCIF);
burial as CA neighbours within 10 Å; contact count in the document;
relative chain position (fraction from the N-terminus); distance in
residues to the nearest terminus; pLDDT; whether the residue sits within
5 of the index wrap; Pfam domain id and whether the residue is within 3
of a domain boundary (InterPLM and Nainani et al. found domain and motif
detectors in ESM-2; the table should be able to see them here).
Pair-level (for contact-arg2, and for query tokens in the readout):
sequence separation (signed and absolute); separation band (6-11,
12-23, 24+); true contact; contact degree; the pair's DSSP classes
(strand-strand, helix-helix, mixed); whether the symmetric statement was
already emitted; number of contacts already emitted for this residue.
Document-level: length; Foldseek cluster id (struct_cluster_id in the
corpus rows); number of contacts; position in the structure section
(fraction of statements emitted so far).

4.4 Harness additions

exp341's harness.py (branch exp/341-contacts-v1-interpretability) has
model loading with the rope repair, residual capture at every layer
boundary, sublayer mean-ablation, and the eval, legacy and panel loaders.
This experiment imports it and adds:

Neuron capture. A hook on each layer's mlp.act_fn output times the
up_proj output (the 8192-vector down_proj reads), with three sinks:
a running histogram per neuron (256 bins, range from a calibration
pass), top-50 and bottom-50 example buffers per neuron (document id,
token index, value), and a dense dump for a fixed subset of 50 documents.
Attention capture. Per-head attention rows at chosen query tokens,
reduced on the fly to the census scores in analysis 3 (the full map is
768 heads × 2,000² × 2 bytes, 6 GB per document, and is not stored).
Gradient forward path. The readout runs under inference_mode;
exp341's A1 never ran for lack of a differentiable path. A second
forward that builds the same prefix and tails but keeps the graph, bf16
autocast with the target logit accumulated in fp32, verified against the
production readout on three proteins before use. This is the main
engineering risk in the plan. Fallback if it slips: analysis 2's
attribution step ranks neurons by ablation of a shortlist instead of by
gradient, and analysis 3 uses activation patching only, which needs no
gradients.
Component interventions. Zero and mean ablation of single heads
(exp341 ablates whole attention sublayers); clamping of named neurons to
a value; patching of a residual slice at chosen (layer, token) from a
donor document.
Teacher-forced full-document pass. The structure section is fed as
in training, so activations at contact-verb, contact-arg1 and
contact-arg2 tokens exist for every emitted contact, and the next-token
distribution is available after every emitted statement.

4.5 Storage and compute

Per-token neuron activations are 196,608 × 2 bytes = 393 KB. S-train at
about 2,000 tokens per document is roughly 4M tokens, so a dense dump of
everything would be 1.6 TB; it is not made. The histogram sink is
196,608 × 256 × 4 bytes = 200 MB; the example buffers about 240 MB; the
50-document dense dump about 40 GB.
Residual dumps for probes: 25 boundaries × 2048 × 2 bytes = 100 KB per
token. Every other boundary and only seq-aa tokens of S-train (about
600k tokens): about 16 GB per model.
Forward throughput measured in exp341: 5,933 tokens/s on one L40S
(g6e.xlarge), 4,150 on the panel runs. One pass over S-train is under 20
minutes; the budget goes to interventions and gradients, not clean
passes.
Placement: the AWS pattern exp341 used (g6e.xlarge L40S and g6.xlarge L4
in us-east-2, outputs under s3://marinfold-exp91-usw2/). Dumps over
10 GB stay in the bucket region; the > 10 GB cross-region rule in
AGENTS.md applies to moving them. This workstation has no GPU
(/dev/nvidia* absent), so harness development also runs on a cloud L4.

5. The three analyses

Analysis 1: representations by layer

Question. No lens signal reaches the query before boundary 12, yet
layers 0-2 are essential and layers 3-10 cost 0.03-0.20 each. What is in
the residual at seq-aa tokens across layers, and how is the residue
index represented so that separation can be computed from two tokens
modulo 2000?

1.1 Index geometry (CPU, first)

Take the 2,000 embedding rows for <p0>..<p1999> and the 2,000
unembedding rows. PCA; then treat each embedding dimension as a function
of the index n and take its discrete Fourier transform. Report the
spectrum's concentration (fraction of variance in the top 5
frequencies) and the frequencies. Nanda et al.'s modular-addition
circuit and Zhou et al.'s Fourier features in pretrained LLMs are the
templates; a circle or a helix of a few frequencies is the expected
form.
Translation invariance: the embeddings' Gram matrix should be close to
circulant if the model treats index as position on a ring. Report the
fraction of Gram variance explained by its circulant projection.
Wrap test on S-synthetic: the same proteins placed so the chain
crosses 1999 to 0, against placements that do not. R-precision on
wrap-crossing pairs versus non-crossing pairs at matched separation.
The corpus has wrapping documents at a rate near L/2000 per document, so
the model has seen them; the question is whether the computation is
ring-shaped or whether wraps are handled worse.
Separation prior at the query: decompose the logit-lens output over
candidates <p_j> at <contact> <p_i> into a term that depends only on
(j − i) mod 2000 and a remainder, per boundary. The boundary where the
remainder overtakes the prior is where content enters the partner
decision; analysis 3 patches there.
Offset and shuffle variance: on 50 S-eval proteins, 8 offsets × 8
shuffles, decompose the variance of P(contact) per pair into offset,
shuffle and residual. exp341 averages 4 seeds that vary both; this says
which one the averaging buys.

1.2 Sequence-section probes

Linear probes on the residual at each stored boundary, fit on S-train
seq-aa tokens (one tenth held out), tested on S-eval, repeated on
exp232:

Chain assembly. From residue i's token, predict the amino acid at
i+1, i+2, i+3, i−1, i−2, i−3 (six 20-way probes). The bag is shuffled,
so the model must have attended to <p(i+1)>'s statement to know this.
The boundary where this becomes decodable is where the chain exists
inside the model. Expected from exp262 and the B1 ablations: by boundary
2 or 3.
Local structure. DSSP 3-state; burial (regression on neighbour
count); contact count; Pfam domain membership (a probe that reaches a
family label from one residue's residual is reading family identity,
which bears on exp341's memorization results).
Pairwise. A rank-32 bilinear probe h_iᵀ U Vᵀ h_j on pairs of
seq-aa residuals, classifying contact and regressing log degree
(exp341 plan section 6.1, never run). Separation-matched negatives, so
the probe cannot pass on separation alone. If contact is decodable from
the sequence section at boundary 8 while the query lens is at chance,
the information exists before the query and the late layers transport
it; if not, the late attention layers compute it.
Geometry. A bilinear probe regressing CA-CA distance (log, capped at
30 Å) from residual pairs at the best boundary from step 3. Then two
tests of whether the decoded quantity behaves like a metric: the
triangle-inequality violation rate over residue triples, and, per
protein, classical MDS of the probe's distance matrix into 3 dimensions
followed by TM-align against the true structure. "Two Stages of
Folding" reports distance-probe R² near 0.9 in late trunk blocks of
ESMFold, OpenFold and Boltz-1, which hold explicit pair
representations; whether a decoder with no pair representation reaches
anything like it is the open point. A rank-32 bilinear form can only
express distances through a 32-dimensional projection of each residue,
so the rank is also swept (8, 32, 128).

Controls. Random-init model; shuffled labels; for the pairwise and
distance probes, a separation-only baseline (predict from |i − j| alone).
Selectivity reported for every probe.

Finding bar. A boundary-by-boundary curve for each quantity with
controls, on two models. For step 4, a TM-score distribution: a median
above 0.5 from a linear distance probe is a representation finding on its
own; a flat curve says the model never forms a global geometry, which is
consistent with the local Jacobian and would itself be the finding.

Cost. Residual dumps of seq-aa tokens at every other boundary for
S-train and S-eval on two models (about 35 GB in-region); 1.1 is CPU and
one readout of 50 × 64 documents; probe fitting a few GPU-hours for the
bilinear ones (subsample to 2M pairs per boundary with all positives
kept). About 10 L40S-hours.

Analysis 2: the neuron audit

Question. What do the 196,608 MLP neurons respond to, how many have
one job, and do the ones with a job do it?

2.1 Collection

One teacher-forced pass over S-train (about 4M tokens), writing per
neuron: a 256-bin histogram, the top-50 and bottom-50 examples with
(document, token index, value), and per-role mean and 99.9th percentile.
The dense dump of 50 documents supports re-analysis without a rerun. Both
poles are kept throughout; Goodfire's audit found the negative pole
carries as much meaning as the positive, and SwiGLU products have a
double-negative regime (negative gate × negative up) that a positive-only
analysis hides.

2.2 Label alignment

Join the top and bottom examples to the label table (4.3). For each
(neuron, pole, label) compute precision of the label among the strongest
200 examples against the label's base rate in the same token role, and
AUROC over the histogram tail. Report, per layer:

the fraction of neuron poles whose strongest examples are at least 80%
one token role;
within a role, the fraction with a single dominant label at 3× base
rate or more, two related labels, or none;
the number of distinct labels found by at least one neuron, and which.

The rates are reported at three threshold settings (purity 70/80/90%,
enrichment 2/3/5×), not one, because they move with the thresholds.
Goodfire's 90% one-job figure for Qwen3-32B is a qualitative comparison
only; the label vocabulary and the domain differ. The content of the
result is the list of labels found and the layers they sit in.

Then the model-written pass on 3,000 neurons, sampled with weight toward
layers 14-23 where the contact work is, plus 300 random-init neurons run
through the same pipeline (Heap et al.'s check). For each pole: 40
strongest examples rendered as (protein, residue, ±5 neighbours, DSSP,
burial, separation, role, Pfam) go to Claude, which writes a one-line job.
The test is a held-out discrimination: 20 further top examples and 20
tokens drawn from the same role at random, shown without values; the
explainer predicts which are top examples; pass is AUROC at least 0.8. At
roughly 5,000 input tokens per pole, 6,600 poles is about 33M input
tokens, well under $100 at Sonnet 5.5 rates.

2.3 Causal tests

Three forms, from cheap to specific:

Strong versus weak. Over S-eval readouts, zero every neuron
activation below its layer's 75th percentile of |value| and score
R-precision; separately zero the top 7%. Goodfire's ConvNeXt numbers
(72% to 71% accuracy from the weakest 75%; to 4% from the strongest 7%)
set the expectation; the contacts-v1 numbers say how concentrated the
computation is.
Job tests. For 100 neurons whose job predicts a readout effect
(examples: "arg2 candidates that are strands when arg1 is a strand",
"separation 3 to 5 from the already-emitted partner", "residues within
5 of the C-terminus"), clamp the neuron to its bottom value and measure
the change in P(contact) on the pairs the job names versus pairs it does
not. A neuron passes if the named pairs move at least 3× the others.
Attribution graphs on raw neurons. For 12 hand-chosen predictions
(a β-hairpin contact, a helix-helix packing contact, a long-range
contact in a protein over 500 residues, a wrong prediction with high
probability, and their counterparts in eval-denovo), first-order
attribution of every neuron in layers 12-23 to the logit of the chosen
<p_j> at the <contact> <p_i> query (activation × gradient, Goodfire's
method on Qwen3-32B and Kimi K3 without transcoders), ranked, then
verified by ablating the top 20 against 20 random neurons of the same
layers. Follow the top nodes back two hops by the same rule. Each graph
is a per-contact circuit sketch; the comparison across cases says
whether the same neurons recur (a contact-type circuit) or every contact
uses its own (a lookup).

2.4 SAEs as a bounded comparison

TopK SAEs (k = 32, width 32,768) on the residual at boundaries 2, 15 and
21 (chain assembly, contact onset, partner selection), trained on the
S-train residual dump from analysis 1. A plain torch implementation keeps
the transformers 4.x pin exp341 chose over nnsight. Each SAE is evaluated
on:

fraction of variance unexplained on held-out tokens;
the label table at the same thresholds as 2.2, latent by latent;
the near-copy rate: for each latent, the largest correlation of its
activation series with any single neuron in the same or earlier layers;
contact-map prediction from latent pairs against neuron pairs against
the analysis 1 bilinear probe (the Reticular evaluation);
for 20 latents with clean labels, clamp and read out, as in 2.3.

Rule fixed in advance. SAE work continues past this step only if (2)
finds a label no neuron found at the same threshold, or (5) moves the
readout where no neuron does. Otherwise the result is the copy rate and
the step closes.

Finding bar for analysis 2. The per-layer label inventory with its
random-init baseline; the strong-versus-weak ablation curve; at least 50
of the 100 job tests passed or a statement of which class fails;
attribution graphs where the top-20 ablation moves P(<p_j>) by at least
5× the random control, with recurring neurons named; the SAE copy rate.

Cost. Collection under 1 L40S-hour plus the dense dump; alignment on
CPU; auto-labelling through the API; job tests 100 × readout on S-eval,
about 2 L40S-hours; attribution graphs 12 cases × gradients through a
1.47B model on a 2,000-token document, under 3 L40S-hours; SAEs 1-2
L40S-hours each. About 15 L40S-hours.

Analysis 3: the contact-decision circuit

Question. Which of the 768 heads do which of the jobs the grammar
requires, what is the algorithm that turns a <contact> <p_i> query into
a partner, and what do already-emitted contacts contribute?

3.1 Head census

For each head, on 300 S-train documents under teacher forcing, attention
mass scores reduced on the fly:

Score	From	To	Null
previous-token	any token	offset 1	exp262 `probe_attention.py` computes this on exp199
statement-binding	`seq-aa`	its own `seq-pos`	1 / (tokens so far)
chain-neighbour(k)	`seq-aa` of residue i	tokens of residues i ± k, k = 1..4	mass to a random residue's tokens
partner-lookup	`contact-arg1` token `<p_i>` (and the query `<p_i>` in the readout)	sequence tokens of residues j that are true contacts of i	mass to non-contact residues at the same separation
Rao contact	`seq-aa` i	`seq-aa` j, symmetrised and APC-corrected	logistic regression over heads against true contacts, as Rao et al. 2021
emitted-contact	`contact-arg1` `<p_i>`	earlier statements containing `<p_i>` or a true partner of i	mass to earlier statements not containing them
sink	any	`<begin_statements>`, `<begin_sequence>`, `<contacts-v1>`	uniform

The Rao score is weaker in a decoder than in ESM: attention is causal and
the bag is shuffled, so for half of all pairs only one direction exists.
Symmetrisation over the 300 documents' shuffles recovers both directions
for most pairs; the score is reported with that caveat.

Causal half: for the top 8 heads on each score, ablate singly and as a
group against 8 random heads, R-precision by band. For partner-lookup
heads, path patching: patch the sequence-section residual of the true
partner j from a document in which j is mutated and measure the drop in
P(<p_j>) at the query through each candidate head.

3.2 The partner decision

Where content enters. Analysis 1.1 step 4 gives the boundary at
which the partner logits stop being a separation prior. Activation
patching: patch the query residual at that boundary from a donor
document with the same length and offset but a different protein; if
P(<p_j>) follows the donor, the partner is decided by then.
How content enters. Path patching through the partner-lookup heads
from residue j's sequence tokens: at which source boundary does
patching residue j's residual stop changing the query's choice of j?
With analysis 1.2 step 3 this says whether the partner is read from a
pair feature already present in the sequence section or built at the
query by comparing i's and j's residuals (an attention-implemented
bilinear match).
The layer-21 dip. The logit lens falls from 0.506 at boundary 19 to
0.412 at 21 and recovers by 24 (exp341). Decompose the layer 20 and 21
MLP outputs into the unembedding's row space and its complement; if the
dip is a large write orthogonal to the position tokens (a working
variable, later removed), the analysis 2 neurons of layer 21 that write
it will have jobs that are not partner identity.

3.3 What emitted contacts contribute

The structure section is autoregressive, so under teacher forcing the
model sees true contacts before predicting the next. The pairwise readout
never shows it any. Measure, on 200 S-train and all S-eval documents:

Conditioning gain. The next-contact distribution after k emitted
true statements, k = 0, 10, 50, 200, all: R-precision of the remaining
contacts as a function of k, and by band. If the model's ranking of the
remaining contacts improves with k, it is doing in-context refinement
from its own output, which exp321's null-sequence guidance treated as a
sequence-independent prior to subtract. The two readings are
distinguished by whether the gain is specific to the true partners or a
generic shift toward already-touched residues.
Duplicate avoidance. P(re-emitting an already-emitted pair, in
either order) across the section; the emitted-contact heads from 3.1
and the neurons whose top examples are contact-verb tokens late in
the section; ablate them and measure the re-emission rate. A
set-membership mechanism over up to 2,700 emitted statements is a kind
of circuit the grammar requires and few language-model circuits have
shown.
Symmetry. Each pair is emitted in a coin-flipped order. Compare
P(<p_j> | <contact> <p_i>) with P(<p_i> | <contact> <p_j>) over
all pairs; report the asymmetry distribution and whether it shrinks
across boundaries under the lens. If a late component enforces it, the
analysis 2 graph for an asymmetric pair should show which.

Finding bar. A head atlas (768 rows, one column per score, with
ablation costs for the top heads); a diagram of the decision path with
patching numbers on each edge; a conditioning-gain curve; a measured
re-emission rate with and without the dedup components.

Cost. Census: one teacher-forced pass over 300 documents with
per-head reductions, under 2 L40S-hours. Ablations: 7 classes × 9
conditions × a clean readout over S-eval (117k tokens) is small; path
patching on 30 proteins about 5 L40S-hours; 3.2 patching on 30 proteins
at a few boundaries about 8; 3.3 is two teacher-forced passes plus the
dedup ablations. About 20 L40S-hours.

6. What would be interesting

Ranked by how much each would change the picture of the model, with the
analysis that delivers it:

A linear distance probe whose MDS reconstructs folds at TM-score above
0.5 (1.2 step 4): the decoder holds a global geometry despite the local
Jacobian.
The opposite: pairwise contact is not decodable from the sequence
section at any boundary and is built at the query by the layer-18-20
heads (1.2 step 3 with 3.2 step 2): the model has no pair
representation and does retrieval at emission time.
Residue index on a Fourier ring with wrap pairs scored as well as
others (1.1): a modular-arithmetic representation learned from protein
data.
A duplicate-avoidance circuit with a measured re-emission rate under
ablation (3.3 step 2).
A conditioning gain that is partner-specific (3.3 step 1): the model
refines its contact map from its own emitted contacts.
Named late-layer neuron jobs that pass clamp tests (2.3 step 2), such
as strand-pairing or helix-packing detectors, and whether they recur
across attribution graphs.
The SAE near-copy rate (2.4) on a protein model.

7. Order and decision points

harness additions ──┬── analysis 1 (index geometry first, CPU)
                    ├── analysis 2 (collection, labels, causal tests, SAEs)
                    └── analysis 3 (census, then patching, then emitted contacts)

The three run in parallel after the harness; they share dumps but not
results, except that 3.2 step 1 uses 1.1 step 4's boundary and 3.3 step 2
uses 2.1's example buffers. Decision points, each with the rule written
here:

After 1.2 step 3: if contact is decodable from seq-aa pairs by boundary
10 with selectivity above 0.2, 3.2 step 2 focuses on transport; if not,
on query-time comparison.
After 2.2: if fewer than half of the poles in layers 14-23 have one job
at the middle threshold setting, 2.3 step 2 samples from the two-job and
unexplained sets as well, and 2.4 is given more weight (the case where
SAEs could earn their place).
After 2.4: continue SAE work only under the rule in 2.4.
After 1.2 on exp232: if a probe curve differs from exp277's by more than
its bootstrap CI at most boundaries, the analysis 2 and 3 results are
reported as properties of exp277, and the writeup says so.

8. Guardrails

Every probe reports selectivity against shuffled labels and the
random-init model. Every auto-labelled neuron set reports the random-init
pass rate.
Probes and top examples show presence; only ablation, clamping and
patching support "does". The writeup keeps the two vocabularies apart.
Interventions are compared against the same number of random components,
always. Patching results are reported with the corruption and the metric
named, since Zhang and Nanda (2023) showed they move with both.
Readout differences are computed in fp32 where they are small: exp341 A2
found bf16 noise far from the contact 0.6 to 5.2 times the fp32 signal.
Strong activations over weak ones; both poles; superposition is the last
explanation offered, after role mixing, two related jobs, and tool error.
No eval-test reads. eval-denovo (19 proteins) is reported but never
carries a claim alone.
Thresholds in 2.2 and the decision rules in section 7 are fixed before
the runs.

9. Budget

L40S-hours by analysis, from exp341's measured throughput and this plan's
token counts; not yet measured:

Item	L40S-hours
Harness, checks, dumps	10
Analysis 1	10
Analysis 2 (incl. SAEs)	15
Analysis 3	20
Sum	55
With reruns (×1.5)	about 80

Plus API calls for about 6,600 neuron-pole explanations and tests, and
in-region S3 for about 100 GB at peak, deleted once CSVs are committed.
exp341's Track A spent 42 GPU-hours on its two A2 Jacobian runs alone, so
this is of the order of one exp341 track.

10. Not in this plan

Cut to hold the cap at three analyses. Each could be a follow-up issue.

Across models (CKA and cross-model patching between exp277, exp232
and exp199; memorization traces and an exposure probe on exp341's
removed/kept panel). Analysis 1's replication on exp232 keeps the
cheapest part. The exposure probe would need exp232 as the control for
removed-versus-kept set differences, since exp341 found removed
structures score 0.11-0.15 above kept ones for reasons still unexplained.
Across checkpoints. Within exp277 only step-213072 (pre-cooldown,
Levanter native on S3) exists besides the final; the stopped
weighted-mixture run kept steps 14,520 to 58,080. Each needs a
Levanter-to-HF conversion this plan has not verified tooling for.
Categorical Jacobian of the sequence section (exp341 A1, never run):
whether the sequence LM stores Potts-like couplings, as Zhang et al.
2024 found in ESM-2. Needs the gradient path; cheap once it exists; the
first technique to add if the cap were four.
Crosscoders for model diffing (exp199 against exp232), which would
give aligned features across two models that share no neuron indexing.
Training-data attribution (TRAK or influence functions), which
would attach exp341's A4 exposure effect to specific training
structures. Very high compute at 1.5B.
Causal abstraction / DAS for the separation computation, once 1.1
proposes a specific ring algorithm.

11. Deliverables

This directory, renamed to the issue number once filed, with a README
in the marinfold_experiment frontmatter format.
harness_ext.py importing exp341's harness; one driver per analysis
step; committed CSVs: index_geometry.csv, probe_curves.csv,
neuron_labels.parquet, neuron_job_tests.csv, attribution_cases/,
sae_eval.csv, head_atlas.csv, circuit_patching.csv,
conditioning_gain.csv; plots for each.
A writeup per analysis in exp341's style (what was found first, then
what ran, results, limits, files), and one synthesis: the layer map of
section 3 filled in with mechanisms.
A history file under history/runs/ for anything that logs to W&B.

12. References

Each entry names the section(s) of this plan that rely on it. Issue
numbers link to Open-Athena/MarinFold; experiment directories are on
main unless a branch is named.

12.1 MarinFold experiments and code

exp341, contacts-v1 interpretability (functional).
Issue #341.
Code and writeups on branch exp/341-contacts-v1-interpretability
under experiments/exp341_evals_contacts_v1_interpretability/:
README.md, research_plan.md, track_a_writeup.md (A2 contact
Jacobian, A3 removed-vs-kept panel, A4 exposure), b1_writeup.md
(layer ablations, logit and tuned lens), harness.py. Outputs under
s3://marinfold-exp91-usw2/exp341/. Cited in §1, §3, §4.4, §4.5, §5
(all three analyses), §7, §9, §10.
exp341 literature survey. compass_artifact_wf-7fc4e0c0-1bb6-5cb3-b490-e2371939091f_text_markdown.md
in the exp341 working directory (untracked). Source of the method
summaries behind §2.1 and §10; the primary papers are listed in §12.2
and §12.3.
exp262, NoPE and token smearing.
Issue #262,
experiments/exp262_models_nope_token_smearing/ (branch
exp/262-nope-token-smearing). Phase 0 attention census
(probe_attention.py: layer-1 previous-token heads, 10 of 768 heads
with offset-1 mass above 0.30) and the "order is nuisance" summary
that cites exp: order-marginalized (soft-target) supervision for contacts-v1 #201. Cited in §3, §5 Analysis 3.1.
exp277, single-MPNN pilot (the model under study).
Issue #277,
experiments/exp277_models_single_mpnn_pilot/. Checkpoint
contacts-v1-exp277-m2-p06-full-epoch-1.5B at step 266344, registered
in marinfold/marinfold/MODELS.yaml. Cited in §4.1 and throughout.
exp232, decontaminated contacts-v1 sweep (replication model).
Issue #232,
experiments/exp232_sweep_cv1_decontam/. Checkpoint
contacts-v1-exp232-m2-p06-train-1.5B. Cited in §4.1, §5 Analysis
1.2, §7, §10.
exp199, contacts-v1 on AFDB plus ESM Atlas.
Issue #199,
experiments/exp199_optimize_contacts_v1_afdb_esm/. The model
exp262's census ran on; the crosscoder diff candidate in §10. Cited in
§4.1, §5 Analysis 3.1, §10.
exp201, order-marginalized (soft-target) supervision for
contacts-v1.
Issue #201
(no directory on main; the 77% figure is quoted from the exp262 and
exp211 READMEs). Cited in §3.
exp218, ProteinGym zero-shot fitness.
Issue #218,
experiments/exp218_evals_proteingym_zero_shot_fitness/. Sequence
section is worth about ESM2-35M. Cited in §3.
exp224, circular permutation of DsbA.
Issue #224,
experiments/exp224_evals_does_marinfold_fold_a_circularly_permuted/.
Cost 0.093 R-precision overall, 0.121 on pairs whose separation
changed. Cited in §3, §5 Analysis 1.1.
exp321, null-sequence contrastive guidance.
Issue #321,
experiments/exp321_evals_null_sequence_contrastive_guidance_for_contacts/.
The generic structure-section prior that Analysis 3.3 separates from
partner-specific conditioning. Cited in §5 Analysis 3.3.
exp91, ESMFold2 structure corpus and the marinfold-exp91-usw2
bucket. Issue #91.
Cited in §4.5 for output placement only.
exp247, protein properties that explain per-protein accuracy.
Issue #247,
experiments/exp247_evals_protein_property_analysis/. Per-protein
property table (length, contact order, long-range fraction, secondary
structure via biotite, training-set proximity) for the 314 FoldBench
monomers; the ready-made source of S-eval property columns in §4.3.
contacts-v1 document format.
marinfold/marinfold/document_structures/contacts_v1/README.md and
.dev/document-format-overview.md. Cited in §2.3, §4.2, §5 Analysis
1.1 and 3.3.
pyconfind. T. O'Donnell, https://github.com/timodonnell/pyconfind;
Python port of ConFind: J. Zhou and G. Grigoryan, "Rapid search for
tertiary fragments reveals protein sequence-structure relationships",
Protein Science 24:508-524 (2015),
https://doi.org/10.1002/pro.2610. Defines the side-chain contact
degree the structure section is built from. Cited in §4.3.

12.2 Interpretability methods

Goodfire neuron audit. N. Cammarata (Goodfire; language-neuron
research by C. Tigges), "Neural networks are already clean", 2026
(the page carries no date; accessed 2026-10-26),
https://already-clean.goodfire.com/. Both-pole neuron audits, the
strong-activation ablation on ConvNeXt, the 90% one-job figure on
Qwen3-32B, the 88% SAE near-copy rate, attribution graphs on raw
neurons. Cited in §2.1, §2.2, §5 Analysis 2 (all steps), §6.
Attribution graphs. E. Ameisen, J. Lindsey, et al., "Circuit
tracing: revealing computational graphs in language models",
Transformer Circuits Thread, March 2025,
https://transformer-circuits.pub/2025/attribution-graphs/methods.html;
J. Lindsey et al., "On the biology of a large language model", same
thread, https://transformer-circuits.pub/2025/attribution-graphs/biology.html.
The transcoder graphs that Goodfire reproduced on raw neurons. Cited
in §2.1, §5 Analysis 2.3.
TopK sparse autoencoders. L. Gao et al., "Scaling and evaluating
sparse autoencoders", ICLR 2025, arXiv:2406.04093,
https://arxiv.org/abs/2406.04093. The SAE variant trained in Analysis
2.4. Cited in §5 Analysis 2.4.
SAE negative results. L. Smith, S. Rajamanoharan, A. Conmy, C.
McDougall, J. Kramár, T. Lieberum, R. Shah, N. Nanda, "Negative results
for SAEs on downstream tasks and deprioritising SAE research",
Google DeepMind, March 2025,
https://www.alignmentforum.org/posts/4uXCAJNuPKtKBsi28/negative-results-for-saes-on-downstream-tasks.
S. Kantamneni et al., "Are sparse autoencoders useful? A case study in
sparse probing", arXiv:2502.16681, https://arxiv.org/abs/2502.16681.
T. Heap, T. Lawson, L. Farnik, L. Aitchison, "Sparse autoencoders can
interpret randomly initialized transformers", arXiv:2501.17727
(retitled "Automated interpretability metrics do not distinguish
trained and random transformers" for ICLR 2026),
https://arxiv.org/abs/2501.17727. Heap et al. is the source of the
random-init control in Analysis 2.2. Cited in §2.1, §5 Analysis 2.2
and 2.4, §8.
Probe selectivity. J. Hewitt and P. Liang, "Designing and
interpreting probes with control tasks", EMNLP 2019, arXiv:1909.03368,
https://arxiv.org/abs/1909.03368. Shuffled-label and random-init
controls for every probe. Cited in §1, §4.1, §5 Analysis 1.2, §8.
Logit lens. nostalgebraist, "Interpreting GPT: the logit lens",
LessWrong, August 2020,
https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens.
The readout exp341 B1 used; its layer-21 dip is Analysis 3.2 step 3.
Cited in §3, §5 Analysis 3.2.
Fourier number representations. N. Nanda, L. Chan, T. Lieberum,
J. Smith, J. Steinhardt, "Progress measures for grokking via
mechanistic interpretability", ICLR 2023, arXiv:2301.05217,
https://arxiv.org/abs/2301.05217. T. Zhou, D. Fu, V. Sharan, R. Jia,
"Pre-trained large language models use Fourier features to compute
addition", NeurIPS 2024, arXiv:2406.03445,
https://arxiv.org/abs/2406.03445. Templates for the position-index
spectrum in Analysis 1.1. Cited in §5 Analysis 1.1, §6.
Activation patching. K. Meng, D. Bau, A. Andonian, Y. Belinkov,
"Locating and editing factual associations in GPT", NeurIPS 2022,
arXiv:2202.05262, https://arxiv.org/abs/2202.05262. S. Heimersheim
and N. Nanda, "How to use and interpret activation patching",
arXiv:2404.15255, https://arxiv.org/abs/2404.15255. F. Zhang and N.
Nanda, "Towards best practices of activation patching in language
models: metrics and methods", ICLR 2024, arXiv:2309.16042,
https://arxiv.org/abs/2309.16042. Zhang and Nanda is the reason §8
requires the metric and corruption to be named with every patching
result. Cited in §4.4, §5 Analysis 3.2, §8.
Path patching. K. Wang, A. Variengien, A. Conmy, B. Shlegeris, J.
Steinhardt, "Interpretability in the wild: a circuit for indirect
object identification in GPT-2 small", ICLR 2023, arXiv:2211.00593,
https://arxiv.org/abs/2211.00593. N. Goldowsky-Dill, C. MacLeod, L.
Sato, A. Arora, "Localizing model behavior with path patching",
arXiv:2304.05969, https://arxiv.org/abs/2304.05969. Cited in §5
Analysis 3.1 and 3.2.
Crosscoders (deferred). J. Lindsey, A. Templeton, J. Marcus, T.
Conerly, J. Batson, C. Olah, "Sparse crosscoders for cross-layer
features and model diffing", Transformer Circuits Thread, October
2024, https://transformer-circuits.pub/2024/crosscoders/index.html.
Cited in §10.
Training-data attribution (deferred). S. M. Park, K. Georgiev, A.
Ilyas, G. Leclerc, A. Madry, "TRAK: attributing model behavior at
scale", ICML 2023, arXiv:2303.14186,
https://arxiv.org/abs/2303.14186. Cited in §10.
Distributed alignment search (deferred). A. Geiger, Z. Wu, C.
Potts, T. Icard, N. Goodman, "Finding alignments between interpretable
causal variables and distributed neural representations", CLeaR 2024,
PMLR 236:160-187, arXiv:2303.02536, https://arxiv.org/abs/2303.02536. Cited in §10.
Centered kernel alignment (deferred). S. Kornblith, M. Norouzi, H.
Lee, G. Hinton, "Similarity of neural network representations
revisited", ICML 2019, arXiv:1905.00414,
https://arxiv.org/abs/1905.00414. Cited in §10.

12.3 Protein-model interpretability

Attention-based contact readout. R. Rao, J. Meier, T. Sercu, S.
Ovchinnikov, A. Rives, "Transformer protein language models are
unsupervised structure learners", ICLR 2021, bioRxiv
2020.12.15.422761, https://doi.org/10.1101/2020.12.15.422761. The
symmetrised, APC-corrected logistic regression over heads in Analysis
3.1. Cited in §5 Analysis 3.1.
Categorical Jacobian (deferred). Z. Zhang, H. K. Wayment-Steele,
G. Brixi, H. Wang, D. Kern, S. Ovchinnikov, "Protein language models
learn evolutionary statistics of interacting sequence motifs", PNAS
121(45):e2406285121 (2024),
https://doi.org/10.1073/pnas.2406285121. exp341's never-run A1 and
the first addition if the cap were four. Cited in §10.
InterPLM. E. Simon and J. Zou, "InterPLM: discovering
interpretable features in protein language models via sparse
autoencoders", Nature Methods 22:2107-2117 (2025),
https://doi.org/10.1038/s41592-025-02836-7, arXiv:2412.12101; code
https://github.com/ElanaPearl/InterPLM. Domain and motif detectors
in ESM-2; the concept-alignment protocol Analysis 2.2 follows. Cited
in §4.3, §5 Analysis 2.2.
Contact circuits in ESM-2. J. Nainani, B. Reimer, C. Watts, D.
Jensen, A. Green, "Mechanistic evidence that motif-gated domain
recognition drives contact prediction in protein language models",
bioRxiv 2025.08.22.671739,
https://doi.org/10.1101/2025.08.22.671739. Cited in §4.3, §5
Analysis 3.
SAEs on the structure model. N. Parsan, D. Yang, J. Yang
(Reticular), "Towards interpretable protein structure prediction with
sparse autoencoders", ICLR 2025 GEM workshop, arXiv:2503.08764,
https://arxiv.org/abs/2503.08764. The contact-map evaluation of SAE
latents in Analysis 2.4. Cited in §5 Analysis 2.4.
Two stages of folding. J. Lu, J. Brinkmann, S. Huber, A. Mueller,
Y. Belinkov, D. Bau, C. Wendler, "Two stages of folding: convergent
mechanisms in AI protein folding trunks", arXiv:2602.06020,
https://arxiv.org/abs/2602.06020. Linear distance probes reaching
R² near 0.9 in ESMFold, OpenFold and Boltz-1; the comparison point for
Analysis 1.2's distance probe. Cited in §5 Analysis 1.2, §6.
ESM-2 and the ESM Atlas. Z. Lin et al., "Evolutionary-scale
prediction of atomic-level protein structure with a language model",
Science 379:1123-1130 (2023),
https://doi.org/10.1126/science.ade2574; Atlas at
https://esmatlas.com. Cited in §2.3, §3.

12.4 Data and labels

AlphaFold Database. M. Varadi et al., "AlphaFold Protein Structure
Database in 2024: providing structure coverage for over 214 million
protein sequences", Nucleic Acids Research 52:D368-D375 (2024),
https://doi.org/10.1093/nar/gkad1011. Source of S-train and its
pLDDT. Cited in §2.3, §4.2.
ProteinGym. P. Notin et al., "ProteinGym: large-scale benchmarks
for protein fitness prediction and design", NeurIPS 2023 Datasets and
Benchmarks, https://proteingym.org. The benchmark behind exp218's
ESM2-35M comparison. Cited in §3.
DSSP. W. Kabsch and C. Sander, "Dictionary of protein secondary
structure: pattern recognition of hydrogen-bonded and geometrical
features", Biopolymers 22:2577-2637 (1983),
https://doi.org/10.1002/bip.360221211. Secondary-structure and
accessibility labels in §4.3.
Pfam and InterPro. J. Mistry et al., "Pfam: the protein families
database in 2021", Nucleic Acids Research 49:D412-D419 (2021),
https://doi.org/10.1093/nar/gkaa913. M. Blum et al., "InterPro: the
protein sequence classification resource in 2025", Nucleic Acids
Research 53:D444-D456 (2025), https://doi.org/10.1093/nar/gkae1082.
Domain boundaries by UniProt accession for S-train. Cited in §4.3.
TM-align. Y. Zhang and J. Skolnick, "TM-align: a protein structure
alignment algorithm based on the TM-score", Nucleic Acids Research
33:2302-2309 (2005), https://doi.org/10.1093/nar/gki524. Scores the
MDS reconstruction from the distance probe in Analysis 1.2.

exp: what contacts-v1 computes (representations, neurons, circuit) #372

Description

Activity

jsilter commented on Oct 27, 2026