[timodonnell]

Train exp277's 1.47B contacts-v1 recipe on 162,574,724 ProteinMPNN redesign documents, excluding native-sequence training documents. The 5e-4 and 1e-3 parents completed exactly two epochs (47,842,378 packed examples, 373769 optimizer updates, final checkpoint 373768). Both one-epoch cooldown forks completed at step 186884. The diverged 2e-3 run was cancelled at approximately 1.369 epochs and is excluded from automatic recovery. No training or evaluation jobs remain active.

Completed checkpoint	Step	Final LM loss	eval-val R (all / long)	eval-denovo R (all / long)
5e-4, two epochs	373768	3.01328707	0.52470 / 0.50813	0.67385 / 0.63709
1e-3, two epochs	373768	3.04156637	0.47779 / 0.45833	0.66229 / 0.61415
5e-4, one epoch, GPU	186884	3.03571987	0.46301 / 0.44184	0.66515 / 0.62680
5e-4, one epoch, TPU	186884	3.03581691	0.46018 / 0.44409	0.66954 / 0.64291
Current full-data default, exp277	266344	2.98440742	0.55510 / 0.53706	0.69640 / 0.67497

The final 5e-4 checkpoint is strongest among these redesign-only runs on natural eval-val proteins: +0.06170 all-range R-precision over its GPU one-epoch fork and −0.03040 versus the full-data default. On eval-denovo its gap to the default is −0.02255; its +0.00431 difference versus the TPU one-epoch fork is a practical tie. These are point estimates; training exposures/schedules differ and eval-denovo has only 19 proteins. A matched redesign-only sequence-KNN null is unavailable.

Both final checkpoints were scored on 97 eval-val + 19 eval-denovo proteins with 100 randomized-prompt rollouts each: temperature 1, top-p 0.95, top-k disabled, seed 0, initial budget 6L+128, approved prefix-preserving continuation within 8192 tokens. Live-contact votes use exp89's fixed candidate universe and metrics. All 23,200 production rollouts completed with zero unfinished samples or extensions. Two independent regional H100 jobs (/bizon/exp364-contact-eval-lr5e-4-a08, /bizon/exp364-contact-eval-lr1e-3-a08) succeeded around November 2, 14:08 EST, with zero failures/preemptions and zero model egress. No eval-test or legacy inference was performed.

Per-protein scores, timings, bootstrap intervals, subgroup summaries, source/file manifests, validation curves and the final comparison figure are committed and publicly available. Final comparison plot. October 21/29 snapshots are preserved separately; October 29 parent scores describe earlier checkpoints.

Checkpoint handoff for #373: use contacts-v1-exp364-mpnn-only-2ep-lr5e-4-1.5B-step-373768. Its verified HF-format export and co-located tokenizer are at:

s3://marin-us-east-02a/MarinFold/exp364_models_mpnn_only_lr_sweep/runs/contacts-v1-exp364-mpnn-only-2ep-lr5e-4-1.5B/hf/step-373768

Exact six-file SHA-256 manifest, key lr5e-4; total 5,885,614,887 bytes, float32 on disk / bfloat16 for inference. Training run. The experiment README records the native full-state path, trained configuration and complete handoff. The separate #373 agent owns its writeup and new diversity/Neff analysis; existing aggregate votes do not replace individual rollout samples needed for that study.

The public HF checkpoint and optional MODELS.yaml entry added by this PR are the one-epoch model, contacts-v1-exp364-mpnn-only-1ep-lr5e-4-cooldown-1.5B, step 186884. The two-epoch weights are currently in CoreWeave S3 and are not yet published or registered under a public nickname. Public evaluation artifacts are available for both. Exp277 remains the default.

Operational code preserves the scientific recipe and checkpoint identity through preemptions, credential rotation and finite-loader/padding fixes. Regional verification checks every exported parameter for finiteness, tokenizer/config presence and exact native trainer/optimizer/LR counters. The approved TPU ingress was 267.73 GB; bounded TPU-checkpoint transfer for CUDA evaluation totaled 8.828 GB. CoreWeave training and final-parent evaluation stayed regional.

Validation: training/loader and padding-gradient regressions; GPU/TPU restore smokes; 37 evaluation/generation checks; three finite-export checks and full final-weight/native-counter verification; 14 registry tests and anonymous one-epoch publication checks; independent completeness/identity validation of every final result and timing row; Ruff; regenerated and inspected figures/summary slides. Latest changes are documentation and handoff details. Tracks #364; the experiment issue remains open.