\n\nHow we went from 0.184 to 0.424 — fixing randomness, diagnosing templates, and redesigning the prediction pipeline
\n
\n\nFinal LB Score: 0.424 · Deterministic · TBM + multi-seed Protenix · Generate → Rank → Select architecture
\n
The competition asks for 5 predicted 3D structures per RNA sequence. The metric takes the best TM-score among your 5 predictions — so the goal is not one perfect prediction, but five diverse, high-quality candidates.
\nWe combined two complementary methods:
\nThe 28 test sequences fall into three natural groups that required different handling:
\nTBM started at 0.184 and grew through several key iterations before becoming the backbone of the final solution.
\nThe original baseline generated 5 predictions by adding Gaussian noise to a single morphed template. The problem: all 5 predictions were nearly identical, wasting the best-of-5 metric.
\nThe fix was to use genuinely different template subsets for each prediction slot — top-4, templates 3–6, templates 5–8, even-indexed, odd-indexed. Each slot blended its assigned templates by coordinate averaging. This created real structural diversity and jumped the score to 0.233.
\nAfter generating 5 predictions, we sorted them by a physics-based pseudo-energy function so the most plausible structure became prediction 1. The energy combined bond length deviation from the ideal 5.9Å C1'–C1' distance and a steric clash penalty for atoms closer than 3.8Å.
\nRather than applying one strategy uniformly, we diagnosed each sequence individually and routed it to the strategy where it performed best. The 7 sequences with no templates at all worked best with a diverse geometry fallback (helix, extended strand, compact globule, sequence-guided, reverse helix).
\nWe replaced the raw alignment score with a composite that factors in length similarity, exact match ratio, and GC content similarity:
\nscore = (align²) × len_sim × (1 + 0.3×exact) × (0.5 + 0.5×gc_sim)\nSquaring the alignment score amplifies strong matches and penalises weak ones.
\nIncreasing TOPK from 8 to 15 and generating up to 12 candidates per sequence gave the scoring function more material to work with. We also replaced Gaussian noise with structured diversity transforms applied in rotation:
\n| Slot | \nTransform | \nDescription | \n
|---|---|---|
| 0 | \nClean | \nBest template, no noise | \n
| 1 | \nTiny noise | \nProportional to alignment uncertainty | \n
| 2 | \nHinge rotation | \nRotates one segment around a pivot | \n
| 3 | \nJitter chains | \nIndependent rigid rotation + translation per chain | \n
| 4 | \nSmooth wiggle | \nSmooth interpolated displacement field | \n
| 5+ | \nGaussian noise | \nFallback | \n
Before any architectural improvements were possible, we had to solve a more fundamental problem: the same notebook was producing scores of 0.392, 0.404, 0.409, and 0.426 on different runs. Without reproducibility, every experiment was noise.
\nA systematic audit found 12 distinct sources of randomness. The most critical:
\nCUBLAS_WORKSPACE_CONFIG set after PyTorch import — this environment variable must be set before torch is imported, or it has no effect on GPU determinismnp.random.seed() inside prediction functions with seed=len(predictions)*7+len(seq) — if a morph failed upstream and len(predictions) differed from expected, the seed changed and all downstream noise changedshortlist_templates sort with no tiebreaker — equal alignment scores had non-deterministic ordering depending on dataframe memory layoutlru_cache(maxsize=200,000) — when the cache filled, eviction order varied between runsSeven targeted changes restored full determinism:
\n# Cell 0 — before ANY import\nimport os, random\nos.environ['CUBLAS_WORKSPACE_CONFIG'] = ':4096:8'\nos.environ['PYTHONHASHSEED'] = '42'\nrandom.seed(42)\nimport numpy as np; np.random.seed(42)\nimport torch; torch.manual_seed(42)\ntorch.use_deterministic_algorithms(True, warn_only=True)\nnp.random.seed() calls replaced with isolated np.random.default_rng(seed=(ridx × 10B + slot × 10007) % 2³²) — seed is globally unique per sequence and slottid as secondary sort key in all template ranking functionslru_cache to maxsize=None0.1\n\nResult: Three consecutive submissions all scored exactly 0.392 — perfect determinism confirmed.
\n
After fixing determinism, every experiment returned 0.392. To understand why, we built a deep diagnostic that computed the TM-score of raw template coordinates before any pipeline processing.
\n| Finding | \nSequences Affected | \nConclusion | \n
|---|---|---|
| High alignment (>0.4) but template TM < 0.2 | \n10 sequences | \nSame sequence, different 3D fold — conformational isomers | \n
| Alignment < 0.3 — no template exists | \n8 sequences | \nGenuinely novel folds — no known analog in any database | \n
| morph_delta ≈ 0 for all sequences | \nAll 28 | \nPipeline is NOT destroying good templates — ceiling is the data | \n
| 9EBP — only sequence with tmpl_TM > 0.35 | \n1 sequence | \nGood template exists, already handled well by pipeline | \n
The most striking finding was 9LEL — alignment score 1.280 (near-identical sequence) but template TM = 0.020 (completely wrong structure). The template in the database folds the same sequence into a different 3D conformation. This is conformational isomerism — common in RNA, and impossible to fix by improving the morphing pipeline.
\n\n\nKey insight: The pipeline was not the problem. The training database simply does not contain structurally correct templates for 18 of 28 test sequences. These were specifically chosen for the competition because they represent novel folds. No amount of pipeline tuning overcomes this — only a better generative model can.
\n
The original pipeline pre-assigned slots before generating any predictions. It counted good TBM candidates (alignment ≥ 0.4), allocated N TBM slots, and sent the remaining 5-N slots to Protenix. The problem: if TBM slot 3 produced a better structure than Protenix slot 1, TBM slot 3 was discarded anyway. The pipeline had no quality gate.
\nWe replaced slot allocation with a generate-then-rank approach:
\nscore_structure()def score_structure(coords):\n diffs = np.linalg.norm(np.diff(coords, axis=0), axis=1)\n bond_pen = np.sum((diffs - 5.9) ** 2) # ideal C1'-C1' = 5.9Å\n dm = distance_matrix(coords, coords)\n clash_pen = np.sum(dm < 3.8) # atomic clash threshold\n return -(clash_pen * 6.0 + bond_pen * 0.3) # higher = better\nTBM candidates also receive an alignment bonus (align² × 8.0) and a length-dependent bonus, ensuring that high-confidence template morphs rank above uncertain ones.
Instead of one fixed seed with N_sample=5, Protenix runs N_sample=1 per seed with a sequence-length-dependent number of seeds:
\n| Sequence Type | \nSeed Count | \nRationale | \n
|---|---|---|
| Short (≤400 nt) | \n7 seeds | \nMore exploration — each run is fast | \n
| Medium (400–1000 nt) | \n3 seeds | \nBalanced exploration vs compute | \n
| 9MME, 9ZCC | \n2 seeds | \nChunked inference (512nt chunks, 128nt overlap, Kabsch-aligned stitching) | \n
| OUR_SEQUENCES | \n0 Protenix | \nTBM is already near-optimal | \n
| Phase | \nKey Change | \nLB Score | \nDelta | \n
|---|---|---|---|
| Baseline TBM | \nTemplate morphing + Gaussian noise | \n0.184 | \n— | \n
| Experiment B | \nStrategic template diversity (different subsets per slot) | \n0.233 | \n+0.049 | \n
| Experiment 1 | \nEnergy-based reranking of 5 predictions | \n0.237 | \n+0.004 | \n
| FINAL routing | \nPer-sequence strategy routing | \n0.242 | \n+0.005 | \n
| Seed fix | \nAll randomness sources fixed — deterministic baseline | \n0.392 | \n— | \n
| Generate→Rank→Select | \nTOPK=15, all diversity transforms, score_structure() | \n0.421 | \n+0.029 | \n
| Protenix tuning | \nSmoothness penalty, small-sequence boost, slot control | \n0.422 | \n+0.001 | \n
| Seed optimization | \nDual seed for long + 7 seeds for short sequences | \n0.424 | \n+0.002 | \n
\n\nNote: The jump from 0.242 to 0.392 reflects the addition of Protenix to the pipeline. The apparent gap is because the deterministic Protenix baseline (0.392) replaced earlier non-deterministic scores which included lucky runs up to 0.426.
\n
The interaction between seed count and ranking strength was non-obvious. More seeds alone did not help — the ranking function had to be strong enough to select the best from a larger pool.
\n| Setup | \nLB Score | \nVerdict | \n
|---|---|---|
| Baseline (5 samples, 1 seed) | \n0.422 | \n— | \n
| + Dual seed for long sequences only | \n0.416 | \n❌ Worse — expensive, low benefit alone | \n
| + 7 seeds for short sequences only | \n0.420 | \n⚠️ Slight drop — ranking not strong enough | \n
| Dual seed (long) + 7 seeds (short) | \n0.424 | \n✅ Best — diversity and ranking work together | \n
\n\nKey insight: More candidates ≠ better predictions. Better candidates + proper ranking = improvement. The dual-seed and 7-seed strategies only worked together because
\nscore_structure()was strong enough to identify the better structures from the larger pool.
Adding validation labels as templates would have given better structural references. However, the alignment search over ~28,000 sequences (train + val) caused a Kaggle timeout every time. The pipeline never finished scoring.
\nBlending the top-2 candidates by coordinate averaging: (c1 + c2) / 2. The hypothesis was that averaging would smooth out errors. In practice, the top candidates were already well-optimised and the average degraded topology rather than improving it.
We diagnosed which diversity transform helped each sequence on the validation set and hardcoded the best transform per sequence. The +0.011 improvement on validation was too small to measure on the leaderboard.
\nSplitting ultra-long sequences into 512nt chunks, running Protenix, and stitching with Kabsch alignment. The stitched output was no better than TBM for these sequences in isolation. Only when combined with dual-seed and the overall generate/rank/select architecture did it contribute.
\nUntil seeds are deterministic, you cannot measure whether a change actually helps. Every experiment before the seed fix was polluted by Protenix variance. The seed fix was not a performance improvement — it was a prerequisite for all further meaningful work.
\nAverage metrics hide what's actually failing. The template TM-score diagnostic — computing TM of raw template coordinates before any pipeline processing — revealed in one experiment that the pipeline was not the bottleneck. Without this, we could have spent days optimising morphing and refinement code that would never move the leaderboard.
\nBest-of-5 scoring means diversity across predictions is strictly more valuable than precision of one. The entire generate/rank/select architecture exists to exploit this. A pipeline that generates 12 diverse candidates and selects the best 5 beats a pipeline that generates 5 careful predictions.
\nWhen a good template exists (alignment > 0.5), TBM is near-perfect and Protenix cannot improve on it. When no good template exists, TBM is useless and Protenix is the only option. Understanding this separation and routing accordingly is what kept the strong sequences strong while giving the weak sequences their best chance.
\n18 of 28 test sequences are genuinely novel RNA folds with no structurally similar solved structure in any public database. This is not a pipeline problem. These sequences were chosen for the competition precisely because they are hard. The only path to improving predictions on them is a better generative model.
\nWe did not change the underlying models. We changed how they are used — when to trust TBM, when to use Protenix, how many candidates to generate, and how to select among them. The 0.029 improvement from 0.392 to 0.421 came entirely from architectural decisions, not from improving the models themselves.
\n\n\nHow we went from 0.184 to 0.424 — fixing randomness, diagnosing templates, and redesigning the prediction pipeline
\n
\n\nFinal LB Score: 0.424 · Deterministic · TBM + multi-seed Protenix · Generate → Rank → Select architecture
\n
The competition asks for 5 predicted 3D structures per RNA sequence. The metric takes the best TM-score among your 5 predictions — so the goal is not one perfect prediction, but five diverse, high-quality candidates.
\nWe combined two complementary methods:
\nThe 28 test sequences fall into three natural groups that required different handling:
\nTBM started at 0.184 and grew through several key iterations before becoming the backbone of the final solution.
\nThe original baseline generated 5 predictions by adding Gaussian noise to a single morphed template. The problem: all 5 predictions were nearly identical, wasting the best-of-5 metric.
\nThe fix was to use genuinely different template subsets for each prediction slot — top-4, templates 3–6, templates 5–8, even-indexed, odd-indexed. Each slot blended its assigned templates by coordinate averaging. This created real structural diversity and jumped the score to 0.233.
\nAfter generating 5 predictions, we sorted them by a physics-based pseudo-energy function so the most plausible structure became prediction 1. The energy combined bond length deviation from the ideal 5.9Å C1'–C1' distance and a steric clash penalty for atoms closer than 3.8Å.
\nRather than applying one strategy uniformly, we diagnosed each sequence individually and routed it to the strategy where it performed best. The 7 sequences with no templates at all worked best with a diverse geometry fallback (helix, extended strand, compact globule, sequence-guided, reverse helix).
\nWe replaced the raw alignment score with a composite that factors in length similarity, exact match ratio, and GC content similarity:
\nscore = (align²) × len_sim × (1 + 0.3×exact) × (0.5 + 0.5×gc_sim)\nSquaring the alignment score amplifies strong matches and penalises weak ones.
\nIncreasing TOPK from 8 to 15 and generating up to 12 candidates per sequence gave the scoring function more material to work with. We also replaced Gaussian noise with structured diversity transforms applied in rotation:
\n| Slot | \nTransform | \nDescription | \n
|---|---|---|
| 0 | \nClean | \nBest template, no noise | \n
| 1 | \nTiny noise | \nProportional to alignment uncertainty | \n
| 2 | \nHinge rotation | \nRotates one segment around a pivot | \n
| 3 | \nJitter chains | \nIndependent rigid rotation + translation per chain | \n
| 4 | \nSmooth wiggle | \nSmooth interpolated displacement field | \n
| 5+ | \nGaussian noise | \nFallback | \n
Before any architectural improvements were possible, we had to solve a more fundamental problem: the same notebook was producing scores of 0.392, 0.404, 0.409, and 0.426 on different runs. Without reproducibility, every experiment was noise.
\nA systematic audit found 12 distinct sources of randomness. The most critical:
\nCUBLAS_WORKSPACE_CONFIG set after PyTorch import — this environment variable must be set before torch is imported, or it has no effect on GPU determinismnp.random.seed() inside prediction functions with seed=len(predictions)*7+len(seq) — if a morph failed upstream and len(predictions) differed from expected, the seed changed and all downstream noise changedshortlist_templates sort with no tiebreaker — equal alignment scores had non-deterministic ordering depending on dataframe memory layoutlru_cache(maxsize=200,000) — when the cache filled, eviction order varied between runsSeven targeted changes restored full determinism:
\n# Cell 0 — before ANY import\nimport os, random\nos.environ['CUBLAS_WORKSPACE_CONFIG'] = ':4096:8'\nos.environ['PYTHONHASHSEED'] = '42'\nrandom.seed(42)\nimport numpy as np; np.random.seed(42)\nimport torch; torch.manual_seed(42)\ntorch.use_deterministic_algorithms(True, warn_only=True)\nnp.random.seed() calls replaced with isolated np.random.default_rng(seed=(ridx × 10B + slot × 10007) % 2³²) — seed is globally unique per sequence and slottid as secondary sort key in all template ranking functionslru_cache to maxsize=None0.1\n\nResult: Three consecutive submissions all scored exactly 0.392 — perfect determinism confirmed.
\n
After fixing determinism, every experiment returned 0.392. To understand why, we built a deep diagnostic that computed the TM-score of raw template coordinates before any pipeline processing.
\n| Finding | \nSequences Affected | \nConclusion | \n
|---|---|---|
| High alignment (>0.4) but template TM < 0.2 | \n10 sequences | \nSame sequence, different 3D fold — conformational isomers | \n
| Alignment < 0.3 — no template exists | \n8 sequences | \nGenuinely novel folds — no known analog in any database | \n
| morph_delta ≈ 0 for all sequences | \nAll 28 | \nPipeline is NOT destroying good templates — ceiling is the data | \n
| 9EBP — only sequence with tmpl_TM > 0.35 | \n1 sequence | \nGood template exists, already handled well by pipeline | \n
The most striking finding was 9LEL — alignment score 1.280 (near-identical sequence) but template TM = 0.020 (completely wrong structure). The template in the database folds the same sequence into a different 3D conformation. This is conformational isomerism — common in RNA, and impossible to fix by improving the morphing pipeline.
\n\n\nKey insight: The pipeline was not the problem. The training database simply does not contain structurally correct templates for 18 of 28 test sequences. These were specifically chosen for the competition because they represent novel folds. No amount of pipeline tuning overcomes this — only a better generative model can.
\n
The original pipeline pre-assigned slots before generating any predictions. It counted good TBM candidates (alignment ≥ 0.4), allocated N TBM slots, and sent the remaining 5-N slots to Protenix. The problem: if TBM slot 3 produced a better structure than Protenix slot 1, TBM slot 3 was discarded anyway. The pipeline had no quality gate.
\nWe replaced slot allocation with a generate-then-rank approach:
\nscore_structure()def score_structure(coords):\n diffs = np.linalg.norm(np.diff(coords, axis=0), axis=1)\n bond_pen = np.sum((diffs - 5.9) ** 2) # ideal C1'-C1' = 5.9Å\n dm = distance_matrix(coords, coords)\n clash_pen = np.sum(dm < 3.8) # atomic clash threshold\n return -(clash_pen * 6.0 + bond_pen * 0.3) # higher = better\nTBM candidates also receive an alignment bonus (align² × 8.0) and a length-dependent bonus, ensuring that high-confidence template morphs rank above uncertain ones.
Instead of one fixed seed with N_sample=5, Protenix runs N_sample=1 per seed with a sequence-length-dependent number of seeds:
\n| Sequence Type | \nSeed Count | \nRationale | \n
|---|---|---|
| Short (≤400 nt) | \n7 seeds | \nMore exploration — each run is fast | \n
| Medium (400–1000 nt) | \n3 seeds | \nBalanced exploration vs compute | \n
| 9MME, 9ZCC | \n2 seeds | \nChunked inference (512nt chunks, 128nt overlap, Kabsch-aligned stitching) | \n
| OUR_SEQUENCES | \n0 Protenix | \nTBM is already near-optimal | \n
| Phase | \nKey Change | \nLB Score | \nDelta | \n
|---|---|---|---|
| Baseline TBM | \nTemplate morphing + Gaussian noise | \n0.184 | \n— | \n
| Experiment B | \nStrategic template diversity (different subsets per slot) | \n0.233 | \n+0.049 | \n
| Experiment 1 | \nEnergy-based reranking of 5 predictions | \n0.237 | \n+0.004 | \n
| FINAL routing | \nPer-sequence strategy routing | \n0.242 | \n+0.005 | \n
| Seed fix | \nAll randomness sources fixed — deterministic baseline | \n0.392 | \n— | \n
| Generate→Rank→Select | \nTOPK=15, all diversity transforms, score_structure() | \n0.421 | \n+0.029 | \n
| Protenix tuning | \nSmoothness penalty, small-sequence boost, slot control | \n0.422 | \n+0.001 | \n
| Seed optimization | \nDual seed for long + 7 seeds for short sequences | \n0.424 | \n+0.002 | \n
\n\nNote: The jump from 0.242 to 0.392 reflects the addition of Protenix to the pipeline. The apparent gap is because the deterministic Protenix baseline (0.392) replaced earlier non-deterministic scores which included lucky runs up to 0.426.
\n
The interaction between seed count and ranking strength was non-obvious. More seeds alone did not help — the ranking function had to be strong enough to select the best from a larger pool.
\n| Setup | \nLB Score | \nVerdict | \n
|---|---|---|
| Baseline (5 samples, 1 seed) | \n0.422 | \n— | \n
| + Dual seed for long sequences only | \n0.416 | \n❌ Worse — expensive, low benefit alone | \n
| + 7 seeds for short sequences only | \n0.420 | \n⚠️ Slight drop — ranking not strong enough | \n
| Dual seed (long) + 7 seeds (short) | \n0.424 | \n✅ Best — diversity and ranking work together | \n
\n\nKey insight: More candidates ≠ better predictions. Better candidates + proper ranking = improvement. The dual-seed and 7-seed strategies only worked together because
\nscore_structure()was strong enough to identify the better structures from the larger pool.
Adding validation labels as templates would have given better structural references. However, the alignment search over ~28,000 sequences (train + val) caused a Kaggle timeout every time. The pipeline never finished scoring.
\nBlending the top-2 candidates by coordinate averaging: (c1 + c2) / 2. The hypothesis was that averaging would smooth out errors. In practice, the top candidates were already well-optimised and the average degraded topology rather than improving it.
We diagnosed which diversity transform helped each sequence on the validation set and hardcoded the best transform per sequence. The +0.011 improvement on validation was too small to measure on the leaderboard.
\nSplitting ultra-long sequences into 512nt chunks, running Protenix, and stitching with Kabsch alignment. The stitched output was no better than TBM for these sequences in isolation. Only when combined with dual-seed and the overall generate/rank/select architecture did it contribute.
\nUntil seeds are deterministic, you cannot measure whether a change actually helps. Every experiment before the seed fix was polluted by Protenix variance. The seed fix was not a performance improvement — it was a prerequisite for all further meaningful work.
\nAverage metrics hide what's actually failing. The template TM-score diagnostic — computing TM of raw template coordinates before any pipeline processing — revealed in one experiment that the pipeline was not the bottleneck. Without this, we could have spent days optimising morphing and refinement code that would never move the leaderboard.
\nBest-of-5 scoring means diversity across predictions is strictly more valuable than precision of one. The entire generate/rank/select architecture exists to exploit this. A pipeline that generates 12 diverse candidates and selects the best 5 beats a pipeline that generates 5 careful predictions.
\nWhen a good template exists (alignment > 0.5), TBM is near-perfect and Protenix cannot improve on it. When no good template exists, TBM is useless and Protenix is the only option. Understanding this separation and routing accordingly is what kept the strong sequences strong while giving the weak sequences their best chance.
\n18 of 28 test sequences are genuinely novel RNA folds with no structurally similar solved structure in any public database. This is not a pipeline problem. These sequences were chosen for the competition precisely because they are hard. The only path to improving predictions on them is a better generative model.
\nWe did not change the underlying models. We changed how they are used — when to trust TBM, when to use Protenix, how many candidates to generate, and how to select among them. The 0.029 improvement from 0.392 to 0.421 came entirely from architectural decisions, not from improving the models themselves.
\n