{"cells":[{"cell_type":"markdown","metadata":{},"source":"# Bend the Knee to the Dinosaurs — execution audit and negative-result record\n\n> ### Provenance notice — please read first\n>\n> **This notebook is a derivative work. It is not an original modelling result.**\n>\n> The entire prediction pipeline — the model families, the released checkpoints,\n> the preprocessing, and the fusion recipe — is inherited from\n> **Mattia Angeli, [*Bend the Knee to the Dinosaurs*](https://www.kaggle.com/code/mattiaangeli/bend-the-knee-to-the-dinosaurs)**\n> (version 27, `scriptVersionId=348195103`), used under the Apache-2.0 licence that\n> Kaggle applies to public notebooks. Roughly **89% of the executable code here is\n> that notebook's**, and 12 cells are byte-identical to it.\n>\n> The public score of **0.940 belongs to that recipe, not to this notebook.** Nothing\n> added here improved it: five controlled variations were submitted and none beat it\n> (three tied at 0.940, two regressed to 0.922). Those results are recorded below.\n>\n> **What this notebook adds** is an audit layer: raw-output guards on every branch, a\n> per-stage reproduction receipt, an independent reconstruction of the final fusion,\n> and a written record of the five failed improvement attempts. That is a\n> documentation contribution on top of someone else's model, and it is the only thing\n> here that is ours to claim.\n\n## Where every component comes from\n\n| Component | Author / source | Licence | Ours? |\n|---|---|---|---|\n| Raptor CoAtNet arm (4 views) | [Mattia Angeli](https://www.kaggle.com/code/mattiaangeli/bend-the-knee-to-the-dinosaurs); weights [dreaddevelopment/raptor-knee-*](https://www.kaggle.com/datasets/dreaddevelopment/raptor-knee-maxspan) | Apache-2.0 | inherited |\n| Residual CoAtNet (ep 4/6/8) | [mattiaangeli/rsna-knee-coat-resgated-ep10-top3](https://www.kaggle.com/datasets/mattiaangeli/rsna-knee-coat-resgated-ep10-top3) | Apache-2.0 | inherited |\n| A5 five-fold DINOv3 | [mattiaangeli/knee-mri-fold-weights](https://www.kaggle.com/datasets/mattiaangeli/knee-mri-fold-weights) | Apache-2.0 | inherited |\n| RadImageNet encoder | [marwanmath/resnet-50-radimagenet-marwan](https://www.kaggle.com/datasets/marwanmath/resnet-50-radimagenet-marwan) | Apache-2.0 | inherited |\n| Rad E9/E11/E13 heads | [antoinegg1](https://www.kaggle.com/datasets/antoinegg1/rsna-knee-e11-diverse-heads-v20) | Apache-2.0 | inherited |\n| Rad branch calibrator | Mattia Angeli (fitted upstream) | Apache-2.0 | inherited |\n| Report-derived labels | [pilkwang/rsna-knee-llm-labels](https://www.kaggle.com/datasets/pilkwang/rsna-knee-llm-labels) | Apache-2.0 | inherited |\n| Final fusion weights | Mattia Angeli | Apache-2.0 | inherited |\n| **DINOv2 slot-attention head** (`SlotHead`, slot priors, slot scheme) | **this author**, first published 2026-08 in [read the report, then the knee](https://www.kaggle.com/code/prvsiyan/rsna-knee-read-the-report-then-the-knee) | Apache-2.0 | **ours** |\n| **Audit layer, stage receipts, fusion reconstruction** | **this author** | Apache-2.0 | **ours** |\n| **Five-submission negative-result record** | **this author** | Apache-2.0 | **ours** |\n\n## A note on how the inherited code is presented here\n\nIn the parent notebook two components are carried in forms that cannot be read or\ndiffed: the Raptor arm as a single escaped string literal run through\n`exec(compile(...))`, and the Rad calibrator as a `zlib`+`base64` payload. Both are\nreproduced in this version **as ordinary readable source** — `raptor_arm.py` written\nout in full, and the calibrator as plain JSON. The numbers are unchanged and the\npredictions are bit-identical; only the presentation differs. Inherited code should\nbe legible enough that a reader can see whose it is.\n"},{"cell_type":"markdown","metadata":{},"source":"## What the five-submission experiment taught us\n\nThe table records **completed scores returned by the competition submissions\nAPI**, not scores inferred from notebook titles or small visible outputs. All\nfive configurations were fixed before their grades were inspected.\n\n| Version | Change from V17 | Submission | Public score | Interpretation |\n|---|---|---|---|---|\n| V17 | Reference | 56107593 | **0.940** | Verified starting recipe |\n| V18 | A: native E11 head residual | 56122095 | **0.940** | No displayed improvement |\n| V19 | B: alternate Rad head residual | 56122238 | **0.940** | No displayed improvement |\n| V20 | G: Raptor intensity-gain integration | 56122418 | **0.922** | Regression |\n| V21 | A + B | 56122582 | **0.940** | No displayed improvement |\n| V22 | A + B + G | 56122723 | **0.922** | Regression repeated with both residuals |\n\n### What worked\n\nThe strongest established result in this lineage is the complete V17 recipe at\n0.940. It combines complementary public image families and preserves their\nparticular preprocessing and fusion rules. Its intermediate outputs and final\nreconstruction checks make the recipe inspectable. The batch also established\nthat the extra candidate computations executed; an attached model or a successful\ncell alone would not have proved that a change reached the final prediction path.\n\nThis is evidence for the **complete recipe**. It is not a controlled measurement\nof each component's individual contribution. We did not remove every component\none at a time and measure an independent public-score gain for each one.\n\n### Why A and B did not establish an improvement\n\nA adds a ranked residual between native E11 predictions and E13 heads evaluated\non the E11 pixel layout. B introduces a residual from the historical alternate\nRad branch. These are real changes to the final graph, but they use related\nheads and a shared RadImageNet encoder rather than independently learned image\nrepresentations. Their influence then passes through further blending and rank\noperations. The transformer/Rad branch has only 40% of the final vote on eleven\ntargets and no vote on lateral meniscus.\n\nThat structure can make a small useful correction hard to detect, or introduce\ndifferent errors with little net gain. The three 0.940 grades cannot distinguish\nthose explanations. They also do not justify increasing the residual weights.\nNeither residual is promoted into this restored default.\n\n### Why G is excluded\n\nG evaluated intensity gains 0.9, 1.0 and 1.1 using integration weights\n1/6, 4/6 and 1/6 while retaining the original Raptor arm weights. This was intended\nto reduce sensitivity to an arbitrary intensity scale. The added views changed\nraw probabilities in the ordinary-run artifacts, confirming that they were active.\n\nBoth G and ABG scored 0.922, a displayed loss of 0.018 against V17. No specific\ndeterministic normalization, UID-alignment or fusion defect was established by\nthe subsequent audit. The evidence rejects **this gain-integration recipe** for\ndeployment. It does not prove that every form of test-time augmentation is harmful.\nExtra inference is worthwhile only when it corrects errors of the final ensemble.\n\n### What identical 0.940 scores do and do not tell us\n\nThree-decimal scores are a coarse observation. Different predictions can produce\nthe same displayed macro AUC. Under ordinary nearest-thousandth rounding, two\nvalues displayed as 0.940 could differ by almost 0.001. Kaggle's exact hidden\nprecision and tie-breaking rule were not established in this investigation.\n\nWe could not retrieve the completed hidden-test prediction files through the\nofficial download endpoint, so we do not report hidden rank correlations,\nper-target AUC deltas or supposed fourth-decimal wins. The three visible studies\nare too few to infer those quantities. The failed and tied attempts remain\ndocumented so another reader does not mistake repeated computation for progress.\n\n## What differs in the leading public notebook\n\nThe September 9 audit of\n[RSNA Baseline V13](https://www.kaggle.com/code/evgendvorkin/rsna-baseline?scriptVersionId=348520554)\nfound the familiar active model families and two material blend differences:\n\n| Operation | This V17 reference | Audited public V13 |\n|---|---|---|\n| Raptor weights, in the four-view order above | 60 / 10 / 10 / 20 | 55 / 10 / 15 / 20 |\n| Hybrid H before the outer blend | Keep its weighted rank values | Rank H again |\n\nBoth notebooks displayed 0.940. Their order in a public notebook listing does\nnot establish which difference helps or prove an extra-decimal advantage. The\npublic notebook listing is also distinct from the overall team leaderboard.\n\nThese differences suggest a small offline comparison: reference, rank-H only,\nweight change only, and both together. There is a substantial validation caveat.\nThe residual-CoAt release's authenticated trainer fits the 4,349 non-expert\nstudies and uses Gold58 to select its published epochs. Those expert studies\nare a selection set for this component, not an untouched test set for selecting\nanother blend. Likewise, evaluating all five deployed fold heads on a training\nrow includes heads trained on that row; genuine out-of-fold inference must use\nonly models that excluded it. [Residual-CoAt source package](https://www.kaggle.com/datasets/mattiaangeli/rsna-knee-coat-resgated-ep10-top3).\n\nUntil the complete prediction provenance is established, a four-way comparison\non contaminated rows would give misleading precision. It is not implemented as\na leaderboard weight sweep here.\n\n## What is new, and what remains research\n\nRelative to the original 0.940 notebook, the completed work adds **a reproducible\nnegative-result record, source-level blend comparison, validation-provenance\nfindings, and a clearer public walkthrough**. It does not establish a new higher\nscoring inference component. The default executable cells are restored exactly\nto V17, and the original image and component notices are retained.\n\nA separate completed study investigated an independently trained ResNet34 image model\nat 288 pixels, initialized from general ImageNet weights rather than an existing\nRSNA-trained checkpoint. It compares ordinary soft-target training with one\nfixed loss that downweights disagreement between two report-label sources.\nEntire expert-containing report groups are excluded: 62 studies rather than\njust the 58 expert rows. The remaining data use 3,478 training and 867 held\nstudies for the first development fold. These are report-group exclusions;\npatient independence has not been established.\n\nThe two models have the same initialization and a fixed 20-epoch schedule, with\nno held-score-based checkpoint selection. Their first evaluation concerns\nreport-derived labels and cannot by itself establish MRI-label accuracy or an\nimprovement to this final ensemble. No prediction from that study is included\nhere. The authenticated fold-0 result did not pass its fixed continuation screen:\n\n| Report-label source | Ordinary loss | Disagreement weighting | Difference | 95% group-bootstrap interval |\n|---|---:|---:|---:|---|\n| Steven v2 | 0.7591 | 0.7612 | +0.0021 | [-0.0088, +0.0127] |\n| Pilkwang v2 | 0.7444 | 0.7420 | -0.0024 | [-0.0129, +0.0080] |\n\nThese are **report-label development scores, not leaderboard scores**. Independent\ncode reproduced the AUCs and 5,000 bootstrap comparisons. Neither trained model\nwas added to this notebook or submitted.\n\nThe postmortem uncovered a design mistake worth making explicit: numeric\nunknown codes were treated as observed labels. Steven's 0.5 means the report\ndoes not address a finding, but the old >=0.5 evaluation counted it as positive.\nPilkwang provides YES/NO/UNK columns; the experiment discarded those verdicts\nand treated its 0.28 unknown score as negative. Steven v2 also fills unknown\nsynovitis from effusion, which is a proxy rather than an explicit statement.\n[Steven label definitions](https://www.kaggle.com/datasets/stevenleehans/rsna-knee-llm-report-labels),\n[Pilkwang label definitions](https://www.kaggle.com/datasets/pilkwang/rsna-knee-llm-labels).\n\nThat limits the inference from this negative result: it does not establish a\n.75 accuracy ceiling for compact CNNs. The failed fold is not being rescored\nwith a more favorable threshold. A separate prospective comparison uses a\ndifferent held fold and explicit missingness masks while retaining the same\nimage model. Its conditional report-label AUC will have a different case mix,\nso it cannot be compared directly with the table above or with competition AUC.\nNo 0.941 claim is made for either experiment.\n\nThis direction is informed by primary competition discussions, with their\nlimitations kept visible. Competitors have reported strong compact image models,\nmixed returns from improved report labels, and broad plateaus across blend\nchoices. Those reports motivate experiments; they are not independently verified\nscores for checkpoints attached to this notebook.\n[Single-model discussion](https://www.kaggle.com/competitions/rsna-knee-abnormality-detection/discussion/735304),\n[three-decimal plateau](https://www.kaggle.com/competitions/rsna-knee-abnormality-detection/discussion/740439),\n[Raptor author discussion](https://www.kaggle.com/competitions/rsna-knee-abnormality-detection/discussion/737696).\n\nThe next useful model must contribute measurable corrections when combined\nwith an honestly evaluated reference. Standalone AUC, low prediction correlation,\nextra parameters and a different architecture each answer only part of that\nquestion. The final twelve-target gain and its uncertainty determine whether\nthe added runtime earns a place in the public inference path.\n"},{"cell_type":"code","metadata":{},"source":"\"\"\"Read-only execution instrumentation; does not change source predictions.\"\"\"\nimport hashlib as _rep_hashlib\nimport json as _rep_json\nimport os as _rep_os\nimport re as _rep_re\nimport sys as _rep_sys\nimport time as _rep_time\nfrom pathlib import Path as _RepPath\n\n_rep_started = _rep_time.time()\n_rep_source_sha256 = \"6b03919bbca5fe61e37c5ea6546609a771cd7db6503121f3e2dacdd8c9b6b052\"\n_rep_messages = []\n_rep_stages = {}\n\nclass _ReproductionOutput:\n    def __init__(self, underlying):\n        self.underlying = underlying\n    def write(self, value):\n        _rep_messages.append(value)\n        return self.underlying.write(value)\n    def flush(self):\n        return self.underlying.flush()\n    def __getattr__(self, name):\n        return getattr(self.underlying, name)\n\n_rep_sys.stdout = _ReproductionOutput(_rep_sys.stdout)\n\ndef _rep_sha(path):\n    digest = _rep_hashlib.sha256()\n    with _RepPath(path).open('rb') as stream:\n        for chunk in iter(lambda: stream.read(8 << 20), b''):\n            digest.update(chunk)\n    return digest.hexdigest()\n\ndef _rep_snapshot(stage):\n    import pandas as _pd\n    import numpy as _np\n    work = _RepPath('/kaggle/working')\n    frame = _pd.read_csv(work / 'submission.csv', dtype={'StudyInstanceUID': str})\n    candidates = [\n        _RepPath('/kaggle/input/competitions/rsna-knee-abnormality-detection'),\n        _RepPath('/kaggle/input/rsna-knee-abnormality-detection'),\n    ]\n    competition = next(p for p in candidates if (p / 'test.csv').is_file())\n    test = _pd.read_csv(competition / 'test.csv', dtype={'StudyInstanceUID': str})\n    schema = list(_pd.read_csv(competition / 'sample_submission.csv', nrows=0).columns)\n    assert list(frame.columns) == schema, (stage, 'columns')\n    assert frame.StudyInstanceUID.tolist() == test.StudyInstanceUID.tolist(), (stage, 'UID/order')\n    assert frame.StudyInstanceUID.is_unique, (stage, 'duplicate UID')\n    values = frame.iloc[:, 1:].to_numpy(_np.float64)\n    assert _np.isfinite(values).all(), (stage, 'nonfinite')\n    assert ((values >= 0) & (values <= 1)).all(), (stage, 'range')\n    destination = work / ('reproduction_' + stage + '.csv')\n    destination.write_bytes((work / 'submission.csv').read_bytes())\n    _rep_stages[stage] = {'rows': len(frame), 'sha256': _rep_sha(destination)}\n    print('[reproduction]', stage, _rep_stages[stage], flush=True)\n\n# Importing torch here creates no random tensors; the source controls its seeds.\nfor _rep_variable in ('OMP_NUM_THREADS', 'OPENBLAS_NUM_THREADS', 'MKL_NUM_THREADS'):\n    _rep_os.environ.setdefault(_rep_variable, '4')\nimport torch as _rep_torch\nassert _rep_torch.cuda.device_count() == 2, 'exactly two GPUs required'\n_rep_gpu_names = [_rep_torch.cuda.get_device_name(i) for i in range(2)]\nassert all('T4' in name for name in _rep_gpu_names), _rep_gpu_names\nprint('[reproduction] T4 preflight', _rep_gpu_names, flush=True)\n","execution_count":null,"outputs":[],"id":"reproduction-002"},{"cell_type":"markdown","metadata":{},"source":"## 1. DINO: read the study and combine twenty members\n\nThe source reader uses DICOM geometry and MRI series descriptions to build its saved slot layout. Each trained member receives the same preprocessing used by its released package. Overlapping slice windows supply the finding-specific pooling votes.\n","id":"reproduction-003"},{"cell_type":"code","metadata":{},"source":"from __future__ import annotations\nimport os\nfor _v in ('OMP_NUM_THREADS', 'OPENBLAS_NUM_THREADS', 'MKL_NUM_THREADS'):\n    os.environ.setdefault(_v, '4')\nimport gc\nimport hashlib\nimport json\nimport re\nimport time\nimport traceback\nimport threading\nfrom concurrent.futures import ThreadPoolExecutor\nfrom pathlib import Path\nimport numpy as np\nimport pandas as pd\nimport pydicom\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\ndef _cuda_execution_probe(index):\n    dev = torch.device(f'cuda:{index}')\n    try:\n        major, minor = torch.cuda.get_device_capability(index)\n        probe = nn.Conv2d(3, 4, kernel_size=3, padding=1).eval().to(dev)\n        with torch.inference_mode():\n            out = probe(torch.zeros((1, 3, 16, 16), device=dev))\n            if tuple(out.shape) != (1, 4, 16, 16):\n                raise RuntimeError(f'unexpected CUDA probe shape {tuple(out.shape)}')\n        torch.cuda.synchronize(index)\n        print(f'cuda:{index} probe PASS (compute {major}.{minor})')\n        del probe, out\n        torch.cuda.empty_cache()\n        return True\n    except Exception as exc:\n        print(f'cuda:{index} probe FAIL ({type(exc).__name__}: {exc}); using CPU fallback')\n        try:\n            torch.cuda.empty_cache()\n        except Exception:\n            pass\n        return False\nDEVS = []\nif torch.cuda.is_available():\n    DEVS = [torch.device(f'cuda:{i}') for i in range(torch.cuda.device_count()) if _cuda_execution_probe(i)]\nif not DEVS:\n    raise RuntimeError('GPU required')\nprint(f'devices: {[str(d) for d in DEVS]}')\nT0 = time.time()\nSEED = 2026\nnp.random.seed(SEED)\ntorch.manual_seed(SEED)\nTARGETS = ['ACL', 'MCL', 'Medial Meniscus', 'Lateral Meniscus', 'Medial OA', 'Lateral OA', 'PF OA', 'Effusion', 'Synovitis', \"Baker's\", 'Contusion', 'Fracture']\nCROP_MM = 130.0\nCACHE_IMG = 336\nGROUP = 3\nN_GROUP_MAX = 1\nCACHE_FRACTION = 0.45\nCACHE_BUDGET_MAX_GB = 24.0\nCACHE_BUDGET_GB = 12.0\nTEST_SHARE = 0.3\nHDR_THREADS = 16\nPIX_THREADS = 12\nORDER_THREADS = 32\nORDER_BUDGET_S = 5400\nRUNS = [{'name': 'r224', 'img': 224}, {'name': 'r336', 'img': 336}]\nEPOCHS = 10\nBATCH_STUDIES = 8\nAUG_ROT_DEG = 8.0\nAUG_SCALE = 0.08\nAUG_SHIFT = 0.05\nAUG_INTENSITY = 0.1\nLAT_MIN_OFFSET_MM = 20.0\nSLICE_BAND = (0.2, 0.8)\nRULES_NATIVE = {'order': 'normal', 'lat': 'centre', 'slot_fallback': False, 'decode_fill': 'nearest'}\nRULES_LEGACY = {'order': 'dominant_axis', 'lat': 'corner_x', 'slot_fallback': True, 'decode_fill': 'zero'}\nRULES = dict(RULES_NATIVE)\nLEGACY_LAT_OFFSET_MM = 5.0\nLR_HEAD = 0.001\nLR_BACKBONE = 8e-06\nUNFREEZE_LAST = 6\nWEIGHT_DECAY = 0.02\nEVAL_BATCH = 8\nTIME_BUDGET = 8.0 * 3600\nSLOTS_RECOVERED = [('SAG_FLUID_FS', 'Sagittal', True, True), ('COR_FLUID_FS', 'Coronal', True, True), ('AX_FLUID_FS', 'Axial', True, True), ('SAG_FLUID_NOFS', 'Sagittal', True, False), ('COR_T1', 'Coronal', False, False), ('SAG_T1', 'Sagittal', False, False)]\nSLOTS_PUBLIC = [('SAG_FLUID', 'Sagittal', None, True), ('COR_FLUID', 'Coronal', None, True), ('AX_FLUID', 'Axial', None, True), ('SAG_STRUCT', 'Sagittal', None, False), ('COR_STRUCT', 'Coronal', None, False), ('AX_STRUCT', 'Axial', None, False)]\nSLOT_SCHEME = os.environ.get('SLOT_SCHEME', 'recovered')\nSLOTS = SLOTS_PUBLIC if SLOT_SCHEME == 'public' else SLOTS_RECOVERED\nN_SLOT = len(SLOTS)\nPOOL_PARTS = {'cls_mean': 2, 'cls_mean_focal': 3}\nSLOT_PRIOR_TABLE = {'ACL': (0, 3, 5), 'MCL': (1, 4), 'Medial Meniscus': (0, 1, 3, 4), 'Lateral Meniscus': (0, 1, 3, 4), 'Medial OA': (1, 4, 5), 'Lateral OA': (1, 4, 5), 'PF OA': (0, 2, 5), 'Effusion': (0, 2), 'Synovitis': (0, 2), \"Baker's\": (0,), 'Contusion': (0, 1, 2), 'Fracture': (0, 1, 2, 4, 5)}\nSLOT_PRIOR_STRENGTH = 0.55\nFATSAT_OPTS = {'FS', 'FATSAT', 'FAT_SAT', 'FSAT'}\n_SEP = re.compile('[_\\\\-.]')\n_FATSAT_RX = re.compile('\\\\bfs\\\\b|fatsat|fat sat|\\\\bstir\\\\b|\\\\bspair\\\\b|\\\\bspir\\\\b|\\\\bwe\\\\b|water excit|\\\\btirm\\\\b|\\\\bsting\\\\b|\\\\bfatsup\\\\b')\n_T1_RX = re.compile('\\\\bt1\\\\b|\\\\bt1w\\\\b')\n_T2_RX = re.compile('\\\\bt2\\\\b|\\\\bt2w\\\\b')\n_PD_RX = re.compile('\\\\bpd\\\\b|\\\\bpdw\\\\b|proton|\\\\bdp\\\\b|dens')","execution_count":null,"outputs":[],"id":"reproduction-004"},{"cell_type":"code","metadata":{},"source":"def log(msg):\n    print(f'[{time.time() - T0:7.1f}s] {msg}', flush=True)\n\ndef find_root():\n    for c in [Path('/kaggle/input/competitions/rsna-knee-abnormality-detection'), Path('/kaggle/input/rsna-knee-abnormality-detection'), Path('data'), Path('.')]:\n        if (c / 'test.csv').is_file() and (c / 'test_series').is_dir():\n            return c\n    base = Path('/kaggle/input')\n    if base.is_dir():\n        for depth1 in sorted((p for p in base.iterdir() if p.is_dir())):\n            for cand in [depth1] + sorted((p for p in depth1.iterdir() if p.is_dir())):\n                if (cand / 'test.csv').is_file():\n                    return cand\n    raise FileNotFoundError(f'competition mount not found (cwd {Path.cwd()}); expected a directory holding test.csv and test_series/')\n\ndef find_dinov2(variant='small'):\n    base = Path('/kaggle/input')\n    if not base.is_dir():\n        return None\n    hits = []\n    for root, dirs, files in os.walk(base):\n        dirs[:] = [d for d in dirs if d not in ('train_series', 'test_series')]\n        if 'config.json' in files and 'dinov2' in root.lower():\n            hits.append(Path(root))\n    for h in hits:\n        if variant in str(h).lower():\n            return h\n    return hits[0] if hits else None\nROOT = find_root()\nlog(f'input root: {ROOT}')\nIMG = CACHE_IMG\n\ndef available_gb():\n    try:\n        with open('/proc/meminfo') as fh:\n            info = {k.strip(): v for k, v in (l.split(':', 1) for l in fh if ':' in l)}\n        return int(info['MemAvailable'].split()[0]) / 1024 ** 2\n    except Exception:\n        return CACHE_BUDGET_GB / CACHE_FRACTION\n\ndef plan_cache(n_study, n_test=0):\n    avail = available_gb()\n    budget = min(avail * CACHE_FRACTION, CACHE_BUDGET_MAX_GB)\n    n_total = n_study + max(n_test, int(TEST_SHARE * n_study))\n    per_slice = n_total * N_SLOT * IMG * IMG\n    afford = int(budget * 1024 ** 3 // max(per_slice, 1))\n    groups = max(1, min(N_GROUP_MAX, afford // GROUP))\n    log(f'memory: {avail:.1f} GB available, {budget:.1f} GB to the cache; sizing for {n_study} train + {n_total - n_study} test studies -> {groups} group(s) of {GROUP} = {groups * GROUP} slices per slot' + (f' (wanted {N_GROUP_MAX})' if groups < N_GROUP_MAX else ''))\n    return groups\nN_GROUP = plan_cache(len(pd.read_csv(ROOT / 'train.csv')), len(pd.read_csv(ROOT / 'test.csv')))\nCACHE_SLICES = GROUP * N_GROUP\nlog(f'cache layout: {N_GROUP} groups x {GROUP} slices = {CACHE_SLICES} per slot')","execution_count":null,"outputs":[],"id":"reproduction-005"},{"cell_type":"code","metadata":{},"source":"HDR_TAGS = ['SeriesDescription', 'SequenceName', 'ScanOptions', 'ScanningSequence', 'RepetitionTime', 'EchoTime', 'Laterality', 'PixelSpacing', 'Rows', 'Columns', 'RescaleSlope', 'RescaleIntercept', 'ImagePositionPatient', 'ImageOrientationPatient']\n\ndef _hdr_vec(s, n):\n    if not isinstance(s, str):\n        return None\n    try:\n        v = [float(x) for x in s.split('|')]\n    except ValueError:\n        return None\n    return np.array(v) if len(v) >= n else None\n\ndef side_from_geometry(h):\n    cx = {}\n    for r in h.itertuples(index=False):\n        ipp = _hdr_vec(getattr(r, 'ImagePositionPatient', None), 3)\n        iop = _hdr_vec(getattr(r, 'ImageOrientationPatient', None), 6)\n        ps = _hdr_vec(getattr(r, 'PixelSpacing', None), 2)\n        rows, cols = (getattr(r, 'Rows', None), getattr(r, 'Columns', None))\n        if ipp is None or iop is None or ps is None or (not rows) or (not cols):\n            continue\n        try:\n            c = ipp[:3] + iop[:3] * ps[1] * float(cols) / 2 + iop[3:6] * ps[0] * float(rows) / 2\n        except (TypeError, ValueError):\n            continue\n        cx.setdefault(r.StudyInstanceUID, []).append(float(c[0]))\n    out = {}\n    for st, xs in cx.items():\n        m = float(np.median(xs))\n        out[st] = None if abs(m) < LAT_MIN_OFFSET_MM else 'R' if m < 0 else 'L'\n    return out\n\ndef side_from_corner_x(h):\n    out = {}\n    for st, g in h.groupby('StudyInstanceUID'):\n        xs = []\n        for r in g.itertuples(index=False):\n            ipp = _hdr_vec(getattr(r, 'ImagePositionPatient', None), 3)\n            if ipp is not None and np.isfinite(ipp).all():\n                xs.append(float(ipp[0]))\n        if not xs:\n            out[st] = None\n            continue\n        x = float(np.median(xs))\n        out[st] = None if abs(x) < LEGACY_LAT_OFFSET_MM else 'R' if x < 0 else 'L'\n    return out\n\ndef lat_of(h, tag=''):\n    geo = side_from_corner_x(h) if RULES['lat'] == 'corner_x' else side_from_geometry(h)\n    d, n_tag, n_geo, n_none, n_disagree = ({}, 0, 0, 0, 0)\n    for st, g in h.groupby('StudyInstanceUID'):\n        v = [str(x).strip().upper() for x in g['Laterality'].dropna()]\n        if RULES['lat'] == 'corner_x' and 'ImageLaterality' in g.columns:\n            v += [str(x).strip().upper() for x in g['ImageLaterality'].dropna()]\n        v = [x[0] for x in v if x and x[0] in ('L', 'R')]\n        side = v[0] if v else None\n        if side is not None:\n            n_tag += 1\n            if geo.get(st) is not None and geo[st] != side:\n                n_disagree += 1\n        else:\n            side = geo.get(st)\n            n_geo += side is not None\n            n_none += side is None\n        d[st] = side\n    log(f'{tag}laterality: {n_tag} from the tag, {n_geo} from geometry, {n_none} unresolved; tag and geometry disagree on {n_disagree} ({n_disagree / max(n_tag, 1):.1%} of the tagged)')\n    return d\n\ndef probe(item):\n    split, study, series, path = item\n    row = {'split': split, 'StudyInstanceUID': study, 'SeriesInstanceUID': series, 'dir': path}\n    try:\n        files = sorted((e.name for e in os.scandir(path) if e.name.endswith('.dcm')))\n        row['files'] = files\n        row['n_slices'] = len(files)\n        if not files:\n            return row\n        ds = pydicom.dcmread(os.path.join(path, files[len(files) // 2]), stop_before_pixels=True, force=True)\n        for t in HDR_TAGS:\n            v = getattr(ds, t, None)\n            if v is None:\n                row[t] = None\n            elif isinstance(v, (list, tuple)) or type(v).__name__ == 'MultiValue':\n                row[t] = '|'.join((str(x) for x in v))\n            else:\n                row[t] = str(v)\n    except Exception as exc:\n        row['err'] = str(exc)[:120]\n    return row\n\ndef walk(split):\n    base = ROOT / split\n    items = []\n    if not base.is_dir():\n        return pd.DataFrame(columns=['split', 'StudyInstanceUID', 'SeriesInstanceUID', 'dir', 'files', 'n_slices'] + HDR_TAGS)\n    for study in os.scandir(base):\n        if study.is_dir():\n            for series in os.scandir(study.path):\n                if series.is_dir():\n                    items.append((split, study.name, series.name, series.path))\n    with ThreadPoolExecutor(max_workers=HDR_THREADS) as pool:\n        rows = list(pool.map(probe, items))\n    return pd.DataFrame(rows)\n\ndef annotate(df):\n    desc = df['SeriesDescription'].fillna('') + ' ' + df['SequenceName'].fillna('')\n    desc = desc.str.lower().str.replace(_SEP, ' ', regex=True)\n    opts = df['ScanOptions'].fillna('').str.upper().str.split('|')\n    opts_fs = opts.apply(lambda ts: any((t.strip() in FATSAT_OPTS for t in ts)))\n    df['fatsat'] = desc.str.contains(_FATSAT_RX) | opts_fs\n    tr = pd.to_numeric(df['RepetitionTime'], errors='coerce')\n    te = pd.to_numeric(df['EchoTime'], errors='coerce')\n    gre = df['ScanningSequence'].fillna('').str.upper().str.contains('GR')\n    t1, t2, pdw = (desc.str.contains(_T1_RX), desc.str.contains(_T2_RX), desc.str.contains(_PD_RX))\n    df['weight'] = np.where(t1 & ~t2 & ~pdw, 'T1', np.where(t2 & ~pdw, 'T2', np.where(pdw, 'PD', np.where(gre, 'GRE', np.where(tr < 800, 'T1', np.where(te > 60, 'T2', np.where(tr >= 800, 'PD', 'UNK')))))))\n    df['fluid'] = np.isin(df['weight'], ['PD', 'T2'])\n    df['px'] = pd.to_numeric(df['PixelSpacing'].fillna('').str.split('|').str[0].replace('', np.nan), errors='coerce')\n    return df","execution_count":null,"outputs":[],"id":"reproduction-006"},{"cell_type":"code","metadata":{},"source":"def pick_slots(series_df, plane_map):\n    series_df = series_df.copy()\n    series_df['plane'] = series_df['SeriesInstanceUID'].map(plane_map)\n    out = {}\n    for study, g in series_df.groupby('StudyInstanceUID'):\n        chosen = {}\n        for name, plane, fluid, fs in SLOTS:\n            sel = (g['plane'] == plane) & (g['fatsat'] == fs)\n            if fluid is not None:\n                sel &= g['fluid'] == fluid\n            cand = g[sel]\n            if len(cand) == 0 and RULES['slot_fallback'] and (fluid is False):\n                cand = g[(g['plane'] == plane) & ~g['fatsat']]\n            if len(cand):\n                chosen[name] = cand.sort_values('n_slices', ascending=False).iloc[0]\n        out[study] = chosen\n    return out","execution_count":null,"outputs":[],"id":"reproduction-007"},{"cell_type":"code","metadata":{},"source":"ORDER_TAGS = [(32, 50), (32, 55), (32, 19)]\nDECODE_FAILED = []\n\n\ndef _natural_key(name):\n    return tuple((int(x) if x.isdigit() else x.lower() for x in re.split('(\\\\d+)', str(name))))\n\ndef _order_dominant_axis(rec):\n    files, d = (rec['files'], rec['dir'])\n    rows = []\n    for pos, f in enumerate(files):\n        ipp = inst = None\n        try:\n            ds = pydicom.dcmread(os.path.join(d, f), force=True, stop_before_pixels=True, specific_tags=['ImagePositionPatient', 'InstanceNumber'])\n            raw = getattr(ds, 'ImagePositionPatient', None)\n            if raw is not None and len(raw) >= 3:\n                c = np.asarray(raw[:3], dtype=np.float64)\n                if np.isfinite(c).all():\n                    ipp = c\n            n = getattr(ds, 'InstanceNumber', None)\n            if n is not None:\n                inst = float(n)\n        except Exception:\n            pass\n        rows.append((f, ipp, inst, pos))\n    placed = [r for r in rows if r[1] is not None]\n    need = max(2, int(0.8 * len(rows)))\n    if len(placed) >= need:\n        xyz = np.stack([r[1] for r in placed])\n        axis = int(np.argmax(np.ptp(xyz, axis=0)))\n        spare = float(np.nanmedian(xyz[:, axis]))\n        rows.sort(key=lambda r: (float(r[1][axis]) if r[1] is not None else spare, r[2] if r[2] is not None else float('inf'), r[3]))\n    elif sum((r[2] is not None for r in rows)) >= need:\n        rows.sort(key=lambda r: (r[2] if r[2] is not None else float('inf'), r[3]))\n    else:\n        rows.sort(key=lambda r: _natural_key(r[0]))\n    return ([r[0] for r in rows], True)\n\ndef order_slices(rec):\n    if RULES['order'] == 'dominant_axis':\n        return _order_dominant_axis(rec)\n    files, d = (rec['files'], rec['dir'])\n    keyed = []\n    for f in files:\n        k = None\n        try:\n            ds = pydicom.dcmread(os.path.join(d, f), force=True, stop_before_pixels=True, specific_tags=ORDER_TAGS)\n            iop = np.asarray(ds.ImageOrientationPatient, dtype=float)\n            ipp = np.asarray(ds.ImagePositionPatient, dtype=float)\n            k = float(np.dot(ipp, np.cross(iop[:3], iop[3:])))\n        except Exception:\n            try:\n                k = float(ds.InstanceNumber)\n            except Exception:\n                k = None\n        keyed.append((k, f))\n    if any((k is None for k, _ in keyed)):\n        return (files, False)\n    return ([f for _, f in sorted(keyed, key=lambda t: t[0])], True)\n\ndef read_slot(rec, n_slice=None, out_size=None):\n    n_slice = GROUP if n_slice is None else n_slice\n    out_size = IMG if out_size is None else out_size\n    files, d, px = (rec.get('ordered') or rec['files'], rec['dir'], rec['px'])\n    n = len(files)\n    if n == 0:\n        return None\n    lo, hi = (int(SLICE_BAND[0] * (n - 1)), int(SLICE_BAND[1] * (n - 1)))\n    idx = np.unique(np.linspace(lo, hi, n_slice).astype(int)) if hi > lo else np.array([n // 2])\n    while len(idx) < n_slice:\n        idx = np.append(idx, idx[-1])\n    planes = []\n    for i in idx[:n_slice]:\n        try:\n            ds = pydicom.dcmread(os.path.join(d, files[int(i)]), force=True)\n            a = ds.pixel_array.astype(np.float32)\n            sl = float(getattr(ds, 'RescaleSlope', 1) or 1)\n            ic = float(getattr(ds, 'RescaleIntercept', 0) or 0)\n            a = a * sl + ic\n        except Exception:\n            a = None\n        planes.append(a)\n    got = [k for k, p in enumerate(planes) if p is not None]\n    if RULES['decode_fill'] == 'zero':\n        if not got:\n            DECODE_FAILED.append(rec.get('SeriesInstanceUID', d))\n        planes = [np.zeros((out_size, out_size), np.float32) if p is None else p for p in planes]\n        got = list(range(len(planes)))\n    if not got:\n        DECODE_FAILED.append(rec.get('SeriesInstanceUID', d))\n        return None\n    if len(got) < len(planes):\n        DECODE_FAILED.append(rec.get('SeriesInstanceUID', d))\n        for k, p in enumerate(planes):\n            if p is None:\n                planes[k] = planes[min(got, key=lambda j: abs(j - k))]\n    shp = planes[0].shape\n    planes = [p if p.shape == shp else np.zeros(shp, np.float32) for p in planes]\n    vol = np.stack(planes)\n    if px and np.isfinite(px) and (px > 0):\n        want = int(round(CROP_MM / px))\n        h, w = shp\n        if 16 < want < min(h, w):\n            cy, cx = (h // 2, w // 2)\n            half = want // 2\n            vol = vol[:, max(0, cy - half):cy + half, max(0, cx - half):cx + half]\n    lo_v, hi_v = np.percentile(vol, [1, 99])\n    vol = np.clip((vol - lo_v) / max(hi_v - lo_v, 1e-06), 0, 1)\n    t = torch.from_numpy(np.ascontiguousarray(vol)).unsqueeze(0)\n    t = F.interpolate(t, size=(out_size, out_size), mode='bilinear', align_corners=False)\n    return (t.squeeze(0) * 255).round().clamp(0, 255).to(torch.uint8)","execution_count":null,"outputs":[],"id":"reproduction-008"},{"cell_type":"code","metadata":{},"source":"def normalise_laterality(img, plane, lat):\n    if lat != 'R':\n        return img\n    if plane in ('Coronal', 'Axial'):\n        return torch.flip(img, dims=[-1])\n    return torch.flip(img, dims=[0])","execution_count":null,"outputs":[],"id":"reproduction-009"},{"cell_type":"code","metadata":{},"source":"ORDER_CACHE = os.environ.get('RSNA_ORDER_CACHE') or None\n\ndef build_cache(slot_map, plane_map, lat_map, tag):\n    studies = sorted(slot_map)\n    sidx = {s: i for i, s in enumerate(studies)}\n    cache = np.zeros((len(studies), N_SLOT, CACHE_SLICES, IMG, IMG), np.uint8)\n    mask = np.zeros((len(studies), N_SLOT), np.float32)\n    log(f'{tag}: cache {cache.shape} = {cache.nbytes / 1024 ** 3:.1f} GB')\n    jobs = [(st, k, plane, slot_map[st][name]) for st in studies for k, (name, plane, _, _) in enumerate(SLOTS) if name in slot_map[st]]\n    n_job = len(jobs)\n    t_ord = time.time()\n    n_slice_total = sum((len(j[3]['files']) for j in jobs))\n    log(f'{tag}: ordering {len(jobs)} slot-series ({n_slice_total} slice headers)')\n    ok = done = 0\n    CHUNK_O = 1024\n    seen = {}\n    if ORDER_CACHE and Path(ORDER_CACHE).is_file():\n        try:\n            import json as _json\n            seen = _json.loads(Path(ORDER_CACHE).read_text())\n        except (OSError, ValueError):\n            seen = {}\n        hit = 0\n        for _, _, _, rec in jobs:\n            e = seen.get(rec['SeriesInstanceUID'])\n            if e and len(e['files']) == len(rec['files']):\n                rec['ordered'] = e['files']\n                ok += int(e['good'])\n                hit += 1\n        jobs = [j for j in jobs if 'ordered' not in j[3]]\n        log(f'{tag}: {hit} slot-series ordered from {ORDER_CACHE}, {len(jobs)} to read')\n    with ThreadPoolExecutor(max_workers=ORDER_THREADS) as pool:\n        for c0 in range(0, len(jobs), CHUNK_O):\n            block = jobs[c0:c0 + CHUNK_O]\n            for (_, _, _, rec), (files, good) in zip(block, pool.map(lambda j: order_slices(j[3]), block)):\n                rec['ordered'] = files\n                ok += int(good)\n                done += 1\n                if ORDER_CACHE:\n                    seen[rec['SeriesInstanceUID']] = {'files': files, 'good': bool(good)}\n            budget = min(ORDER_BUDGET_S, max(60.0, (TIME_BUDGET - (time.time() - T0)) * 0.35))\n            if time.time() - t_ord > budget:\n                log(f'{tag}: ordering budget spent at {done}/{len(jobs)}; the rest keep file order')\n                break\n    if ORDER_CACHE and done:\n        import json as _json\n        _t = Path(ORDER_CACHE).with_suffix('.tmp')\n        _t.write_text(_json.dumps(seen))\n        _t.replace(Path(ORDER_CACHE))\n    log(f'{tag}: ordered {ok}/{n_job} by geometry ({n_job - ok} kept arbitrary) in {time.time() - t_ord:.0f}s')\n    jobs = [(st, k, plane, slot_map[st][name]) for st in studies for k, (name, plane, _, _) in enumerate(SLOTS) if name in slot_map[st]]\n    log(f'{tag}: decoding {len(jobs)} slot-series')\n    n_failed_before = len(DECODE_FAILED)\n    CHUNK = 512\n    done = 0\n    with ThreadPoolExecutor(max_workers=PIX_THREADS) as pool:\n        for c0 in range(0, len(jobs), CHUNK):\n            block = jobs[c0:c0 + CHUNK]\n            for (st, k, plane, _), img in zip(block, pool.map(lambda j: read_slot(j[3], CACHE_SLICES, IMG), block)):\n                done += 1\n                if img is None:\n                    continue\n                cache[sidx[st], k] = normalise_laterality(img, plane, lat_map.get(st)).numpy()\n                mask[sidx[st], k] = 1.0\n            if done % 4096 < CHUNK:\n                log(f'  {tag} {done}/{len(jobs)}')\n            if time.time() - T0 > TIME_BUDGET:\n                log(f'  {tag}: time budget reached during decode')\n                break\n    n_failed = len(DECODE_FAILED) - n_failed_before\n    log(f'{tag}: {int(mask.sum())}/{len(jobs)} slots filled' + (f'; {n_failed} series had a slice that would not decode' if n_failed else ''))\n    gc.collect()\n    return (studies, cache, mask)","execution_count":null,"outputs":[],"id":"reproduction-010"},{"cell_type":"code","metadata":{},"source":"class SlotHead(nn.Module):\n\n    def __init__(self, dim, n_slot, n_out, hidden=256, p=0.2, prior=False):\n        super().__init__()\n        self.proj = nn.Sequential(nn.LayerNorm(dim), nn.Linear(dim, hidden), nn.GELU())\n        self.slot_emb = nn.Parameter(torch.randn(n_slot, hidden) * 0.02)\n        self.query = nn.Parameter(torch.randn(n_out, hidden) * 0.02)\n        self.drop = nn.Dropout(p)\n        self.out = nn.Linear(hidden, n_out)\n        self.hidden = hidden\n        p_ = torch.zeros(n_out, n_slot)\n        if prior and n_slot == len(SLOTS) and (n_out == len(TARGETS)):\n            for t, slots in SLOT_PRIOR_TABLE.items():\n                if t in TARGETS:\n                    p_[TARGETS.index(t), list(slots)] = SLOT_PRIOR_STRENGTH\n        self.prior = prior\n        if prior:\n            self.register_buffer('slot_prior', p_)\n\n    def forward(self, x, mask):\n        h = self.proj(x) + self.slot_emb\n        att = torch.einsum('bsh,oh->bos', h, self.query) / self.hidden ** 0.5\n        if self.prior:\n            att = att + self.slot_prior.unsqueeze(0)\n        att = att.masked_fill(mask.unsqueeze(1) < 0.5, -10000.0).softmax(-1)\n        ctx = self.drop(torch.einsum('bos,bsh->boh', att, h))\n        return (ctx * self.out.weight.unsqueeze(0)).sum(-1) + self.out.bias","execution_count":null,"outputs":[],"id":"reproduction-011"},{"cell_type":"code","metadata":{},"source":"class Model(nn.Module):\n\n    def __init__(self, backbone, dim, pool='cls_mean', prior=False):\n        super().__init__()\n        self.backbone = backbone\n        self.pool = pool\n        self.head = SlotHead(dim * POOL_PARTS[pool], N_SLOT, len(TARGETS), prior=prior)\n        self.register_buffer('mean', torch.tensor([0.485, 0.456, 0.406]).view(1, 3, 1, 1))\n        self.register_buffer('std', torch.tensor([0.229, 0.224, 0.225]).view(1, 3, 1, 1))\n\n    def forward(self, imgs, mask, img_size=None):\n        B, S = imgs.shape[:2]\n        x = imgs.reshape(B * S, *imgs.shape[2:]).float().div_(255.0)\n        if img_size is not None and img_size != x.shape[-1]:\n            x = F.interpolate(x, size=(img_size, img_size), mode='bilinear', align_corners=False)\n        x = (x - self.mean) / self.std\n        out = self.backbone(pixel_values=x).last_hidden_state\n        patch = out[:, 1:]\n        parts = [out[:, 0], patch.mean(1)]\n        if self.pool == 'cls_mean_focal':\n            k = max(1, patch.shape[1] // 8)\n            parts.append(patch.topk(k, dim=1).values.mean(1))\n        feat = torch.cat(parts, dim=1).reshape(B, S, -1)\n        return self.head(feat, mask)","execution_count":null,"outputs":[],"id":"reproduction-012"},{"cell_type":"code","metadata":{},"source":"def build_model(unfreeze_last, source=None, variant='small', pool='cls_mean', prior=False):\n    from transformers import AutoModel\n    p = source if source is not None else find_dinov2(variant)\n    if p is None:\n        raise FileNotFoundError('DINOv2 weights not attached')\n    bb = AutoModel.from_pretrained(str(p))\n    n_layer = len(bb.encoder.layer)\n    for prm in bb.parameters():\n        prm.requires_grad = False\n    for blk in bb.encoder.layer[max(0, n_layer - unfreeze_last):]:\n        for prm in blk.parameters():\n            prm.requires_grad = True\n    for prm in bb.layernorm.parameters():\n        prm.requires_grad = True\n    dim = bb.config.hidden_size\n    trainable = sum((p.numel() for p in bb.parameters() if p.requires_grad))\n    log(f'backbone: {n_layer} blocks, last {unfreeze_last} trainable ({trainable / 1000000.0:.1f}M params), feature dim {dim * POOL_PARTS[pool]}')\n    return Model(bb, dim, pool=pool, prior=prior)","execution_count":null,"outputs":[],"id":"reproduction-013"},{"cell_type":"code","metadata":{},"source":"FINGERPRINT_TOL = 0.002\n\ndef fingerprint(model, dev, img_size, n_slot=None, group=None, seed=None):\n    n_slot = N_SLOT if n_slot is None else n_slot\n    group = GROUP if group is None else group\n    seed = SEED if seed is None else seed\n    g = torch.Generator().manual_seed(seed)\n    imgs = torch.randint(0, 256, (2, n_slot, group, img_size, img_size), generator=g, dtype=torch.uint8).to(dev)\n    mask = torch.ones(2, n_slot, device=dev)\n    mask[1, -1] = 0.0\n    was_training = model.training\n    model.eval()\n    with torch.no_grad():\n        out = model(imgs, mask, img_size).float().cpu().numpy()\n    if was_training:\n        model.train()\n    return out\n\ndef check_fingerprint(model, dev, img_size, expected, tol=FINGERPRINT_TOL, tag=''):\n    got = fingerprint(model, dev, img_size)\n    exp = np.asarray(expected, np.float32)\n    if got.shape != exp.shape:\n        raise WeightsError(f'{tag}fingerprint shape {got.shape} != stored {exp.shape}: the architecture is not the one these weights were fitted to')\n    d = float(np.abs(got - exp).max())\n    if d > tol:\n        raise WeightsError(f'{tag}fingerprint differs by {d:.4g} (tolerance {tol:g}). The weights load but do not compute what they computed when fitted - preprocessing, resolution or architecture has moved between the two runs.')\n    log(f'{tag}fingerprint matches within {d:.2g}')\n    return d\n\nclass WeightsError(RuntimeError):\n    pass\n\ndef find_weights(name='manifest.json'):\n    import json\n    base = Path('/kaggle/input')\n    if not base.is_dir():\n        return None\n    for root, dirs, files in os.walk(base):\n        dirs[:] = [d for d in dirs if d not in ('train_series', 'test_series')]\n        if name not in files:\n            continue\n        try:\n            man = json.loads((Path(root) / name).read_text())\n        except (OSError, ValueError):\n            continue\n        if isinstance(man.get('members'), list) and man['members']:\n            missing = [m['file'] for m in man['members'] if not (Path(root) / m['file']).is_file()]\n            if missing:\n                raise WeightsError(f\"{root} holds a manifest listing {len(man['members'])} members but {len(missing)} of their files are absent (first {missing[0]!r})\")\n            return Path(root)\n    return None\nTTA_OVERLAP = True\nTTA_POOL = 'prob'\nPUBLIC_FRONTIER_TARGET_POOL = {'Fracture': 'max', 'Contusion': 'max', 'Medial Meniscus': 'max', 'Lateral Meniscus': 'max', 'ACL': 'top2', 'MCL': 'top2', \"Baker's\": 'max'}\nTTA_TARGET_POOL = {**PUBLIC_FRONTIER_TARGET_POOL, 'Synovitis': 'original_mean'}\n# No-extra-pass diversity branch: smooth focal pooling is evaluated from\n# the same no-jitter public-member windows already used by the parent.\nLEGACY_FOLD_SOFTPOOL_BETA = {\n    'ACL': 6.0, 'MCL': 6.0,\n    'Medial Meniscus': 8.0, 'Lateral Meniscus': 8.0,\n    \"Baker's\": 8.0, 'Contusion': 8.0, 'Fracture': 10.0,\n}\nLEGACY_FOLD_SOFTPOOL_ALPHA = {\n    'ACL': 0.20, 'MCL': 0.20,\n    'Medial Meniscus': 0.25, 'Lateral Meniscus': 0.25,\n    \"Baker's\": 0.20, 'Contusion': 0.20, 'Fracture': 0.15,\n}\nLEGACY_MEMBER_WEIGHT_BY_TARGET = {'Lateral Meniscus': 15.0, 'Medial OA': 2.5, 'Lateral OA': 15.0, 'Contusion': 5.0}\n\ndef window_starts(n_slice, group, overlap=None):\n    overlap = TTA_OVERLAP if overlap is None else overlap\n    if overlap and n_slice >= group:\n        return list(range(n_slice - group + 1))\n    return [g * group for g in range(max(n_slice // group, 1))]\n\ndef apply_target_window_pool(values, probs, logits, original_probs, mapping, target_idx):\n    for target, mode in mapping.items():\n        j = target_idx[target]\n        if mode == 'max':\n            values[:, j] = probs[:, :, j].max(0).values\n        elif mode == 'mean':\n            values[:, j] = probs[:, :, j].mean(0)\n        elif mode == 'logit_mean':\n            values[:, j] = torch.sigmoid(logits[:, :, j].mean(0))\n        elif mode == 'original_mean':\n            values[:, j] = original_probs[:, :, j].mean(0)\n        elif mode in ('top2', 'top3'):\n            k = min(int(mode[3:]), probs.shape[0])\n            values[:, j] = probs[:, :, j].topk(k, dim=0).values.mean(0)\n        else:\n            raise ValueError(f'unknown TTA pooling mode for {target}: {mode}')\n    return values\n\ndef legacy_fold_soft_window_pool(original_probs, target_idx):\n    values = original_probs.mean(0).clone()\n    for target, beta in LEGACY_FOLD_SOFTPOOL_BETA.items():\n        j = target_idx[target]\n        x = original_probs[:, :, j]\n        weight = torch.softmax(float(beta) * x, dim=0)\n        values[:, j] = (weight * x).sum(0)\n    return values\n\n@torch.no_grad()\ndef predict_member(model, cache, mask, idx, dev, img_size, group=None, pool=None, starts=None, jitter=False, jitter_seed=SEED, return_public_frontier=False):\n    group = GROUP if group is None else group\n    pool = TTA_POOL if pool is None else pool\n    starts = window_starts(cache.shape[2], group) if starts is None else list(starts)\n    if not starts:\n        raise ValueError('predict_member was given no windows to average over')\n    target_idx = {t: j for j, t in enumerate(TARGETS)}\n    unknown = (set(TTA_TARGET_POOL) | set(PUBLIC_FRONTIER_TARGET_POOL)) - set(target_idx)\n    if unknown:\n        raise ValueError(f'unknown target(s) in TTA_TARGET_POOL: {unknown}')\n    jitter_gen = torch.Generator(device=dev)\n    jitter_gen.manual_seed(int(jitter_seed) % (2 ** 63 - 1))\n    model.eval()\n    out, public_frontier_out, public_soft_out = ([], [], [])\n    for b in range(0, len(idx), EVAL_BATCH):\n        sel = idx[b:b + EVAL_BATCH]\n        m = torch.from_numpy(mask[sel]).to(dev)\n        win_probs, win_logits, win_original_probs = ([], [], [])\n        for st in starts:\n            rows = torch.from_numpy(np.ascontiguousarray(cache[sel, :, st:st + group])).to(dev)\n            views = [rows] + ([augment(rows, generator=jitter_gen)] if jitter else [])\n            view_probs, view_logits = ([], [])\n            for view in views:\n                with torch.autocast('cuda', enabled=dev.type == 'cuda'):\n                    z = model(view, m, img_size).float()\n                view_logits.append(z)\n                view_probs.append(torch.sigmoid(z))\n            win_logits.append(torch.stack(view_logits).mean(0))\n            win_probs.append(torch.stack(view_probs).mean(0))\n            win_original_probs.append(view_probs[0])\n        probs = torch.stack(win_probs)\n        logits = torch.stack(win_logits)\n        original_probs = torch.stack(win_original_probs)\n        v = torch.sigmoid(logits.mean(0)) if pool == 'logit' else probs.mean(0)\n        v = apply_target_window_pool(v, probs, logits, original_probs, TTA_TARGET_POOL, target_idx)\n        out.append(v.cpu().numpy())\n        if return_public_frontier:\n            public_v = apply_target_window_pool(original_probs.mean(0), original_probs, logits, original_probs, PUBLIC_FRONTIER_TARGET_POOL, target_idx)\n            public_frontier_out.append(public_v.cpu().numpy())\n            public_soft = legacy_fold_soft_window_pool(original_probs, target_idx)\n            public_soft_out.append(public_soft.cpu().numpy())\n    primary = np.concatenate(out) if out else np.zeros((0, len(TARGETS)), np.float32)\n    if not return_public_frontier:\n        return primary\n    public_frontier = np.concatenate(public_frontier_out) if public_frontier_out else np.zeros((0, len(TARGETS)), np.float32)\n    public_soft = np.concatenate(public_soft_out) if public_soft_out else np.zeros((0, len(TARGETS)), np.float32)\n    return (primary, public_frontier, public_soft)\nBUILD_LOCK = threading.Lock()\nSTATE_LOCK = threading.Lock()\nLEGACY_BUNDLE_FILE = 'rsna_20260807_v1.pt'\nLEGACY_WEIGHT = 0.5\n\ndef find_legacy_bundle():\n    base = Path('/kaggle/input')\n    if not base.is_dir():\n        return None\n    for root, dirs, files in os.walk(base):\n        dirs[:] = [d for d in dirs if d not in ('train_series', 'test_series')]\n        if LEGACY_BUNDLE_FILE in files:\n            return Path(root) / LEGACY_BUNDLE_FILE\n    return None\n\ndef legacy_group_members():\n    return {}\n\ndef _run_member(path, m, dev, Cte, Mte, idx, starts, jitter):\n    t0 = time.time()\n    with BUILD_LOCK:\n        if 'state' in m:\n            state, fp = (m['state'], None)\n        else:\n            ck = torch.load(Path(path) / m['file'], map_location='cpu', weights_only=False)\n            state, fp = (ck['model'], ck.get('fingerprint'))\n        model = build_model(int(m['config']['unfreeze_last']), variant=m['config']['variant'], pool=m['config'].get('pool', 'cls_mean'), prior=bool(m['config'].get('prior', False))).to(dev)\n        model.load_state_dict(state)\n        if fp is not None:\n            check_fingerprint(model, dev, IMG, fp, tag=f\"{m['id']}: \")\n        else:\n            log(f\"  {m['id']}: no stored fingerprint (legacy bundle) -- accepted at reduced weight\")\n    t_ready = time.time()\n    jitter_seed = SEED + int(hashlib.sha256(str(m['id']).encode()).hexdigest()[:8], 16)\n    public_member = 'state' not in m\n    predicted = predict_member(model, Cte, Mte, idx, dev, IMG, starts=starts, jitter=jitter, jitter_seed=jitter_seed, return_public_frontier=public_member)\n    if public_member:\n        p, public_p, public_soft = predicted\n    else:\n        p, public_p, public_soft = (predicted, None, None)\n    t_done = time.time()\n    del model, state\n    gc.collect()\n    if dev.type == 'cuda':\n        with torch.cuda.device(dev):\n            torch.cuda.empty_cache()\n    passes = len(starts) * (2 if jitter else 1)\n    return (p, public_p, public_soft, (t_ready - t0, (t_done - t_ready) / max(passes, 1)))\n\ndef _combine(per_member):\n    all_ids = sorted({s for m in per_member for s in m['ids']})\n    pos = {s: i for i, s in enumerate(all_ids)}\n    acc = np.zeros((len(all_ids), len(TARGETS)), np.float64)\n    tot = np.zeros(len(TARGETS), np.float64)\n    for m in per_member:\n        target_weight = m.get('target_weight')\n        w = np.asarray(target_weight if target_weight is not None else [float(m.get('weight', 1.0))] * len(TARGETS), dtype=np.float64)\n        if w.shape != (len(TARGETS),) or np.any(w < 0):\n            raise ValueError(f\"invalid target weights for {m.get('id')}: {w}\")\n        r = pd.DataFrame(m['pred']).rank(pct=True).to_numpy()\n        acc[[pos[s] for s in m['ids']]] += r * w[None, :]\n        tot += w\n    if np.any(tot <= 0):\n        raise ValueError(f'at least one target has no ensemble vote: {tot}')\n    return (all_ids, acc / tot[None, :])\n\ndef combine_public_members_by_fold(per_member, pred_key='pred'):\n    # Raw-average the four members within each fold, rank each fold,\n    # then give all five folds equal weight.\n    all_ids = sorted({study for member in per_member for study in member['ids']})\n    position = {study: i for i, study in enumerate(all_ids)}\n    groups = {}\n    for i, member in enumerate(per_member):\n        fold = member.get('fold')\n        key = f'fold_{fold}' if fold is not None else f'member_{i}'\n        groups.setdefault(key, []).append(member)\n    fold_ranks, diagnostics = ([], [])\n    for key, members_in_fold in sorted(groups.items()):\n        matrices = []\n        for member in members_in_fold:\n            values = np.full((len(all_ids), len(TARGETS)), np.nan, np.float64)\n            values[[position[study] for study in member['ids']]] = np.asarray(member[pred_key], np.float64)\n            if np.isnan(values).any():\n                raise WeightsError(f\"{member.get('id')}: incomplete {pred_key} coverage\")\n            matrices.append(values)\n        raw_fold_mean = np.mean(matrices, axis=0)\n        fold_ranks.append(pd.DataFrame(raw_fold_mean).rank(method='average', pct=True).to_numpy(np.float64))\n        diagnostics.append({'ensemble_group': key, 'members': len(members_in_fold)})\n    if len(fold_ranks) != 5:\n        raise WeightsError(f'legacy branch requires five folds, found {len(fold_ranks)}')\n    return all_ids, np.mean(fold_ranks, axis=0), pd.DataFrame(diagnostics)\n\ndef blend_legacy_frontier_and_soft(frontier_rank, soft_rank):\n    output = np.asarray(frontier_rank, np.float64).copy()\n    for j, target in enumerate(TARGETS):\n        alpha = float(LEGACY_FOLD_SOFTPOOL_ALPHA.get(target, 0.0))\n        if alpha:\n            output[:, j] = (1.0 - alpha) * frontier_rank[:, j] + alpha * soft_rank[:, j]\n    return output\n\ndef infer_from_package(path, dev=None):\n    man = json.loads((Path(path) / 'manifest.json').read_text())\n    members = man['members']\n    log(f'weights package: {len(members)} member(s) from {path}; {len(DEVS)} device(s)')\n    test_df = pd.read_csv(ROOT / 'test.csv')\n    test_series = pd.read_csv(ROOT / 'test_series.csv')\n    plane_map = dict(zip(test_series['SeriesInstanceUID'], test_series['Anatomical_Plane']))\n    hte = annotate(walk('test_series'))\n    log(f'test header pass: {len(hte)} series')\n    groups = {}\n    for m in members:\n        groups.setdefault(m['pixel_group'], []).append(m)\n    groups.update(legacy_group_members())\n    per_member, public_frontier_members = ([], [])\n    est = {'fixed': None, 'win': None}\n\n    def bank(m, ids, pred, starts, jitter, public_pred=None, public_soft=None):\n        if float(np.std(pred)) < 1e-09:\n            log(f\"  {m['id']}: degenerate predictions; not banked\")\n            return\n        with STATE_LOCK:\n            per_member.append({'id': m['id'], 'fold': m.get('fold'), 'ids': ids, 'pred': pred, 'weight': m.get('weight', 1.0), 'target_weight': m.get('target_weight'), 'holdout': m.get('holdout')})\n            if public_pred is not None and len(starts) == len(starts_full):\n                if float(np.std(public_pred)) < 1e-09:\n                    raise WeightsError(f\"{m['id']}: degenerate public-frontier prediction\")\n                public_frontier_members.append({'id': m['id'], 'fold': m.get('fold'), 'ids': ids, 'pred': public_pred, 'soft_pred': public_soft})\n            elif public_pred is not None:\n                log(f\"  {m['id']}: public-frontier vote omitted because only {len(starts)} / {len(starts_full)} windows completed\")\n            log(f\"  banked {m['id']} fold {m.get('fold', '?')} ({len(starts)} window(s); {len(per_member)} member(s)\")\n    for gi, (key, gm) in enumerate(groups.items(), 1):\n        cfg = json.loads(key)\n        adopt_config_globals(cfg)\n        log(f\"decode group {gi}/{len(groups)}: {cfg['img']}px x {cfg['slices']} slices, crop {cfg['crop_mm']} mm -> {len(gm)} member(s)\")\n        st_te, Cte, Mte = build_cache(pick_slots(hte, plane_map), plane_map, lat_of(hte, 'test '), f'test g{gi}')\n        idx = np.arange(len(st_te))\n        starts_full = window_starts(Cte.shape[2], GROUP)\n        pending = sorted(gm, key=lambda m: -(m.get('holdout') or 0))\n        left_after = sum((len(g) for j, (_, g) in enumerate(groups.items(), 1) if j > gi))\n\n        def pop_next():\n            with STATE_LOCK:\n                if not pending:\n                    return (None, None, False)\n                left = TIME_BUDGET - (time.time() - T0)\n                remaining = len(pending) + left_after\n                slots_left = -(-remaining // len(DEVS))\n                starts, jit = (starts_full, False)\n                if est['fixed'] is not None and est['win'] is not None:\n                    afford = max(left * 0.9, 0.0)\n                    room = afford / max(slots_left, 1)\n                    if est['fixed'] + est['win'] > room:\n                        log(f'  {left / 60:.0f} min left: surrendering {len(pending)} member(s); not one more fits')\n                        pending.clear()\n                        return (None, None, False)\n                    per_win = est['win'] * (2 if jit else 1)\n                    n_win = int((room - est['fixed']) / per_win) if per_win > 0 else len(starts_full)\n                    n_win = max(1, min(len(starts_full), n_win))\n                    if n_win < len(starts_full):\n                        mid = (len(starts_full) - n_win) // 2\n                        starts = starts_full[mid:mid + n_win]\n                return (pending.pop(0), starts, jit)\n\n        def worker(dev):\n            others = [d for d in DEVS if d is not dev]\n            while True:\n                m, starts, jit = pop_next()\n                if m is None:\n                    return\n                for attempt, d in enumerate([dev] + others[:1]):\n                    try:\n                        p, public_p, public_soft, (fs, ws) = _run_member(path, m, d, Cte, Mte, idx, starts, jit)\n                        with STATE_LOCK:\n                            est['fixed'], est['win'] = (fs, ws)\n                        bank(m, st_te, p, starts, jit, public_p, public_soft)\n                        break\n                    except Exception as exc:\n                        log(f\"  MEMBER {m['id']} failed on {d} ({type(exc).__name__}: {exc}); \" + ('retrying on peer device' if attempt == 0 and others else 'dropped -- costs one vote, not the run'))\n                        if d.type == 'cuda':\n                            with torch.cuda.device(d):\n                                torch.cuda.empty_cache()\n        threads = [threading.Thread(target=worker, args=(d,)) for d in DEVS]\n        for t in threads:\n            t.start()\n        for t in threads:\n            t.join()\n        del Cte, Mte\n        gc.collect()\n    if len(public_frontier_members) != len(members):\n        raise WeightsError(f'public-frontier inference incomplete: {len(public_frontier_members)} / {len(members)} members')\n    frontier_ids, frontier_acc = _combine(public_frontier_members)\n    sub = write_submission(frontier_acc, frontier_ids, test_df, 'submission.csv')\n    log(f'submission.csv = exact no-jitter public-frontier rank mean of {len(public_frontier_members)} member(s); {sub.shape}')\n    return sub\n\ndef adopt_config_globals(cfg):\n    global IMG, CACHE_IMG, GROUP, CACHE_SLICES, N_GROUP, CROP_MM, SLICE_BAND, RULES\n    CACHE_IMG = IMG = int(cfg['img'])\n    GROUP = int(cfg['group'])\n    CACHE_SLICES = int(cfg['slices'])\n    N_GROUP = max(CACHE_SLICES // GROUP, 1)\n    CROP_MM = float(cfg['crop_mm'])\n    SLICE_BAND = tuple((float(x) for x in cfg['band']))\n    rules = cfg.get('rules') or RULES_NATIVE\n    unknown = {k: v for k, v in rules.items() if k not in RULES_NATIVE or v not in (RULES_NATIVE[k], RULES_LEGACY[k])}\n    if unknown:\n        raise WeightsError(f'the members record pixel rules this pipeline cannot reproduce: {unknown}')\n    RULES = {**RULES_NATIVE, **rules}\n    if [s[0] for s in SLOTS] != list(cfg['slots']):\n        raise WeightsError(f\"the members were fitted on slots {cfg['slots']} and this pipeline defines {[s[0] for s in SLOTS]}; a weight would be read against the wrong slot\")","execution_count":null,"outputs":[],"id":"reproduction-014"},{"cell_type":"code","metadata":{},"source":"def write_submission(pred, studies, test_df, path):\n    sub = pd.DataFrame(pd.DataFrame(pred).rank(pct=True).values, columns=TARGETS)\n    sub.insert(0, 'StudyInstanceUID', studies)\n    sub = test_df[['StudyInstanceUID']].merge(sub, on='StudyInstanceUID', how='left')\n    sub[TARGETS] = sub[TARGETS].fillna(0.5)\n    sub.to_csv(path, index=False)\n    return sub\n\ndef _v37_validate_submission(path, test_df, tag):\n    frame = pd.read_csv(path)\n    expected = ['StudyInstanceUID'] + TARGETS\n    if list(frame.columns) != expected:\n        raise ValueError(f'{tag}: columns differ from the competition contract')\n    if len(frame) != len(test_df) or not frame['StudyInstanceUID'].is_unique:\n        raise ValueError(f'{tag}: row count or StudyInstanceUID uniqueness failed')\n    if set(frame['StudyInstanceUID'].astype(str)) != set(test_df['StudyInstanceUID'].astype(str)):\n        raise ValueError(f'{tag}: StudyInstanceUID set differs from test.csv')\n    values = frame[TARGETS].to_numpy(np.float64)\n    if not np.isfinite(values).all():\n        raise ValueError(f'{tag}: non-finite prediction')\n    return frame\n\ndef main():\n    pkg = find_weights()\n    if pkg is None:\n        raise WeightsError('required public checkpoint manifest not found')\n    infer_from_package(pkg, DEVS[0])\n    _v37_validate_submission('submission.csv', pd.read_csv(ROOT / 'test.csv'), 'public frontier')\n","execution_count":null,"outputs":[],"id":"reproduction-015"},{"cell_type":"code","metadata":{},"source":"\"\"\"Check raw DINO output before the inherited merge/fill/rank writer.\"\"\"\n_rep_original_dino_writer = write_submission\n_rep_original_dino_combine = _combine\n_rep_dino_raw_members_checked = 0\n\ndef _rep_checked_dino_combine(members):\n    global _rep_dino_raw_members_checked\n    import numpy as _np\n    import pandas as _pd\n    wanted = _pd.read_csv(ROOT / 'test.csv', dtype={'StudyInstanceUID': str}).StudyInstanceUID.tolist()\n    assert len(members) == 20, 'DINO raw member count'\n    assert len({member['id'] for member in members}) == 20, 'DINO duplicate members'\n    for member in members:\n        actual = [str(study) for study in member['ids']]\n        raw = _np.asarray(member['pred'])\n        assert raw.shape == (len(wanted), 12), 'DINO member raw shape'\n        assert _np.isfinite(raw).all(), 'DINO member raw nonfinite'\n        assert len(actual) == len(set(actual)) == len(wanted), 'DINO member raw UID uniqueness'\n        assert set(actual) == set(wanted), 'DINO member missing test studies'\n        _rep_dino_raw_members_checked += 1\n    return _rep_original_dino_combine(members)\n\ndef _rep_checked_dino_writer(pred, studies, test_df, path):\n    import numpy as _np\n    raw = _np.asarray(pred)\n    actual = [str(study) for study in studies]\n    wanted = test_df['StudyInstanceUID'].astype(str).tolist()\n    assert raw.shape == (len(wanted), 12), 'DINO raw prediction shape'\n    assert _np.isfinite(raw).all(), 'DINO raw nonfinite predictions'\n    assert len(actual) == len(set(actual)) == len(wanted), 'DINO raw UID uniqueness'\n    assert set(actual) == set(wanted), 'DINO raw missing test studies'\n    return _rep_original_dino_writer(pred, studies, test_df, path)\n\nwrite_submission = _rep_checked_dino_writer\n_combine = _rep_checked_dino_combine\n_rep_dino_guard_installed = True\n","execution_count":null,"outputs":[],"id":"reproduction-016"},{"cell_type":"code","metadata":{},"source":"main()\nlog('done')","execution_count":null,"outputs":[],"id":"reproduction-017"},{"cell_type":"code","metadata":{},"source":"_rep_snapshot('dino')\n","execution_count":null,"outputs":[],"id":"reproduction-018"},{"cell_type":"markdown","metadata":{},"source":"## 2. A5: add five attention-pooling folds\n\nThese folds use their own saved DINOv3 configuration. Their contribution is fixed at 45% of the DINO/A5 blend. The source restores its earlier globals afterwards because the next stage reuses that reader.\n","id":"reproduction-019"},{"cell_type":"code","metadata":{},"source":"_A5_SAVED = dict(globals())\nimport gc, os, time, warnings\nfrom concurrent.futures import ProcessPoolExecutor, as_completed\nfrom pathlib import Path\nimport cv2\nimport numpy as np\nimport pandas as pd\nimport pydicom\nimport timm\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nwarnings.filterwarnings('ignore')\ncv2.setNumThreads(1)\nCROP_MM = 130.0\nSIZE = 336\nSLICE_BAND = (0.12, 0.88)\nN_SLICE = 16\nINTENSITY = 'slice'\nSLOTS = [('Sagittal', 1), ('Sagittal', 0), ('Coronal', 1), ('Coronal', 0), ('Axial', 1), ('Axial', 0)]\nN_SLOT = len(SLOTS)\nLABELS = ['ACL', 'MCL', 'Medial Meniscus', 'Lateral Meniscus', 'Medial OA', 'Lateral OA', 'PF OA', 'Effusion', 'Synovitis', \"Baker's\", 'Contusion', 'Fracture']\n\ndef _find_dir(*names):\n    root = Path('/kaggle/input')\n    cand = []\n    for n in names:\n        cand += [root / n, root / 'competitions' / n, root / 'datasets' / n]\n        for parent in (root / 'datasets', root / 'competitions', root):\n            if parent.is_dir():\n                try:\n                    cand += [d / n for d in parent.iterdir() if d.is_dir()]\n                except OSError:\n                    pass\n    for p in cand:\n        if p.is_dir():\n            return p\n    return None\nCOMP = _find_dir('rsna-knee-abnormality-detection')\nCKPT = _find_dir('knee-mri-fold-weights')\nassert COMP is not None, 'competition data not attached'\nassert CKPT is not None, 'fold weights not attached'\nassert (COMP / 'sample_submission.csv').exists(), f'no competition data at {COMP}'\nassert list(CKPT.glob('*_f*.pt')), f'no checkpoints at {CKPT}'\nDEV = 'cuda' if torch.cuda.is_available() else 'cpu'\n","execution_count":null,"outputs":[],"id":"reproduction-020"},{"cell_type":"code","metadata":{},"source":"SERIES_ROOT = COMP / 'test_series'\nif not SERIES_ROOT.exists():\n    SERIES_ROOT = COMP / 'train_series'\nprint('series root:', SERIES_ROOT)\n\ndef ordered_files(sdir, cap=64):\n    keyed = []\n    for f in sdir.glob('*.dcm'):\n        try:\n            ds = pydicom.dcmread(str(f), stop_before_pixels=True)\n            keyed.append((int(ds.InstanceNumber), str(f)))\n        except Exception:\n            continue\n        if len(keyed) >= cap * 4:\n            break\n    return [f for _, f in sorted(keyed)]\n\ndef series_side(path):\n    try:\n        return float(pydicom.dcmread(path, stop_before_pixels=True).ImagePositionPatient[0])\n    except Exception:\n        return 0.0\n\ndef read_crop(path):\n    try:\n        ds = pydicom.dcmread(path)\n        arr = ds.pixel_array.astype(np.float32)\n    except Exception:\n        return None\n    try:\n        ps = float(ds.PixelSpacing[0])\n    except Exception:\n        ps = CROP_MM / max(arr.shape)\n    half = int(round(CROP_MM / ps / 2))\n    cy, cx = (arr.shape[0] // 2, arr.shape[1] // 2)\n    y0, y1 = (max(0, cy - half), min(arr.shape[0], cy + half))\n    x0, x1 = (max(0, cx - half), min(arr.shape[1], cx + half))\n    crop = arr[y0:y1, x0:x1]\n    return None if crop.size == 0 else crop\n\ndef window(crop, lo, hi, flip):\n    c = np.clip((crop - lo) / max(hi - lo, 1e-06), 0, 1)\n    img = cv2.resize(c, (SIZE, SIZE), interpolation=cv2.INTER_AREA)\n    return img[:, ::-1].copy() if flip else img\n\ndef render(path, flip):\n    crop = read_crop(path)\n    if crop is None:\n        return None\n    lo, hi = np.percentile(crop[::4, ::4], [1, 99])\n    return window(crop, lo, hi, flip)\n\ndef build_study(args):\n    idx, study, recs = args\n    out = np.zeros((N_SLOT, N_SLICE, SIZE, SIZE), np.uint8)\n    mask = np.zeros(N_SLOT, np.uint8)\n    rows = pd.DataFrame(recs)\n    if len(rows):\n        for s_i, (plane, fs) in enumerate(SLOTS):\n            sub = rows[(rows.Anatomical_Plane == plane) & (rows.Fat_Suppression == fs)]\n            if sub.empty:\n                continue\n            files = ordered_files(SERIES_ROOT / study / sub.iloc[0].SeriesInstanceUID)\n            if not files:\n                continue\n            flip = plane != 'Sagittal' and series_side(files[0]) < 0\n            lo, hi = SLICE_BAND\n            i0 = int(round(lo * (len(files) - 1)))\n            i1 = int(round(hi * (len(files) - 1)))\n            avail = list(range(i0, i1 + 1))\n            if len(avail) >= N_SLICE:\n                picks = [avail[int(round(t))] for t in np.linspace(0, len(avail) - 1, N_SLICE)]\n                off = 0\n            else:\n                picks, off = (avail, (N_SLICE - len(avail)) // 2)\n            if INTENSITY == 'series':\n                crops = [read_crop(files[p]) for p in picks]\n                got = [x for x in crops if x is not None]\n                if got:\n                    samp = np.concatenate([x[::4, ::4].ravel() for x in got])\n                    lo_, hi_ = np.percentile(samp, [1, 99])\n                    for c, x in enumerate(crops):\n                        if x is None:\n                            x = read_crop(files[min(len(files) - 1, picks[c] + 1)])\n                        if x is not None:\n                            out[s_i, off + c] = (window(x, lo_, hi_, flip) * 255).astype(np.uint8)\n            else:\n                for c, p in enumerate(picks):\n                    img = render(files[p], flip)\n                    if img is None:\n                        img = render(files[min(len(files) - 1, p + 1)], flip)\n                    if img is not None:\n                        out[s_i, off + c] = (img * 255).astype(np.uint8)\n            mask[s_i] = len(picks)\n    return (idx, out, mask)\nsub_df = pd.read_csv(COMP / 'sample_submission.csv')\nser_csv = pd.read_csv(COMP / 'test_series.csv')\nif not (COMP / 'test_series').exists():\n    ser_csv = pd.read_csv(COMP / 'train_series.csv')\nser_csv = ser_csv.loc[:, ~ser_csv.columns.duplicated()]\nstudies = sub_df.StudyInstanceUID.tolist()\nby = {s: g.to_dict('records') for s, g in ser_csv[ser_csv.StudyInstanceUID.isin(set(studies))].groupby('StudyInstanceUID')}\nprint(f'{len(studies):,} test studies, {len(by):,} with series metadata')","execution_count":null,"outputs":[],"id":"reproduction-021"},{"cell_type":"code","metadata":{},"source":"N_SLOT_TYPES, MASK_IDX = (6, 0)\n\ndef segment_softmax(scores, sidx, B):\n    T, K = scores.shape\n    idx = sidx.unsqueeze(1).expand(-1, K)\n    m = torch.full((B, K), float('-inf'), device=scores.device, dtype=scores.dtype)\n    m = m.scatter_reduce(0, idx, scores, reduce='amax', include_self=True)\n    e = (scores - m[sidx]).exp()\n    s = torch.zeros(B, K, device=scores.device, dtype=scores.dtype).index_add_(0, sidx, e)\n    return e / s[sidx].clamp(min=1e-06)\n\nclass MeanMaxPool(nn.Module):\n\n    def forward(self, f, sidx, B, slot=None, return_attn=False):\n        D = f.shape[1]\n        cnt = torch.zeros(B, device=f.device, dtype=f.dtype).index_add_(0, sidx, torch.ones(f.shape[0], device=f.device, dtype=f.dtype))\n        mean = torch.zeros(B, D, device=f.device, dtype=f.dtype).index_add_(0, sidx, f)\n        mean = mean / cnt.clamp(min=1).unsqueeze(1)\n        mx = torch.full((B, D), -10000.0, device=f.device, dtype=f.dtype)\n        mx = mx.scatter_reduce(0, sidx.unsqueeze(1).expand(-1, D), f, reduce='amax', include_self=True)\n        return (torch.cat([mean, mx], 1), None)\n\nclass LabelAttentionPool(nn.Module):\n\n    def __init__(self, d, n_labels=12, n_heads=4, slot_bias=True):\n        super().__init__()\n        self.d, self.k, self.h = (d, n_labels, n_heads)\n        self.q = nn.Parameter(torch.randn(n_labels, d) * 0.02)\n        self.key, self.val = (nn.Linear(d, d), nn.Linear(d, d))\n        self.slot_bias = nn.Parameter(torch.zeros(n_labels, N_SLOT_TYPES + 1)) if slot_bias else None\n\n    def forward(self, f, sidx, B, slot=None, return_attn=False):\n        scores = self.key(f) @ self.q.t() / self.d ** 0.5\n        if self.slot_bias is not None and slot is not None:\n            scores = scores + self.slot_bias.t()[slot]\n        a = segment_softmax(scores, sidx, B)\n        out = torch.zeros(B, self.k, self.d, device=f.device, dtype=f.dtype)\n        out = out.index_add_(0, sidx, a.unsqueeze(-1) * self.val(f).unsqueeze(1))\n        return (out, a)\n\nclass TokenXAttnPool(nn.Module):\n\n    def __init__(self, d, n_labels=12, n_heads=6, dropout=0.2):\n        super().__init__()\n        self.d, self.k = (d, n_labels)\n        self.q = nn.Parameter(torch.randn(n_labels, d) * 0.02)\n        self.slot_emb = nn.Embedding(N_SLOT_TYPES + 1, d, padding_idx=0)\n        self.kv_norm = nn.LayerNorm(d)\n        self.attn = nn.MultiheadAttention(d, n_heads, dropout=dropout, batch_first=True)\n\n    def forward(self, tok, sidx, B, slot=None, return_attn=False):\n        T, N, D = tok.shape\n        cnt = torch.bincount(sidx, minlength=B)\n        S = int(cnt.max().item())\n        starts = torch.cumsum(cnt, 0) - cnt\n        pos = torch.arange(T, device=tok.device) - starts[sidx]\n        kv = tok + self.slot_emb(slot).unsqueeze(1)\n        pad = tok.new_zeros(B, S, N, D)\n        pad[sidx, pos] = kv\n        keep = torch.zeros(B, S, dtype=torch.bool, device=tok.device)\n        keep[sidx, pos] = True\n        kpm = ~keep.repeat_interleave(N, dim=1)\n        pad = self.kv_norm(pad.reshape(B, S * N, D))\n        q = self.q.unsqueeze(0).expand(B, -1, -1)\n        att, w = self.attn(q, pad, pad, key_padding_mask=kpm, need_weights=return_attn, average_attn_weights=True)\n        cls = tok[:, 0]\n        mean = torch.zeros(B, D, device=tok.device, dtype=tok.dtype).index_add_(0, sidx, cls) / cnt.clamp(min=1).unsqueeze(1)\n        mx = torch.full((B, D), -10000.0, device=tok.device, dtype=tok.dtype)\n        mx = mx.scatter_reduce(0, sidx.unsqueeze(1).expand(-1, D), cls, reduce='amax', include_self=True)\n        base = torch.cat([mean, mx], 1).unsqueeze(1).expand(-1, self.k, -1)\n        return (torch.cat([att, base], -1), w)\n\nclass ViTSlotToken(nn.Module):\n\n    def __init__(self, vit, n_cat, dim=None):\n        super().__init__()\n        self.vit = vit\n        d = dim or vit.embed_dim\n        self.tok = nn.Embedding(n_cat + 1, d, padding_idx=MASK_IDX)\n        self.num_features = vit.num_features\n        self._orig_prefix = getattr(vit, 'num_prefix_tokens', 1)\n        vit.num_prefix_tokens = self._orig_prefix + 1\n        for blk in vit.blocks:\n            a = getattr(blk, 'attn', None)\n            if a is not None and hasattr(a, 'num_prefix_tokens'):\n                a.num_prefix_tokens = a.num_prefix_tokens + 1\n\n    @staticmethod\n    def _maybe(mod, x):\n        return x if mod is None else mod(x)\n\n    def forward_features(self, x, cat):\n        v = self.vit\n        x = v.patch_embed(x)\n        pos = v._pos_embed(x)\n        rope = None\n        if isinstance(pos, tuple):\n            x, rope = pos\n        else:\n            x = pos\n        x = self._maybe(getattr(v, 'patch_drop', None), x)\n        x = self._maybe(getattr(v, 'norm_pre', None), x)\n        npt = self._orig_prefix\n        tok = self.tok(cat).unsqueeze(1)\n        x = torch.cat([x[:, :npt], tok, x[:, npt:]], dim=1)\n        if rope is not None:\n            if getattr(v, 'rope_mixed', False):\n                for i, blk in enumerate(v.blocks):\n                    x = blk(x, rope=rope[i])\n            else:\n                for blk in v.blocks:\n                    x = blk(x, rope=rope)\n        else:\n            x = v.blocks(x)\n        return v.norm(x)\n\n    def forward_head(self, x, pre_logits=True):\n        return self.vit.forward_head(x, pre_logits=pre_logits)\nIMAGENET_MEAN = (0.485, 0.456, 0.406)\nIMAGENET_STD = (0.229, 0.224, 0.225)\n\nclass _GatedDepthBlock(nn.Module):\n\n    def __init__(self, n_slice, dropout=0.0, ls_init=0.1):\n        super().__init__()\n        self.norm = nn.GroupNorm(1, n_slice)\n        self.v = nn.Conv2d(n_slice, n_slice, 1)\n        self.g = nn.Conv2d(n_slice, n_slice, 1)\n        self.out = nn.Conv2d(n_slice, n_slice, 1)\n        self.gamma = nn.Parameter(torch.full((n_slice, 1, 1), ls_init))\n        self.drop = nn.Dropout2d(dropout) if dropout else nn.Identity()\n\n    def forward(self, x):\n        z = self.norm(x)\n        return x + self.gamma * self.drop(self.out(self.v(z) * F.silu(self.g(z))))\n\nclass DepthCompress(nn.Module):\n\n    def __init__(self, n_slice=16, out_ch=3, depth=1, dropout=0.0, ls_init=0.1, imagenet=True, proj_noise=0.25):\n        super().__init__()\n        self.imagenet = imagenet\n        self.blocks = nn.ModuleList([_GatedDepthBlock(n_slice, dropout, ls_init) for _ in range(depth)])\n        self.proj = nn.Conv2d(n_slice, out_ch, 1, bias=True)\n        if imagenet:\n            self.register_buffer('mu', torch.tensor(IMAGENET_MEAN).view(1, -1, 1, 1))\n            self.register_buffer('sd', torch.tensor(IMAGENET_STD).view(1, -1, 1, 1))\n\n    def forward(self, x):\n        keep = (x.amax(dim=1, keepdim=True) > 0).to(x.dtype)\n        z = x\n        for b in self.blocks:\n            z = b(z)\n        z = self.proj(z)\n        if self.imagenet:\n            z = (z - self.mu.to(z.dtype)) / self.sd.to(z.dtype)\n        return z * keep\nN_PLANE, N_CONTRAST = (3, 2)\n_PLANE_OF = lambda s: torch.clamp(s - 1, 0, 5) // 2\n_CONTRAST_OF = lambda s: torch.clamp(s - 1, 0, 5) % 2\n\nclass SlotDepthMixer(nn.Module):\n\n    def __init__(self, n_slice=16, ksize=5, alpha_max=0.25):\n        super().__init__()\n        self.n_slice, self.ksize, self.r = (n_slice, ksize, ksize // 2)\n        self.alpha_max = alpha_max\n        b = torch.tensor([1.0, 4.0, 6.0, 4.0, 1.0])\n        self.register_buffer('base', b.log()[self.r:])\n        n_u = self.r + 1\n        self.shared = nn.Parameter(torch.zeros(n_u))\n        self.plane_k = nn.Parameter(torch.zeros(N_PLANE, n_u))\n        self.contrast_k = nn.Parameter(torch.zeros(N_CONTRAST, n_u))\n        self.g0 = nn.Parameter(torch.zeros(()))\n        self.gate_p = nn.Parameter(torch.zeros(N_PLANE))\n        self.gate_c = nn.Parameter(torch.zeros(N_CONTRAST))\n        idx = torch.arange(n_slice)\n        self.register_buffer('off', idx[None, :] - idx[:, None])\n\n    def kernel(self, slot):\n        p, c = (_PLANE_OF(slot), _CONTRAST_OF(slot))\n        half = self.base + self.shared + self.plane_k[p] + self.contrast_k[c]\n        full = torch.cat([half.flip(-1)[..., :self.r], half], dim=-1)\n        return F.softmax(full, dim=-1)\n\n    def alpha(self, slot):\n        p, c = (_PLANE_OF(slot), _CONTRAST_OF(slot))\n        return self.alpha_max * torch.tanh(self.g0 + self.gate_p[p] + self.gate_c[c])\n\n    def forward(self, x, slot, vmask):\n        T, S, H, W = x.shape\n        if vmask is None:\n            raise ValueError('stem=mixer requires the padding mask')\n        k = self.kernel(slot)\n        v = vmask.to(k.dtype)\n        d = self.off + self.r\n        inb = (d >= 0) & (d < self.ksize)\n        kk = k[:, d.clamp(0, self.ksize - 1)] * inb\n        M = kk * v[:, None, :]\n        den = M.sum(-1, keepdim=True)\n        eye = torch.eye(S, device=x.device, dtype=M.dtype).expand(T, S, S)\n        ok = (den > 1e-06) & v[:, :, None].bool()\n        M = torch.where(ok, M / den.clamp(min=1e-06), eye)\n        a = self.alpha(slot)[:, None, None]\n        Aop = ((1.0 - a) * eye + a * M).to(x.dtype)\n        if x.is_contiguous(memory_format=torch.channels_last) and (not x.is_contiguous()):\n            y = torch.bmm(x.permute(0, 2, 3, 1).reshape(T, H * W, S), Aop.transpose(1, 2))\n            return y.reshape(T, H, W, S).permute(0, 3, 1, 2)\n        return torch.bmm(Aop, x.reshape(T, S, H * W)).reshape(T, S, H, W)\n\ndef _seg_mean_max(v, sidx, B):\n    D = v.shape[1]\n    cnt = torch.zeros(B, device=v.device, dtype=v.dtype).index_add_(0, sidx, torch.ones(v.shape[0], device=v.device, dtype=v.dtype))\n    mean = torch.zeros(B, D, device=v.device, dtype=v.dtype).index_add_(0, sidx, v)\n    mean = mean / cnt.clamp(min=1).unsqueeze(1)\n    mx = torch.full((B, D), -10000.0, device=v.device, dtype=v.dtype)\n    mx = mx.scatter_reduce(0, sidx.unsqueeze(1).expand(-1, D), v, reduce='amax', include_self=True)\n    return torch.cat([mean, mx], 1)\n\ndef _pad_kv(x, sidx, B, norm):\n    T, P, D = x.shape\n    cnt = torch.bincount(sidx, minlength=B)\n    S = int(cnt.max().item())\n    starts = torch.cumsum(cnt, 0) - cnt\n    pos = torch.arange(T, device=x.device) - starts[sidx]\n    pad = x.new_zeros(B, S, P, D)\n    pad[sidx, pos] = x\n    keep = torch.zeros(B, S, dtype=torch.bool, device=x.device)\n    keep[sidx, pos] = True\n    return (norm(pad.reshape(B, S * P, D)), ~keep.repeat_interleave(P, dim=1))\n\nclass _GatedDelta(nn.Module):\n\n    def __init__(self, d, n_labels, n_heads, dropout):\n        super().__init__()\n        self.q = nn.Parameter(torch.randn(n_labels, d) * 0.02)\n        self.kv_norm = nn.LayerNorm(d)\n        self.attn = nn.MultiheadAttention(d, n_heads, dropout=dropout, batch_first=True)\n        self.d_norm = nn.LayerNorm(d)\n        self.dw = nn.Parameter(torch.randn(n_labels, d) * (1.0 / d ** 0.5))\n        self.db = nn.Parameter(torch.zeros(n_labels))\n        self.gate = nn.Parameter(torch.zeros(n_labels))\n\n    def delta(self, pat, sidx, B, return_attn):\n        kv, kpm = _pad_kv(pat, sidx, B, self.kv_norm)\n        q = self.q.unsqueeze(0).expand(B, -1, -1)\n        att, w = self.attn(q, kv, kv, key_padding_mask=kpm, need_weights=return_attn, average_attn_weights=True)\n        return ((self.d_norm(att) * self.dw).sum(-1) + self.db, w)\n\nclass TokenResidualPool(_GatedDelta):\n\n    def __init__(self, d, n_labels=12, n_heads=6, pe=64, dropout=0.2):\n        super().__init__(d, n_labels, n_heads, dropout)\n        self.base = nn.Sequential(nn.LayerNorm(2 * d + pe), nn.Dropout(dropout), nn.Linear(2 * d + pe, n_labels))\n\n    def forward(self, tok, slot, sidx, B, pres, return_attn=False):\n        base = self.base(torch.cat([_seg_mean_max(tok[:, 1:].mean(1), sidx, B), pres], 1))\n        d_, w = self.delta(tok[:, 1:], sidx, B, return_attn)\n        return (base + self.gate * d_, w)\n\nclass CodexResidualPool(_GatedDelta):\n\n    def __init__(self, d, n_labels=12, n_heads=6, pe=64, dropout=0.2):\n        super().__init__(d, n_labels, n_heads, dropout)\n        self.base = nn.Sequential(nn.LayerNorm(2 * d + pe), nn.Dropout(dropout), nn.Linear(2 * d + pe, n_labels))\n\n    def forward(self, tok, slot, sidx, B, pres, return_attn=False):\n        base = self.base(torch.cat([_seg_mean_max(tok[:, 0], sidx, B), pres], 1))\n        d_, w = self.delta(tok[:, 1:], sidx, B, return_attn)\n        return (base + self.gate * d_, w)\n\nclass ClsAddPool(nn.Module):\n\n    def __init__(self, d, n_labels=12, pe=64, dropout=0.2):\n        super().__init__()\n        self.net = nn.Sequential(nn.LayerNorm(4 * d + pe), nn.Dropout(dropout), nn.Linear(4 * d + pe, n_labels))\n\n    def forward(self, tok, slot, sidx, B, pres, return_attn=False):\n        return (self.net(torch.cat([_seg_mean_max(tok[:, 1:].mean(1), sidx, B), _seg_mean_max(tok[:, 0], sidx, B), pres], 1)), None)\n\nclass Readout(nn.Module):\n\n    def __init__(self, pool, d, n_labels=12, pe=64):\n        super().__init__()\n        self.pool_kind, self.k = (pool, n_labels)\n        self.pres_emb = nn.Embedding(N_SLOT_TYPES + 1, pe, padding_idx=0)\n        if pool in ('xres', 'clsadd', 'xcodex'):\n            self.pool = {'xres': TokenResidualPool, 'clsadd': ClsAddPool, 'xcodex': CodexResidualPool}[pool](d, n_labels, pe=pe)\n        elif pool in ('attn', 'xattn'):\n            if pool == 'xattn':\n                self.pool = TokenXAttnPool(d, n_labels)\n                wd = 3 * d + pe\n            else:\n                self.pool = LabelAttentionPool(d, n_labels)\n                wd = d + pe\n            self.norm = nn.LayerNorm(wd)\n            self.w = nn.Parameter(torch.randn(n_labels, wd) * (1.0 / wd ** 0.5))\n            self.b = nn.Parameter(torch.zeros(n_labels))\n        else:\n            self.pool = MeanMaxPool()\n            self.net = nn.Sequential(nn.LayerNorm(2 * d + pe), nn.Dropout(0.2), nn.Linear(2 * d + pe, n_labels))\n        self.drop = nn.Dropout(0.2)\n\n    def forward(self, f, slot, sidx, B, return_attn=False):\n        pe = self.pres_emb(slot)\n        pres = torch.zeros(B, pe.shape[1], device=f.device, dtype=f.dtype).index_add_(0, sidx, pe)\n        if self.pool_kind in ('xres', 'clsadd', 'xcodex'):\n            return self.pool(f, slot, sidx, B, pres)[0]\n        pooled, attn = self.pool(f, sidx, B, slot=slot, return_attn=return_attn)\n        if self.pool_kind in ('attn', 'xattn'):\n            x = torch.cat([pooled, pres.unsqueeze(1).expand(-1, self.k, -1)], -1)\n            x = self.drop(self.norm(x))\n            return (x * self.w).sum(-1) + self.b\n        return self.net(torch.cat([pooled, pres], 1))\n\nclass Net(nn.Module):\n\n    def __init__(self, enc, cond, n_meta=0, pool='mean_max', stem='native', n_slice=16):\n        super().__init__()\n        self.enc, self.cond = (enc, cond)\n        self.compress = DepthCompress(n_slice, 3) if stem == 'compress' else None\n        self.mixer = SlotDepthMixer(n_slice) if stem == 'mixer' else None\n        self.tokens = pool in ('xattn', 'xres', 'clsadd', 'xcodex')\n        D = enc.num_features\n        self.meta_mlp = nn.Sequential(nn.LayerNorm(n_meta), nn.Linear(n_meta, 128), nn.GELU(), nn.Linear(128, D)) if n_meta > 0 else None\n        self.readout = Readout(pool, D)\n        if cond == 'post':\n            self.slot_emb = nn.Embedding(N_SLOT_TYPES + 1, D, padding_idx=MASK_IDX)\n\n    def forward(self, im, slot, smeta, sidx, B, vm=None):\n        if self.mixer is not None:\n            im = self.mixer(im, slot, vm)\n        if self.compress is not None:\n            im = self.compress(im)\n        f = self.enc.forward_features(im, slot) if self.cond == 'token' else self.enc.forward_features(im)\n        if self.tokens:\n            inner = getattr(self.enc, 'vit', self.enc)\n            orig = getattr(self.enc, '_orig_prefix', getattr(inner, 'num_prefix_tokens', 1))\n            f = torch.cat([f[:, :1], f[:, orig:]], 1)\n        else:\n            f = self.enc.forward_head(f, pre_logits=True)\n            if f.dim() > 2:\n                f = f.flatten(1)\n        ex = (lambda v: v.unsqueeze(1)) if self.tokens else lambda v: v\n        if self.cond == 'post':\n            f = f + ex(self.slot_emb(slot))\n        if self.meta_mlp is not None and smeta.shape[1] > 0:\n            mt = self.meta_mlp(smeta)\n            f = torch.cat([f, mt.unsqueeze(1)], 1) if self.tokens else f + mt\n        return self.readout(f, slot, sidx, B)\nmodels = []\nfor ckpt_path in sorted(CKPT.glob('*_f*.pt')):\n    z = torch.load(ckpt_path, map_location='cpu', weights_only=False)\n    cfg = z['cfg']\n    _stem = cfg.get('stem', 'native')\n    _in = 3 if _stem == 'compress' else cfg.get('n_slice', 16)\n    enc = timm.create_model(cfg['backbone'], pretrained=False, num_classes=0, in_chans=_in, **{'img_size': cfg['img']} if 'vit_' in cfg['backbone'] else {})\n    if cfg['cond'] == 'token':\n        enc = ViTSlotToken(enc, N_SLOT_TYPES)\n    m = Net(enc, cfg['cond'], cfg.get('n_meta', 0), cfg['pool'], stem=_stem, n_slice=cfg.get('n_slice', 16))\n    missing, unexpected = m.load_state_dict(z['state_dict'], strict=False)\n    assert not [k for k in missing if not k.startswith('enc.')], f'missing {missing[:5]}'\n    assert not unexpected, f'unexpected {unexpected[:5]}'\n    models.append(m.eval())\n    print(f\"loaded {ckpt_path.name}  fold {z['fold']}  {cfg['backbone']} pool={cfg['pool']} meta={cfg['meta']}\")\nCFG = cfg\nassert CFG.get('n_meta', 0) == 0, f\"checkpoint expects {CFG['n_meta']} metadata features -- build slot_meta for the TEST studies and pass it to predict() before submitting\"\nprint(f\"\\n{len(models)} fold models ready | input norm: {CFG.get('norm', 'none')}\")","execution_count":null,"outputs":[],"id":"reproduction-022"},{"cell_type":"code","metadata":{},"source":"AMP_PREF = 'bf16'\n\ndef amp_for(dev):\n    if not str(dev).startswith('cuda'):\n        return (torch.float32, False)\n    cc = torch.cuda.get_device_capability(dev)\n    if AMP_PREF == 'bf16':\n        return (torch.bfloat16, True)\n    if AMP_PREF == 'fp16':\n        return (torch.float16, True)\n    if AMP_PREF == 'fp32':\n        return (torch.float32, False)\n    return (torch.bfloat16 if cc >= (8, 0) else torch.float16, True)\nAMP_DT, AMP_ON = amp_for(DEV)\nWORKERS = max(1, min(4, os.cpu_count() or 4))\nCHUNK = 48\nMICRO = 8\nmodels = [m.to(DEV).eval() for m in models]\nprint(f\"device {DEV} | amp {str(AMP_DT).split('.')[-1]} (on={AMP_ON}) | workers {WORKERS} | chunk {CHUNK} | micro {MICRO}\")\n\ndef _norm_(im):\n    k = CFG.get('norm', 'none')\n    if k == 'zscore':\n        m = (im > 0).float()\n        n = m.sum(dim=(1, 2, 3), keepdim=True).clamp(min=1.0)\n        mu = (im * m).sum(dim=(1, 2, 3), keepdim=True) / n\n        var = (((im - mu) * m) ** 2).sum(dim=(1, 2, 3), keepdim=True) / n\n        return (im - mu) / (var.sqrt() + 1e-06) * m\n    if k == 'imagenet':\n        m = (im > 0).float()\n        return (im - 0.485) / 0.229 * m\n    return im\n\n@torch.no_grad()\ndef _micro(images, masks):\n    dev = DEV\n    ims, slots, sidx, vms = ([], [], [], [])\n    for b in range(len(masks)):\n        present = np.nonzero(masks[b] > 0)[0]\n        if len(present) == 0:\n            continue\n        blk = images[b][present]\n        ims.append(torch.from_numpy(blk))\n        vms.append(torch.from_numpy(blk.reshape(blk.shape[0], blk.shape[1], -1).max(2) > 0))\n        slots.append(torch.from_numpy(present + 1).long())\n        sidx.append(torch.full((len(present),), b, dtype=torch.long))\n    out = np.full((len(models), len(masks), len(LABELS)), np.nan, np.float32)\n    if not ims:\n        return out\n    im = _norm_(torch.cat(ims).to(dev, non_blocking=True).float().div_(255.0))\n    sl = torch.cat(slots).to(dev)\n    si = torch.cat(sidx).to(dev)\n    vm = torch.cat(vms).to(dev)\n    sm = torch.zeros(len(sl), CFG.get('n_meta', 0), device=dev)\n    per = torch.zeros(len(models), len(masks), len(LABELS), device=dev, dtype=torch.float32)\n    with torch.autocast('cuda' if str(dev).startswith('cuda') else 'cpu', dtype=AMP_DT, enabled=AMP_ON):\n        for fold_index, model in enumerate(models):\n            per[fold_index] = torch.sigmoid(\n                model(im, sl, sm, si, len(masks), vm=vm).float()\n            )\n    got = per.cpu().numpy()\n    keep = np.array([(masks[b] > 0).any() for b in range(len(masks))])\n    out[:, keep] = got[:, keep]\n    return out\n\ndef predict(images, masks):\n    out = np.full((len(models), len(masks), len(LABELS)), np.nan, np.float32)\n    for a in range(0, len(masks), MICRO):\n        b = min(a + MICRO, len(masks))\n        out[:, a:b] = _micro(images[a:b], masks[a:b])\n    return out\n\n# Macro ROC-AUC depends on ordering, so combine fold orderings rather\n# than allowing a fold's probability scale to dominate the mean.\npreds = np.full((len(models), len(studies), len(LABELS)), np.nan, np.float32)\nt0, done = (time.time(), 0)\nwith ProcessPoolExecutor(max_workers=WORKERS) as ex:\n    for c0 in range(0, len(studies), CHUNK):\n        block = studies[c0:c0 + CHUNK]\n        imgs = np.zeros((len(block), N_SLOT, N_SLICE, SIZE, SIZE), np.uint8)\n        msks = np.zeros((len(block), N_SLOT), np.uint8)\n        futs = [ex.submit(build_study, (i, s, by.get(s, []))) for i, s in enumerate(block)]\n        for f in as_completed(futs):\n            try:\n                i, a, k = f.result()\n                imgs[i], msks[i] = (a, k)\n            except Exception as e:\n                print(f'  study failed: {type(e).__name__}: {e}')\n        preds[:, c0:c0 + len(block)] = predict(imgs, msks)\n        done += len(block)\n        el = time.time() - t0\n        print(f'  {done:,}/{len(studies):,}  {el / 60:.1f}m  eta {el / done * (len(studies) - done) / 60:.1f}m', flush=True)\n        del imgs, msks\n        gc.collect()\nprint(f'\\ninference done in {(time.time() - t0) / 60:.1f} min')\nA5_W = 0.45\nA5_LABELS = list(LABELS)\n_a5_ok = np.isfinite(preds).all(axis=(0, 2))\n_a5_rank_mean = np.zeros((len(studies), len(LABELS)), np.float64)\nfor fold_index in range(preds.shape[0]):\n    fold = preds[fold_index][_a5_ok]\n    ordinal = fold.argsort(0).argsort(0).astype(np.float64)\n    _a5_rank_mean[_a5_ok] += ordinal / max(len(fold) - 1, 1)\n_a5_rank_mean /= preds.shape[0]\n_a5_rank_mean[~_a5_ok] = np.nan\nA5_PREDS = dict(zip(\n    sub_df['StudyInstanceUID'].astype(str), _a5_rank_mean.astype(np.float32)\n))\nfor _a5k, _a5v in _A5_SAVED.items():\n    globals()[_a5k] = _a5v\ndel _A5_SAVED, _a5k, _a5v","execution_count":null,"outputs":[],"id":"reproduction-023"},{"cell_type":"code","metadata":{},"source":"_a5_sub = pd.read_csv('/kaggle/working/submission.csv',\n                      dtype={'StudyInstanceUID': str})\nassert _a5_sub.columns.tolist()[1:] == A5_LABELS, 'submission schema drift'\nif A5_W > 0:\n    _a5_ours = np.stack([A5_PREDS[_u]\n                         for _u in _a5_sub['StudyInstanceUID'].astype(str)])\n    _a5_base_rank = _a5_sub[A5_LABELS].rank(method='average', pct=True)\n    _a5_ours_rank = pd.DataFrame(_a5_ours, columns=A5_LABELS,\n                                 index=_a5_sub.index).rank(method='average', pct=True)\n    _a5_sub[A5_LABELS] = (1.0 - A5_W) * _a5_base_rank + A5_W * _a5_ours_rank\n    assert np.isfinite(_a5_sub[A5_LABELS].to_numpy()).all()\n    _a5_sub.to_csv('/kaggle/working/submission.csv', index=False)","execution_count":null,"outputs":[],"id":"reproduction-024"},{"cell_type":"code","metadata":{},"source":"_rep_snapshot('a5')\n","execution_count":null,"outputs":[],"id":"reproduction-025"},{"cell_type":"markdown","metadata":{},"source":"## 3. RadImageNet: reproduce the reference and E13 branch\n\nThis stage retains the exact source layouts, head identities, and fixed calibration coefficients. The alternative Rad output is diagnostic only. The reference branch is the one passed forward.\n","id":"reproduction-026"},{"cell_type":"code","metadata":{},"source":"\"\"\"Check every Rad matrix before ranking can conceal nonfinite values.\"\"\"\n_rep_rad_rank_checks = 0\n_rep_rad_guard_installed = False\n_rep_rad_previous_profile = _rep_sys.getprofile()\n\ndef _rep_on_rad_main(frame, event, arg):\n    global _rep_rad_guard_installed\n    if event != 'call' or frame.f_code.co_name != '_rad_main' or '_rad_rank_columns' not in frame.f_globals:\n        if _rep_rad_previous_profile is not None:\n            _rep_rad_previous_profile(frame, event, arg)\n        return\n    namespace = frame.f_globals\n    original_rank = namespace['_rad_rank_columns']\n\n    def checked_rad_rank(values):\n        global _rep_rad_rank_checks\n        import numpy as _np\n        raw = _np.asarray(values)\n        assert raw.ndim == 2 and raw.shape[1] == 12, 'Rad raw shape'\n        assert _np.isfinite(raw).all(), 'Rad raw nonfinite before ranking'\n        _rep_rad_rank_checks += 1\n        return original_rank(values)\n\n    namespace['_rad_rank_columns'] = checked_rad_rank\n    _rep_rad_guard_installed = True\n    _rep_sys.setprofile(_rep_rad_previous_profile)\n\n_rep_sys.setprofile(_rep_on_rad_main)\n","execution_count":null,"outputs":[],"id":"reproduction-027"},{"cell_type":"code","metadata":{},"execution_count":null,"outputs":[],"source":"%%writefile /kaggle/working/rad_calibrator.json\n{\n \"coef\": [\n  [\n   0.054988985480198795,\n   0.0,\n   -0.0,\n   0.0,\n   -0.006905034746797682,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   -0.007598718307712681,\n   0.004775188808852552,\n   -0.0,\n   0.0,\n   0.0,\n   -0.004984520904425084,\n   0.0,\n   -0.006541710133337969,\n   -0.0,\n   -0.003752718730904078,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0028342800789145934,\n   -0.0,\n   0.012842247615159527,\n   0.0,\n   -0.0,\n   0.004720465029524873,\n   -0.0,\n   -0.0,\n   -0.003820481837620495,\n   0.021610275070444572,\n   -0.0,\n   -0.009703704271490837,\n   0.0,\n   -0.0007892533771826419,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.002190521845853738,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0018799414367212483,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.008708677037306389,\n   0.0,\n   0.0,\n   -0.0006364643665921704,\n   -0.0,\n   0.0,\n   -0.0007433491597744793,\n   -0.0,\n   -0.0,\n   -0.0022552694692117214,\n   0.08996657756223722,\n   0.0,\n   -0.0,\n   0.0027210612157672017,\n   -0.0,\n   0.0,\n   -0.009935986032994056,\n   0.0014937801726411662,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0041838897324036756,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0\n  ],\n  [\n   -0.0,\n   0.05407410902522933,\n   -0.0,\n   -0.0,\n   -0.00955325444490328,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   -0.002741621763298323,\n   0.0,\n   0.0030651354435245284,\n   0.0,\n   0.005261907711238927,\n   -0.0,\n   -0.0,\n   -2.246328164835325e-06,\n   -0.0,\n   -0.007235628651266947,\n   0.0,\n   -0.0,\n   -0.0,\n   0.002244318533778753,\n   0.0009848822815680745,\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0017347234462540013,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.001643286147786208,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.032048814954222495,\n   -0.0,\n   -0.0,\n   -0.002201853534088179,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0\n  ],\n  [\n   -0.0,\n   -0.0,\n   0.0591118991425821,\n   -0.006355044471722653,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.007699542581442222,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.048274735575991476,\n   -0.0,\n   -0.0,\n   -0.008539297840185648,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.004472020944327244,\n   0.0,\n   -0.0,\n   -0.00347437691921257,\n   0.05861944258061976,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.006976785974184935,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.004010865327087766,\n   0.0,\n   0.0,\n   0.0,\n   -0.0005908157372598818,\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.00500875228252724,\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.12402375784783327,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0015649029295778055,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0027029555164037418,\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0\n  ],\n  [\n   0.0,\n   -0.0,\n   -0.0,\n   0.020022960936525995,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0017738172621121892,\n   -0.0,\n   -0.0023183591962150705,\n   0.0,\n   -0.007515949427429685,\n   -0.010155250767449142,\n   0.0004984806412923339,\n   -0.0,\n   0.0,\n   -0.005354229345626853,\n   0.0,\n   0.0,\n   -0.0007398489398975964,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.004071983093585694,\n   -0.0,\n   0.01700346063545301,\n   -0.0,\n   0.0058490781650871355,\n   -0.001005834776031484,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0009063914656936253,\n   0.000214420591619211,\n   0.0005584969828176026,\n   -0.0,\n   -0.0020443851541547286,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.007754847683752665,\n   -0.0005033977078919417,\n   0.000708682810146001,\n   -0.0,\n   0.0,\n   -0.003502795873998033,\n   0.0,\n   0.0,\n   -0.0028597700818967357,\n   -0.0,\n   0.11847279874112572,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0015607762716186155,\n   -0.0,\n   -0.005456887523612534,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0\n  ],\n  [\n   0.0,\n   -0.0,\n   0.011905299490447545,\n   -0.0,\n   0.051327525539440685,\n   0.0,\n   0.006490303077739882,\n   0.0,\n   0.009189655557113569,\n   -0.0032555342419950226,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.015598360106187889,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0007355085953963836,\n   0.0,\n   0.0,\n   0.0,\n   -0.005692959829707796,\n   0.008383567458721805,\n   -0.0004342984045223054,\n   0.040683776777621715,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.003946854023482639,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.003675523046569264,\n   -0.0,\n   -0.0,\n   -0.006683593070962812,\n   0.001598200174796656,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0009560338374132588,\n   0.0,\n   0.0014865501499527398,\n   0.0,\n   0.0,\n   -0.006465356514571239,\n   0.0,\n   -0.0,\n   0.08801358135090863,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0032578169698303334,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0022856634143569687,\n   -0.0008211041163776929,\n   0.0,\n   0.0,\n   0.001979434428479871,\n   -0.0006904492760499002\n  ],\n  [\n   0.0,\n   -0.0010201300673726335,\n   -0.0,\n   0.003330913654546868,\n   0.0,\n   0.03804589769802522,\n   0.0036582135089306815,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0074698905145500535,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.004181180657534594,\n   -0.0,\n   0.0,\n   -0.0014097935467749104,\n   0.0,\n   -0.008364014702270928,\n   -0.0,\n   0.0,\n   0.0,\n   0.031234909820757367,\n   0.0,\n   0.0,\n   0.001761560363567976,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.00010302276929695168,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0036722429881927794,\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.028070173174009682,\n   -0.0,\n   0.03932353957184419,\n   0.0,\n   0.052709270489857116,\n   4.30670771118039e-05,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.005897795403501839,\n   -0.0,\n   -0.0017091642652794067,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0011445129464464573,\n   0.0,\n   -3.599923112263568e-05\n  ],\n  [\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.00398886886838279,\n   0.024531745408578224,\n   0.0,\n   0.003472383623711957,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0024216642234806546,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.04441771278056577,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.04208386438405934,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -8.871585477522171e-05,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.01794873111002335,\n   -0.010965712486455531,\n   0.0,\n   -0.0,\n   -0.0003112348219671695,\n   -0.0,\n   0.00033734018896491223,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.002767382390020047,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.006553844259659735,\n   -0.0,\n   0.002859144951920367,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.1080204096279449,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.008286727880437668,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0016705366518583718,\n   0.0,\n   0.003966228970353484,\n   0.0,\n   -0.0,\n   -0.0036942492702868837,\n   0.0,\n   2.0158865558700288e-05,\n   -0.0,\n   -0.003290955791230868,\n   0.0,\n   2.931550215656576e-05\n  ],\n  [\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.15721758161271948,\n   0.0,\n   -0.0009277854332745394,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0035078908538144857,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0029922774103666346,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0015069965777220378,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0034575286435870912,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.004879501088574802,\n   0.0,\n   -0.0,\n   0.0,\n   0.00107584083385987,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.07386105032184348,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   -0.0006098366519528874,\n   0.000710602647614561,\n   0.0,\n   0.00029648470600281623,\n   0.0,\n   0.0006669555200166979,\n   0.0,\n   0.0001247447196589612,\n   0.0\n  ],\n  [\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.010971916263113718,\n   0.0,\n   0.0005950879974343855,\n   0.020934939880106,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.00716089007036752,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.012582689029033865,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.05233942478784284,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.003853509415892567,\n   -0.009802578580289266,\n   0.0,\n   0.0,\n   0.013526997046656204,\n   0.0064884417795260645,\n   -0.0,\n   0.0002747792719802145,\n   0.00794104167516065,\n   0.0032425867028947433,\n   -0.0\n  ],\n  [\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.07766030485486214,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0016984387848328866,\n   -0.02318816122584484,\n   -0.0,\n   0.0008232018620112374,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0030093149725252876,\n   0.0,\n   0.015496588831608236,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.003985489123804257,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.08934738817249994,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0031358753239660443,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.002757717666564488,\n   0.0,\n   0.0,\n   0.0,\n   -0.001572020817838309\n  ],\n  [\n   0.0,\n   0.0,\n   -0.007645376480798367,\n   -0.0,\n   -0.013152846307009906,\n   -0.0,\n   -0.013405770463176186,\n   0.0,\n   -0.0,\n   -0.0126185896490272,\n   0.08456110280686102,\n   0.0010498490952287943,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.026761069960320228,\n   0.004881051675806471,\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0029949120807847875,\n   0.0,\n   0.0,\n   -0.0013947178093899313,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   -0.00942848401064722,\n   -0.0,\n   -0.0,\n   -0.00022519045066539146,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0457033966423892,\n   0.027947704687921546,\n   0.0,\n   -0.0017566658004398543,\n   -0.011670049510825595,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0007825317116991779,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0\n  ],\n  [\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.001502844459484492,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0020872198960683284,\n   0.009847417955540042,\n   0.004860013798480458,\n   0.0,\n   -0.0,\n   -0.0012632551037073973,\n   0.0,\n   -0.007111539097901386,\n   -0.0,\n   -0.00195199372867498,\n   0.0,\n   -0.0,\n   -0.0,\n   0.003484155321535836,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0005385599274495165,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.002005337879973018,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   0.04784159508857366,\n   0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   -0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0,\n   0.0,\n   0.0,\n   -0.0,\n   0.0\n  ]\n ],\n \"gate\": [\n  \"ACL\",\n  \"Medial OA\",\n  \"Lateral OA\",\n  \"PF OA\",\n  \"Effusion\",\n  \"Baker's\",\n  \"Contusion\"\n ],\n \"groups\": [\n  [\n   \"ACL\",\n   \"MCL\",\n   \"Contusion\",\n   \"Fracture\"\n  ],\n  [\n   \"Medial Meniscus\",\n   \"Lateral Meniscus\"\n  ],\n  [\n   \"Medial OA\",\n   \"Lateral OA\",\n   \"PF OA\"\n  ],\n  [\n   \"Effusion\",\n   \"Synovitis\",\n   \"Baker's\"\n  ]\n ],\n \"intercept\": [\n  0.27594726240322665,\n  0.19664147502522833,\n  0.46103077516667196,\n  0.26000148037766385,\n  0.348880476077348,\n  0.267855131253255,\n  0.4114411126395088,\n  0.4677221586179633,\n  0.2775079245121116,\n  0.3059966406870837,\n  0.26281611268502275,\n  0.1933670474221893\n ],\n \"mean\": [\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  -8.169035821569329e-19,\n  4.0845179107846643e-19,\n  8.169035821569329e-19,\n  0.0,\n  -2.0422589553923322e-19,\n  -7.147906343873162e-19,\n  1.0211294776961661e-19,\n  2.0422589553923322e-19,\n  -6.1267768661769965e-19,\n  -8.169035821569329e-19,\n  3.0633884330884983e-19,\n  -1.2253553732353993e-18,\n  -4.0845179107846643e-19,\n  3.0633884330884983e-19,\n  6.1267768661769965e-19,\n  -1.021129477696166e-18,\n  -2.0422589553923322e-19,\n  -9.190165299265494e-19,\n  2.0422589553923322e-19,\n  2.0422589553923322e-19,\n  -2.0422589553923322e-19,\n  -1.2253553732353993e-18,\n  -1.0211294776961661e-19,\n  0.0,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  0.5001149689583813,\n  5.526557829386066,\n  2.2368360542653485,\n  1.9521729133134054,\n  1.337548861807312,\n  3.1772821338238675,\n  1.0584042308576684,\n  1.0482869625201197,\n  1.0705909404460796,\n  3.1772821338238675,\n  1.0584042308576684,\n  1.0482869625201197,\n  1.0705909404460796\n ],\n \"protocol_columns\": [\n  \"n_series\",\n  \"n_Sag\",\n  \"n_Cor\",\n  \"n_Axi\",\n  \"Fat\",\n  \"Fat_Sag\",\n  \"Fat_Cor\",\n  \"Fat_Axi\",\n  \"Flu\",\n  \"Flu_Sag\",\n  \"Flu_Cor\",\n  \"Flu_Axi\"\n ],\n \"scale\": [\n  0.28867512696347697,\n  0.28867512696347697,\n  0.28867512696347697,\n  0.28867512695294856,\n  0.28867512696347697,\n  0.28867512696347697,\n  0.28867512696347697,\n  0.28867512696347697,\n  0.28867512696347697,\n  0.28867512696347697,\n  0.28867512696347697,\n  0.28867512695294856,\n  0.28867512338382056,\n  0.28867512338382056,\n  0.28867512338382056,\n  0.28867512338382056,\n  0.28867512338382056,\n  0.28867512338382056,\n  0.288675123394349,\n  0.288675123394349,\n  0.288675123394349,\n  0.288675123394349,\n  0.28867512336276374,\n  0.28867512338382056,\n  0.28867512696347697,\n  0.28867512695294856,\n  0.28867512695294856,\n  0.28867512696347697,\n  0.28867512696347697,\n  0.28867512696347697,\n  0.28867512696347697,\n  0.28867512695294856,\n  0.2886751269529485,\n  0.28867512695294856,\n  0.28867512696347697,\n  0.28867512694242015,\n  0.23019299127577333,\n  0.20945811170519504,\n  0.18896664844677993,\n  0.22034647621868486,\n  0.15727493499021805,\n  0.19246986757904713,\n  0.18959582474964737,\n  0.1500338770229888,\n  0.17335524281933992,\n  0.21499556517074797,\n  0.18526691377924037,\n  0.24722042176621192,\n  0.23300663477492883,\n  0.2351154922532429,\n  0.19851192071060803,\n  0.22614032204035772,\n  0.16824789069383667,\n  0.19316845808174274,\n  0.19078421415473018,\n  0.17455770295025105,\n  0.1939184152151192,\n  0.2380093052256761,\n  0.2054846773727961,\n  0.25238186441777205,\n  0.2560260929356755,\n  0.2563420389842119,\n  0.2669423487470595,\n  0.2578640024055912,\n  0.2718501229846065,\n  0.26593176523552875,\n  0.26652763985208816,\n  0.2716787563858938,\n  0.26822049981618185,\n  0.2551672579287264,\n  0.26457231958493127,\n  0.24970143393957778,\n  0.21746552440992822,\n  0.24075616342318434,\n  0.2586368483721014,\n  0.23201516104802145,\n  1.392601296101108,\n  0.5943047831182455,\n  0.7471955639897663,\n  0.6579157483703758,\n  0.6336751078357071,\n  0.41835483650318195,\n  0.35622110896336706,\n  0.26841408540232553,\n  0.6336751078357071,\n  0.41835483650318195,\n  0.35622110896336706,\n  0.26841408540232553\n ]\n}\n"},{"cell_type":"code","metadata":{},"execution_count":null,"outputs":[],"source":"from __future__ import annotations\n# ---------------------------------------------------------------------------\n# Rad branch calibrator - INHERITED, NOT FITTED BY THIS NOTEBOOK.\n# Carried in the parent notebook as a zlib+base64 string; written out above as\n# plain JSON instead. Same numbers, readable. It standardises 88 protocol\n# features (mean/scale) and applies a 12x88 linear map + intercept per finding.\n# Origin: Mattia Angeli, \"Bend the Knee to the Dinosaurs\" (Apache-2.0).\n# ---------------------------------------------------------------------------\n# Surgical reproduction of V48's deployed prediction branch.\n#\n# The pinned reference Rad family is fused with correct-contract E13, then\n# the same E13 heads run on the E11 layout at 0.15. No twin/legacy wrapper\n# follows it, matching the branch that produced V48's visible submission.\n\nimport contextlib as _rad_contextlib\nimport gc as _rad_gc\nimport hashlib as _rad_hashlib\nimport json as _rad_json\nimport os as _rad_os\nimport re as _rad_re\nimport time as _rad_time\nfrom concurrent.futures import ThreadPoolExecutor as _RadThreadPool\nfrom pathlib import Path as _RadPath\n\nimport numpy as _rad_np\nimport pandas as _rad_pd\nimport pydicom as _rad_pydicom\nimport torch as _rad_torch\nimport torch.nn as _rad_nn\nimport torch.nn.functional as _rad_F\nfrom torchvision.models import resnet50 as _rad_resnet50\n\n_RAD_LABELS = [\n    'ACL', 'MCL', 'Medial Meniscus', 'Lateral Meniscus', 'Medial OA',\n    'Lateral OA', 'PF OA', 'Effusion', 'Synovitis', \"Baker's\",\n    'Contusion', 'Fracture',\n]\n_RAD_ALPHA = 0.50\n_RAD_EXCLUDE = (\"Baker's\", 'Fracture')\n_RAD_HEADS_SHA256 = '54f657826b3458a7ba3d462e198ba380732f2b136246182312704929874a9a2c'\n_RAD_REFERENCE_HEADS_SHA256 = '0f465649799ecfbccaac1767844639e7ced44e1bc9babde6e4bac7c5d9b89eaa'\n_RAD_ENCODER_SHA256 = '08629f7e7bd3e29b8ee9522ca3f65ce4d010a7ddf74f0ea3c7e3f3d0bbab0734'\n_RAD_E13_HEADS_SHA256 = 'ad9f19af73bfdf4e49263c0e45060dc3cb239e1195039b26dc8c0a3a6bcd1a8a'\n_RAD_E13_MEMBER_WEIGHT = 0.50\n_RAD_V48_SECOND_ALPHA = 0.15\n_RAD_TWIN_ALT_WEIGHT = 0.500001\n_RAD_TOKEN_DIM, _RAD_HEAD_DIM = 2048, 512\n\n_RAD_E11_SLOTS = [\n    ('SAG_NOFS', 'Sagittal', None, False),\n    ('COR_NOFS', 'Coronal', None, False),\n    ('AX_NOFS', 'Axial', None, False),\n    ('SAG_FS', 'Sagittal', None, True),\n]\n_RAD_E11_CROP_MM = 130.0\n_RAD_E11_CACHE_SLICES = 8\n_RAD_E11_IMG = 224\n\n_RAD_E13_SLOTS = [\n    ('SAG_FS', 'Sagittal', None, True),\n    ('COR_FS', 'Coronal', None, True),\n    ('AX_FS', 'Axial', None, True),\n    ('SAG_NOFS', 'Sagittal', None, False),\n]\n_RAD_E13_CROP_MM = 130.0\n_RAD_E13_CACHE_SLICES = 8\n_RAD_E13_IMG = 224\n\n# Our independently trained five-fold family.  Its preprocessing and estimator\n# are preserved from V35: native DICOM geometry/fat-sat handling and a mean of\n# per-fold percentile ranks (rather than v15's rank of the probability mean).\n_OUR_N_SLOT, _OUR_N_SLICE, _OUR_IMG = 3, 8, 224\n\n# Exact V40/E10 test representation: three fat-suppressed planes, eight\n# acquired slices per plane, full frame, legacy ordering/laterality/fill.\nSLOTS = [\n    ('SAG_FS', 'Sagittal', None, True),\n    ('COR_FS', 'Coronal', None, True),\n    ('AX_FS', 'Axial', None, True),\n]\nN_SLOT = len(SLOTS)\nCACHE_SLICES = 8\nIMG = CACHE_IMG = 224\nCROP_MM = 10_000.0\nSLICE_BAND = (0.2, 0.8)\nRULES = dict(RULES_LEGACY)\nTIME_BUDGET = 8.0 * 3600\n\n\ndef _rad_log(message):\n    print(f'[Rad-dual5] {message}', flush=True)\n\n\ndef _rad_sha256(path, chunk=8 << 20):\n    digest = _rad_hashlib.sha256()\n    with open(path, 'rb') as handle:\n        for block in iter(lambda: handle.read(chunk), b''):\n            digest.update(block)\n    return digest.hexdigest()\n\n\ndef _rad_find_file(name, expected_sha=None, explicit_env=None):\n    if explicit_env and _rad_os.environ.get(explicit_env):\n        candidates = [_RadPath(_rad_os.environ[explicit_env])]\n    else:\n        candidates = []\n        base = _RadPath('/kaggle/input')\n        if base.is_dir():\n            for root, dirs, files in _rad_os.walk(base):\n                dirs[:] = [d for d in dirs if d not in ('train_series', 'test_series')]\n                if name in files:\n                    candidates.append(_RadPath(root) / name)\n    if not candidates:\n        raise FileNotFoundError(f'V36 missing input artifact {name}')\n    for path in candidates:\n        if expected_sha is None or _rad_sha256(path) == expected_sha:\n            return path\n    raise RuntimeError(f'V36 found {name}, but no copy has the required SHA-256')\n\n\nclass _RadEncoder(_rad_nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.backbone = _rad_nn.Sequential(\n            *list(_rad_resnet50(weights=None).children())[:-2]\n        )\n\n    def forward(self, image):\n        return self.backbone(image).mean(dim=(2, 3))\n\n\nclass _RadHead(_rad_nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.project = _rad_nn.Sequential(\n            _rad_nn.LayerNorm(_RAD_TOKEN_DIM),\n            _rad_nn.Linear(_RAD_TOKEN_DIM, _RAD_HEAD_DIM),\n            _rad_nn.GELU(),\n        )\n        self.plane = _rad_nn.Parameter(_rad_torch.randn(N_SLOT, _RAD_HEAD_DIM) * .01)\n        self.position = _rad_nn.Parameter(_rad_torch.randn(CACHE_SLICES, _RAD_HEAD_DIM) * .01)\n        self.query = _rad_nn.Parameter(_rad_torch.randn(len(_RAD_LABELS), _RAD_HEAD_DIM) * .02)\n        self.attn = _rad_nn.MultiheadAttention(\n            _RAD_HEAD_DIM, 8, dropout=.10, batch_first=True\n        )\n        self.fuse = _rad_nn.Sequential(\n            _rad_nn.LayerNorm(_RAD_HEAD_DIM * 4),\n            _rad_nn.Linear(_RAD_HEAD_DIM * 4, _RAD_HEAD_DIM),\n            _rad_nn.GELU(),\n            _rad_nn.Dropout(.15),\n        )\n        self.weight = _rad_nn.Parameter(\n            _rad_torch.randn(len(_RAD_LABELS), _RAD_HEAD_DIM) * .02\n        )\n        self.bias = _rad_nn.Parameter(_rad_torch.zeros(len(_RAD_LABELS)))\n\n    def forward(self, feature, mask):\n        token = self.project(feature.float())\n        token = token.view(len(token), N_SLOT, CACHE_SLICES, _RAD_HEAD_DIM)\n        token = token + self.plane[None, :, None] + self.position[None, None]\n        token = token.flatten(1, 2)\n        key_padding = mask <= 0\n        all_empty = key_padding.all(1)\n        if all_empty.any():\n            key_padding = key_padding.clone()\n            key_padding[all_empty, 0] = False\n        query = self.query.unsqueeze(0).expand(len(token), -1, -1)\n        attended = query + self.attn(\n            query, token, token, key_padding_mask=key_padding, need_weights=False\n        )[0]\n        denominator = mask.sum(1, keepdim=True).clamp_min(1).unsqueeze(-1)\n        mean = (token * mask.unsqueeze(-1)).sum(1, keepdim=True) / denominator\n        mean = mean.expand(-1, len(_RAD_LABELS), -1)\n        fused = self.fuse(_rad_torch.cat(\n            [attended, mean, _rad_torch.abs(attended - mean), attended * mean], dim=-1\n        ))\n        return (fused * self.weight.unsqueeze(0)).sum(-1) + self.bias\n\n\ndef _rad_load_public_heads(device, expected_sha):\n    heads_path = _rad_find_file('v52_radimagenet_heads.pt', expected_sha)\n    payload = _rad_torch.load(heads_path, map_location='cpu', weights_only=True)\n    expected = {\n        'version': 'v52-radimagenet-resnet50-official-1',\n        'targets': _RAD_LABELS,\n        'encoder_sha256': _RAD_ENCODER_SHA256,\n        'encoder_source_commit': '0ce16f7375db4236e646829d1eca61cdb4282133',\n        'img': 224,\n        'slices_per_plane': 8,\n        'feature': 'global_average_pool',\n    }\n    for key, value in expected.items():\n        if payload.get(key) != value:\n            raise RuntimeError(f'public-v15 head contract drift for {key}')\n    folds = payload.get('folds')\n    if not isinstance(folds, list) or len(folds) != 5:\n        raise RuntimeError('public-v15 bundle requires exactly five heads')\n    if sorted(int(record.get('fold', -1)) for record in folds) != list(range(5)):\n        raise RuntimeError('public-v15 fold identity drift')\n    heads = []\n    for record in folds:\n        head = _RadHead().to(device).eval()\n        head.load_state_dict(record['state_dict'], strict=True)\n        heads.append(head)\n    return heads, str(heads_path)\n\n\ndef _rad_load_e13_heads(device):\n    # V48 used an unqualified filename shared by E11 and E13. Resolve the\n    # intended E13 bundle by content and validate its complete pixel contract.\n    heads_path = _rad_find_file('v52_e11_heads.pt', _RAD_E13_HEADS_SHA256)\n    payload = _rad_torch.load(heads_path, map_location='cpu', weights_only=False)\n    expected = {\n        'version': 'e11-radimagenet-resnet50-diverse-1',\n        'targets': _RAD_LABELS,\n        'encoder_sha256': _RAD_ENCODER_SHA256,\n        'slots': [list(slot) for slot in _RAD_E13_SLOTS],\n        'crop_mm': _RAD_E13_CROP_MM,\n        'img': _RAD_E13_IMG,\n        'slices_per_plane': _RAD_E13_CACHE_SLICES,\n        'feature': 'global_average_pool',\n    }\n    for key, value in expected.items():\n        if payload.get(key) != value:\n            raise RuntimeError(f'E13 head contract drift for {key}')\n    folds = payload.get('folds')\n    if not isinstance(folds, list) or len(folds) != 5:\n        raise RuntimeError('E13 bundle requires exactly five heads')\n    if sorted(int(record.get('fold', -1)) for record in folds) != list(range(5)):\n        raise RuntimeError('E13 fold identity drift')\n    heads = []\n    for record in folds:\n        head = _RadHead().to(device).eval()\n        head.load_state_dict(record['state_dict'], strict=True)\n        heads.append(head)\n    return heads, str(heads_path)\n\n\ndef _rad_load_models(device):\n    encoder_path = _rad_find_file(\n        'ResNet50.pt', _RAD_ENCODER_SHA256, explicit_env='RSNA_RAD_WEIGHT_PATH'\n    )\n    encoder = _RadEncoder()\n    encoder.load_state_dict(\n        _rad_torch.load(encoder_path, map_location='cpu', weights_only=True), strict=True\n    )\n    if sum(parameter.numel() for parameter in encoder.parameters()) != 23_508_032:\n        raise RuntimeError('V36 RadImageNet encoder parameter-count drift')\n    encoder.eval().to(device)\n    for parameter in encoder.parameters():\n        parameter.requires_grad_(False)\n    if device.type == 'cuda' and _rad_torch.cuda.device_count() > 1:\n        encoder = _rad_nn.DataParallel(\n            encoder, device_ids=list(range(_rad_torch.cuda.device_count()))\n        )\n\n    alt_heads, alt_path = _rad_load_public_heads(device, _RAD_HEADS_SHA256)\n    reference_heads, reference_path = _rad_load_public_heads(\n        device, _RAD_REFERENCE_HEADS_SHA256\n    )\n    if alt_path == reference_path:\n        raise RuntimeError('twin E10 branches resolved to the same artifact')\n    return encoder, alt_heads, reference_heads, str(encoder_path), alt_path, reference_path\n\n\n@_rad_torch.inference_mode()\ndef _rad_encode(encoder, pixels, slot_mask, device):\n    n, slots, slices, height, width = pixels.shape\n    features = _rad_np.zeros(\n        (n, slots * slices, _RAD_TOKEN_DIM), _rad_np.float16\n    )\n    token_mask = _rad_np.repeat(slot_mask[:, :, None], slices, axis=2).reshape(n, -1)\n    valid = _rad_np.flatnonzero(token_mask.reshape(-1) > 0)\n    flat = pixels.reshape(-1, height, width)\n    batch = 192 if device.type == 'cuda' and _rad_torch.cuda.device_count() > 1 else (\n        96 if device.type == 'cuda' else 8\n    )\n    for start in range(0, len(valid), batch):\n        indices = valid[start:start + batch]\n        image = _rad_torch.from_numpy(flat[indices]).to(device).float().div_(127.5).sub_(1.0)\n        image = image.unsqueeze(1).expand(-1, 3, -1, -1).contiguous()\n        amp = (_rad_torch.autocast('cuda')\n               if device.type == 'cuda' else _rad_contextlib.nullcontext())\n        with amp:\n            feature = encoder(image)\n        values = feature.float().cpu().numpy()\n        if not _rad_np.isfinite(values).all():\n            raise RuntimeError('V36 non-finite RadImageNet feature')\n        features.reshape(-1, _RAD_TOKEN_DIM)[indices] = values.astype(_rad_np.float16)\n    return features, token_mask.astype(_rad_np.float32)\n\n\n@_rad_torch.inference_mode()\ndef _rad_predict_head(head, features, masks, device, batch=64):\n    predictions = []\n    for start in range(0, len(features), batch):\n        image = _rad_torch.from_numpy(features[start:start + batch]).to(device)\n        mask = _rad_torch.from_numpy(masks[start:start + batch]).to(device)\n        amp = (_rad_torch.autocast('cuda')\n               if device.type == 'cuda' else _rad_contextlib.nullcontext())\n        with amp:\n            predictions.append(_rad_torch.sigmoid(head(image, mask)).float().cpu())\n    return _rad_torch.cat(predictions).numpy()\n\n\ndef _rad_rank_columns(values):\n    return _rad_pd.DataFrame(\n        _rad_np.asarray(values, dtype=_rad_np.float64)\n    ).rank(method='average', pct=True).to_numpy(_rad_np.float64)\n\n\ndef _rad_validate(frame, expected_ids):\n    if frame.columns.tolist() != ['StudyInstanceUID', *_RAD_LABELS]:\n        raise RuntimeError('V36 submission schema drift')\n    ids = frame['StudyInstanceUID'].astype(str).tolist()\n    if ids != list(map(str, expected_ids)) or len(ids) != len(set(ids)):\n        raise RuntimeError('V36 submission study identity/order drift')\n    values = frame[_RAD_LABELS].to_numpy(_rad_np.float64)\n    if not _rad_np.isfinite(values).all() or values.min() < 0 or values.max() > 1:\n        raise RuntimeError('V36 invalid submission values')\n\n\ndef _rad_main():\n    started = _rad_time.time()\n    work = _RadPath(_rad_os.environ.get('RSNA_RAD_OUTPUT_DIR', '/kaggle/working'))\n    primary = work / 'submission.csv'\n    if not primary.is_file():\n        raise FileNotFoundError('V37 requires the completed DINO parent submission.csv')\n    test = _rad_pd.read_csv(ROOT / 'test.csv', dtype={'StudyInstanceUID': str})\n    expected_ids = test['StudyInstanceUID'].astype(str).tolist()\n    baseline = _rad_pd.read_csv(primary, dtype={'StudyInstanceUID': str})\n    _rad_validate(baseline, expected_ids)\n\n    device = _rad_torch.device('cuda:0' if _rad_torch.cuda.is_available() else 'cpu')\n    if device.type != 'cuda':\n        raise RuntimeError('V37 RadImageNet inference requires CUDA')\n    (encoder, public_heads, reference_heads, encoder_path,\n     public_heads_path, reference_heads_path) = _rad_load_models(device)\n\n    # Family 1: public v15/E10 legacy pixels.  Keep this path bit-for-bit as in\n    # V36, including rank(mean(fold probability)).\n    test_series = _rad_pd.read_csv(\n        ROOT / 'test_series.csv',\n        dtype={'StudyInstanceUID': str, 'SeriesInstanceUID': str},\n    )\n    plane = dict(zip(test_series.SeriesInstanceUID, test_series.Anatomical_Plane))\n    headers = annotate(walk('test_series'))\n    studies, pixels, slot_mask = build_cache(\n        pick_slots(headers, plane), plane, lat_of(headers, 'test-e10 '), 'test-e10'\n    )\n    by_uid = {str(uid): index for index, uid in enumerate(studies)}\n    missing = [uid for uid in expected_ids if uid not in by_uid]\n    if missing:\n        raise RuntimeError(f'{len(missing)} test studies absent from public-v15 cache')\n    order = _rad_np.asarray([by_uid[uid] for uid in expected_ids], dtype=_rad_np.int64)\n    pixels, slot_mask = pixels[order], slot_mask[order]\n    token_count = int(\n        _rad_np.repeat(slot_mask[:, :, None], CACHE_SLICES, axis=2).sum()\n    )\n    if token_count < int(0.85 * len(test) * N_SLOT * CACHE_SLICES):\n        raise RuntimeError(f'insufficient acquired public-v15 test slices: {token_count}')\n\n    features, token_mask = _rad_encode(encoder, pixels, slot_mask, device)\n    del pixels, slot_mask, headers\n    _rad_gc.collect()\n    public_fold_predictions = [\n        _rad_predict_head(head, features, token_mask, device)\n        for head in public_heads\n    ]\n    reference_fold_predictions = [\n        _rad_predict_head(head, features, token_mask, device)\n        for head in reference_heads\n    ]\n    if len(public_fold_predictions) != 5 or len(reference_fold_predictions) != 5:\n        raise RuntimeError('twin E10 inference did not use all ten heads')\n\n    # Preserve each public recipe's rank(mean(fold probability)) estimator.\n    public_probability = _rad_np.mean(_rad_np.stack(public_fold_predictions), axis=0)\n    reference_probability = _rad_np.mean(\n        _rad_np.stack(reference_fold_predictions), axis=0\n    )\n    public_rank = _rad_rank_columns(public_probability)\n    reference_rank = _rad_rank_columns(reference_probability)\n    del (public_heads, reference_heads, public_fold_predictions,\n         reference_fold_predictions, public_probability, reference_probability,\n         features, token_mask)\n    _rad_gc.collect()\n    _rad_torch.cuda.empty_cache()\n    _rad_log(\n        f'twin public-v15 families complete ({public_heads_path}; {reference_heads_path})'\n    )\n\n    # One new member from V48: three fat-sensitive planes plus a sagittal\n    # structural anchor, all at a 130 mm crop. Average ranks inside the Rad block\n    # and re-rank the result exactly as V48 does before the unchanged E10 vote.\n    globals().update(\n        SLOTS=list(_RAD_E13_SLOTS),\n        N_SLOT=len(_RAD_E13_SLOTS),\n        CACHE_SLICES=int(_RAD_E13_CACHE_SLICES),\n        IMG=int(_RAD_E13_IMG),\n        CACHE_IMG=int(_RAD_E13_IMG),\n        CROP_MM=float(_RAD_E13_CROP_MM),\n        RULES=dict(RULES_LEGACY),\n    )\n    e13_heads, e13_path = _rad_load_e13_heads(device)\n    headers = annotate(walk('test_series'))\n    studies, pixels, slot_mask = build_cache(\n        pick_slots(headers, plane), plane, lat_of(headers, 'test-e13 '), 'test-e13'\n    )\n    by_uid = {str(uid): index for index, uid in enumerate(studies)}\n    missing = [uid for uid in expected_ids if uid not in by_uid]\n    if missing:\n        raise RuntimeError(f'{len(missing)} test studies absent from E13 cache')\n    order = _rad_np.asarray([by_uid[uid] for uid in expected_ids], dtype=_rad_np.int64)\n    pixels, slot_mask = pixels[order], slot_mask[order]\n    e13_token_count = int(\n        _rad_np.repeat(slot_mask[:, :, None], CACHE_SLICES, axis=2).sum()\n    )\n    if e13_token_count < int(0.85 * len(test) * N_SLOT * CACHE_SLICES):\n        raise RuntimeError(f'insufficient acquired E13 test slices: {e13_token_count}')\n    e13_features, e13_token_mask = _rad_encode(\n        encoder, pixels, slot_mask, device\n    )\n    del pixels, slot_mask, headers\n    _rad_gc.collect()\n    e13_predictions = [\n        _rad_predict_head(head, e13_features, e13_token_mask, device)\n        for head in e13_heads\n    ]\n    if len(e13_predictions) != 5:\n        raise RuntimeError('E13 inference did not use all five heads')\n    e13_probability = _rad_np.mean(_rad_np.stack(e13_predictions), axis=0)\n    if (\n        e13_probability.shape != (len(test), len(_RAD_LABELS))\n        or not _rad_np.isfinite(e13_probability).all()\n    ):\n        raise RuntimeError(f'invalid E13 prediction shape/value: {e13_probability.shape}')\n    e13_rank = _rad_rank_columns(e13_probability)\n    public_rank = _rad_rank_columns(\n        (1.0 - _RAD_E13_MEMBER_WEIGHT) * public_rank\n        + _RAD_E13_MEMBER_WEIGHT * e13_rank\n    )\n    reference_rank = _rad_rank_columns(\n        (1.0 - _RAD_E13_MEMBER_WEIGHT) * reference_rank\n        + _RAD_E13_MEMBER_WEIGHT * e13_rank\n    )\n    # V48 resolves this same bundle again after switching pixel layouts.\n    del (e13_predictions, e13_probability, e13_rank,\n         e13_features, e13_token_mask)\n    _rad_gc.collect()\n    _rad_torch.cuda.empty_cache()\n    _rad_log(\n        f'E13 FS-crop member complete at Rad-block weight '\n        f'{_RAD_E13_MEMBER_WEIGHT:.2f} ({e13_path})'\n    )\n\n    # E10 keeps its audited 0.50 parent/Rad vote. The two excluded findings\n    # remain the raw parent values, matching the audited deployment.\n    baseline_rank = _rad_rank_columns(baseline[_RAD_LABELS].to_numpy())\n\n    def _rad_e10_branch(head_rank):\n        branch = baseline.copy()\n        for index, target in enumerate(_RAD_LABELS):\n            if target not in _RAD_EXCLUDE:\n                branch[target] = (\n                    (1.0 - _RAD_ALPHA) * baseline_rank[:, index]\n                    + _RAD_ALPHA * head_rank[:, index]\n                )\n        return branch\n\n    candidate_alt = _rad_e10_branch(public_rank)\n    candidate_reference = _rad_e10_branch(reference_rank)\n    for branch in (candidate_alt, candidate_reference):\n        for target in _RAD_EXCLUDE:\n            if not _rad_np.array_equal(\n                branch[target].to_numpy(), baseline[target].to_numpy()\n            ):\n                raise RuntimeError(f'E10 failed to preserve raw parent values for {target}')\n        _rad_validate(branch, expected_ids)\n    # Diagnostic E10 output only; the final equal rank mean is formed after E11\n    # and the legacy-DINO tie-break have completed independently in each branch.\n    candidate = baseline.copy()\n    alt_e10_rank = _rad_rank_columns(candidate_alt[_RAD_LABELS].to_numpy())\n    reference_e10_rank = _rad_rank_columns(\n        candidate_reference[_RAD_LABELS].to_numpy()\n    )\n    candidate[_RAD_LABELS] = (\n        _RAD_TWIN_ALT_WEIGHT * alt_e10_rank\n        + (1.0 - _RAD_TWIN_ALT_WEIGHT) * reference_e10_rank\n    )\n    _rad_validate(candidate, expected_ids)\n    e10_path = work / 'submission_e10_v2.csv'\n    candidate.to_csv(e10_path, index=False)\n    _rad_log(\n        f'twin E10 branches complete at alpha={_RAD_ALPHA:.2f}; '\n        f'preserved raw={list(_RAD_EXCLUDE)}'\n    )\n\n    # V48's successful run selected the E13 bundle a second time after\n    # installing the older E11 slot order. Express that observed behavior\n    # directly, without relying on duplicate-filename directory order.\n    globals().update(\n        SLOTS=list(_RAD_E11_SLOTS),\n        N_SLOT=len(_RAD_E11_SLOTS),\n        CACHE_SLICES=int(_RAD_E11_CACHE_SLICES),\n        IMG=int(_RAD_E11_IMG),\n        CACHE_IMG=int(_RAD_E11_IMG),\n        CROP_MM=float(_RAD_E11_CROP_MM),\n        RULES=dict(RULES_LEGACY),\n    )\n    headers = annotate(walk('test_series'))\n    studies, pixels, slot_mask = build_cache(\n        pick_slots(headers, plane), plane,\n        lat_of(headers, 'test-v48-pass2 '), 'test-v48-pass2'\n    )\n    by_uid = {str(uid): index for index, uid in enumerate(studies)}\n    missing = [uid for uid in expected_ids if uid not in by_uid]\n    if missing:\n        raise RuntimeError(f'{len(missing)} test studies absent from V48 pass-2 cache')\n    order = _rad_np.asarray([by_uid[uid] for uid in expected_ids], dtype=_rad_np.int64)\n    pixels, slot_mask = pixels[order], slot_mask[order]\n    v48_pass2_token_count = int(\n        _rad_np.repeat(slot_mask[:, :, None], CACHE_SLICES, axis=2).sum()\n    )\n    if v48_pass2_token_count < int(0.55 * len(test) * N_SLOT * CACHE_SLICES):\n        raise RuntimeError(\n            f'insufficient acquired V48 pass-2 slices: {v48_pass2_token_count}'\n        )\n    v48_features, v48_token_mask = _rad_encode(\n        encoder, pixels, slot_mask, device\n    )\n    del pixels, slot_mask, headers\n    _rad_gc.collect()\n    v48_pass2_predictions = [\n        _rad_predict_head(head, v48_features, v48_token_mask, device)\n        for head in e13_heads\n    ]\n    if len(v48_pass2_predictions) != 5:\n        raise RuntimeError('V48 second pass did not use all five E13 heads')\n    v48_pass2_probability = _rad_np.mean(\n        _rad_np.stack(v48_pass2_predictions), axis=0\n    )\n    if (\n        v48_pass2_probability.shape != (len(test), len(_RAD_LABELS))\n        or not _rad_np.isfinite(v48_pass2_probability).all()\n    ):\n        raise RuntimeError(\n            f'invalid V48 pass-2 prediction: {v48_pass2_probability.shape}'\n        )\n    v48_pass2_rank = _rad_rank_columns(v48_pass2_probability)\n\n    reference_branch = candidate_reference.copy()\n    reference_branch[_RAD_LABELS] = _rad_rank_columns(\n        (1.0 - _RAD_V48_SECOND_ALPHA)\n        * _rad_rank_columns(candidate_reference[_RAD_LABELS].to_numpy())\n        + _RAD_V48_SECOND_ALPHA * v48_pass2_rank\n    )\n    _rad_validate(reference_branch, expected_ids)\n    _rad_log(\n        f'V48 second E13 pass complete at alpha '\n        f'{_RAD_V48_SECOND_ALPHA:.2f} on the E11 slot layout'\n    )\n\n    # V48 deploys the pinned reference branch directly after the second pass.\n    # The alternative-head twin and legacy-DINO tie-break are not part of .917.\n    _RAD_CAL = _rad_json.loads(\n        _RadPath('/kaggle/working/rad_calibrator.json').read_text())\n    _RAD_CAL_GATE = set(_RAD_CAL['gate'])\n    _RAD_CAL_W = 0.40\n\n    def _rad_cal_protocol(uids):\n        frame = _rad_pd.read_csv(\n            ROOT / 'test_series.csv',\n            dtype={'StudyInstanceUID': str, 'SeriesInstanceUID': str},\n        )\n        frame['StudyInstanceUID'] = frame['StudyInstanceUID'].astype(str)\n        index = _rad_pd.Index([str(u) for u in uids], name='StudyInstanceUID')\n        table = _rad_pd.DataFrame(index=index)\n        table['n_series'] = frame.groupby(\n            'StudyInstanceUID').size().reindex(index).fillna(0)\n        for plane in ('Sagittal', 'Coronal', 'Axial'):\n            part = frame[frame['Anatomical_Plane'].astype(str) == plane]\n            table[f'n_{plane[:3]}'] = part.groupby(\n                'StudyInstanceUID').size().reindex(index).fillna(0)\n        for flag in ('Fat_Suppression', 'Fluid_Sensitive'):\n            marked = frame[_rad_pd.to_numeric(\n                frame[flag], errors='coerce').fillna(0) > 0]\n            table[flag[:3]] = marked.groupby(\n                'StudyInstanceUID').size().reindex(index).fillna(0)\n            for plane in ('Sagittal', 'Coronal', 'Axial'):\n                part = marked[marked['Anatomical_Plane'].astype(str) == plane]\n                table[f'{flag[:3]}_{plane[:3]}'] = part.groupby(\n                    'StudyInstanceUID').size().reindex(index).fillna(0)\n        if list(table.columns) != list(_RAD_CAL['protocol_columns']):\n            raise RuntimeError('calibration protocol layout mismatch')\n        return table.to_numpy(_rad_np.float64)\n\n    def _rad_calibrate(branch):\n        base = baseline_rank\n        public = reference_rank\n        pass2 = v48_pass2_rank\n        mean = (base + public + pass2) / 3.0\n        blocks = [base, public, pass2, public - base, pass2 - base, mean]\n        for _grp in _RAD_CAL['groups']:\n            cols = [_RAD_LABELS.index(t) for t in _grp]\n            blocks.append(mean[:, cols].mean(axis=1, keepdims=True))\n        blocks.append(_rad_cal_protocol(expected_ids))\n        x = _rad_np.concatenate(blocks, axis=1)\n        centre = _rad_np.asarray(_RAD_CAL['mean'], _rad_np.float64)\n        spread = _rad_np.asarray(_RAD_CAL['scale'], _rad_np.float64)\n        coef = _rad_np.asarray(_RAD_CAL['coef'], _rad_np.float64)\n        bias = _rad_np.asarray(_RAD_CAL['intercept'], _rad_np.float64)\n        if x.shape[1] != coef.shape[1]:\n            raise RuntimeError(\n                f'calibration expects {coef.shape[1]} columns, built {x.shape[1]}')\n        adjusted = _rad_rank_columns(((x - centre) / spread) @ coef.T + bias)\n        out = branch.copy()\n        values = out[_RAD_LABELS].to_numpy(_rad_np.float64).copy()\n        for index, target in enumerate(_RAD_LABELS):\n            if target in _RAD_CAL_GATE:\n                values[:, index] = (\n                    (1.0 - _RAD_CAL_W) * values[:, index]\n                    + _RAD_CAL_W * adjusted[:, index]\n                )\n        out[_RAD_LABELS] = _rad_rank_columns(values)\n        _rad_validate(out, expected_ids)\n        return out\n\n    final = _rad_calibrate(reference_branch)\n    globals()['V18_CALIBRATOR_APPLIED'] = True\n    globals()['V18_CAL_GATE'] = tuple(sorted(_RAD_CAL_GATE))\n    _rad_validate(final, expected_ids)\n    temporary = primary.with_suffix('.csv.tmp')\n    final.to_csv(temporary, index=False)\n    _rad_os.replace(temporary, primary)\n\n    receipt = {\n        'recipe': 'V48 deployed reference branch: correct E13@0.50-inside-Rad -> E10@0.50 -> same E13 on E11 layout@0.15',\n        'e13_member_weight_inside_rad': _RAD_E13_MEMBER_WEIGHT,\n        'e10_alpha': _RAD_ALPHA,\n        'e10_preserved_targets': list(_RAD_EXCLUDE),\n        'v48_second_alpha': _RAD_V48_SECOND_ALPHA,\n        'reference_heads_sha256': _RAD_REFERENCE_HEADS_SHA256,\n        'e13_heads_sha256': _RAD_E13_HEADS_SHA256,\n        'v48_second_heads_sha256': _RAD_E13_HEADS_SHA256,\n        'v48_second_slots': [list(slot) for slot in _RAD_E11_SLOTS],\n        'encoder_sha256': _RAD_ENCODER_SHA256,\n        'test_studies': len(expected_ids),\n        'e10_tokens': token_count,\n        'v48_second_tokens': v48_pass2_token_count,\n        'e13_tokens': e13_token_count,\n        'submission_sha256': _rad_sha256(primary),\n    }\n    (work / 'v50_v2_repro_receipt.json').write_text(\n        _rad_json.dumps(receipt, indent=2, sort_keys=True) + '\\n'\n    )\n    del (encoder, e13_heads, v48_pass2_predictions,\n         v48_pass2_probability, v48_pass2_rank, v48_features, v48_token_mask)\n    _rad_gc.collect()\n    _rad_torch.cuda.empty_cache()\n    _rad_log(\n        f'V48 reference branch complete; reference-v15={reference_heads_path}; '\n        f'e13-two-pass={e13_path}; second_alpha='\n        f'{_RAD_V48_SECOND_ALPHA:.2f}; encoder={encoder_path}; '\n        f'elapsed={(_rad_time.time()-started)/60:.1f}m'\n    )\n\n\n_rad_main()\n"},{"cell_type":"code","metadata":{},"source":"_rep_snapshot('rad')\n","execution_count":null,"outputs":[],"id":"reproduction-029"},{"cell_type":"markdown","metadata":{},"source":"## 4. CoAtNet: combine Raptor views and residual checkpoints\n\nFour Raptor views are averaged at 60/10/10/20 before ranking. Three residual checkpoints contribute 40% inside this branch. The final fusion gives it 60% for most findings and 100% for lateral meniscus. The prediction code below is unchanged from the scored source.\n","id":"reproduction-030"},{"cell_type":"code","metadata":{},"execution_count":null,"outputs":[],"source":"%%writefile /kaggle/working/raptor_arm.py\n# ===========================================================================\n# Raptor CoAtNet arm - INHERITED CODE, NOT WRITTEN BY THIS NOTEBOOK'S AUTHOR.\n#\n# Origin : Mattia Angeli, \"Bend the Knee to the Dinosaurs\"\n#          https://www.kaggle.com/code/mattiaangeli/bend-the-knee-to-the-dinosaurs\n# Weights: dreaddevelopment/raptor-knee-{maxspan,native384,native384dense}\n#          https://www.kaggle.com/datasets/dreaddevelopment/raptor-knee-maxspan\n# License: Apache-2.0 (Kaggle default for public notebooks).\n#\n# In the parent notebook this module was carried as a single escaped string\n# literal and run with exec(compile(...)). It is reproduced here verbatim as\n# ordinary readable source so that it can be read, diffed and audited. The\n# code is unchanged; only its presentation is.\n# ===========================================================================\nimport os, glob, time, gc, hashlib\nos.environ.setdefault('HF_HUB_OFFLINE', '1')\nos.environ.setdefault('TRANSFORMERS_OFFLINE', '1')\nos.environ.setdefault('HF_HUB_DISABLE_TELEMETRY', '1')\nimport numpy as np\nimport torch, torch.nn as nn, torch.nn.functional as F\nimport timm\ntorch.backends.cudnn.benchmark = True\ntorch.backends.cuda.matmul.allow_tf32 = True\nIMG = 336\nCROP_MM = 140.0\nSPAN_LO, SPAN_HI = 0.02, 0.98\nSLOTS = [(\"Sagittal\", 1, 18), (\"Sagittal\", 0, 14),\n         (\"Coronal\", 1, 12), (\"Coronal\", 0, 8), (\"Axial\", -1, 12)]\nMAXS = sum(slot[2] for slot in SLOTS)\nK_EVAL = 62\nNORM = \"imagenet\"\nLAB = [\"ACL\", \"MCL\", \"Medial Meniscus\", \"Lateral Meniscus\", \"Medial OA\",\n       \"Lateral OA\", \"PF OA\", \"Effusion\", \"Synovitis\", \"Baker's\",\n       \"Contusion\", \"Fracture\"]\n_MEAN = torch.tensor([0.485, 0.456, 0.406]).view(3, 1, 1)\n_STD = torch.tensor([0.229, 0.224, 0.225]).view(3, 1, 1)\n_SLOTS64 = [(\"Sagittal\", 1, 18), (\"Sagittal\", 0, 14),\n            (\"Coronal\", 1, 12), (\"Coronal\", 0, 8), (\"Axial\", -1, 12)]\n_SLOTS44 = [(\"Sagittal\", 1, 12), (\"Sagittal\", 0, 10),\n            (\"Coronal\", 1, 8), (\"Coronal\", 0, 6), (\"Axial\", -1, 8)]\nARMS = [\n    {\"name\": \"maxspan-v5\", \"file\": \"raptor_ft_coatnet_v5_full_swa.pt\",\n     \"arch\": \"coatnet_rmlp_2_rw_384.sw_in12k_ft_in1k\", \"res\": 384,\n     \"img\": 336, \"slots\": _SLOTS64, \"span\": (0.02, 0.98), \"k_eval\": 62,\n     \"reverse\": False, \"w\": 0.55},\n    {\"name\": \"native384dense-v10\", \"file\": \"raptor_ft_coatnet_v10_full.pt\",\n     \"arch\": \"coatnet_rmlp_2_rw_384.sw_in12k_ft_in1k\", \"res\": 384,\n     \"img\": 384, \"slots\": _SLOTS64, \"span\": (0.02, 0.98), \"k_eval\": 62,\n     \"reverse\": False, \"w\": 0.1},\n    {\"name\": \"maxspan-v5-reverse\", \"file\": \"raptor_ft_coatnet_v5_full_swa.pt\",\n     \"arch\": \"coatnet_rmlp_2_rw_384.sw_in12k_ft_in1k\", \"res\": 384,\n     \"img\": 336, \"slots\": _SLOTS64, \"span\": (0.02, 0.98), \"k_eval\": 62,\n     \"reverse\": True, \"w\": 0.15},\n    {\"name\": \"native384-v8\", \"file\": \"raptor_ft_coatnet_v8_full_swa.pt\",\n     \"arch\": \"coatnet_rmlp_2_rw_384.sw_in12k_ft_in1k\", \"res\": 384,\n     \"img\": 384, \"slots\": _SLOTS44, \"span\": (0.06, 0.94), \"k_eval\": 42,\n     \"reverse\": False, \"w\": 0.2},\n]\n\ndef build_backbone(arch, pretrained=False):\n    hybrid = arch.startswith(('maxvit', 'maxxvit', 'coatnet', 'coat_', 'convnext'))\n    is_vit = not hybrid and any((k in arch for k in ('vit', 'deit', 'dinov2', 'eva', 'beit')))\n    kw = dict(pretrained=pretrained, num_classes=0, in_chans=3)\n    if is_vit:\n        kw.update(global_pool='token', dynamic_img_size=True)\n    else:\n        kw.update(global_pool='avg')\n    return timm.create_model(arch, **kw)\n\nclass RaptorClassifier(nn.Module):\n\n    def __init__(self, backbone, F_dim=768, n=12, drop=0.2):\n        super().__init__()\n        self.backbone = backbone\n        self.norm = nn.LayerNorm(F_dim)\n        self.att = nn.Sequential(nn.Linear(F_dim, 256), nn.Tanh(), nn.Dropout(drop), nn.Linear(256, n))\n        self.clsW = nn.Parameter(torch.zeros(n, F_dim))\n        self.clsb = nn.Parameter(torch.zeros(n))\n        nn.init.trunc_normal_(self.clsW, std=0.02)\n        self.n = n\n\n    def encode(self, x):\n        B, K = x.shape[:2]\n        f = self.backbone(x.flatten(0, 1))\n        return f.view(B, K, -1)\n\n    def head(self, feats):\n        h = self.norm(feats)\n        a = self.att(h)\n        a = torch.softmax(a, dim=1)\n        pooled = torch.einsum('bkn,bkf->bnf', a, h)\n        logits = (pooled * self.clsW).sum(-1) + self.clsb\n        return logits\n\n    def forward(self, x):\n        return self.head(self.encode(x))\n\ndef load_model(pt_path, arch_default, res_default, device, ngpu=1):\n    ck = torch.load(pt_path, map_location='cpu', weights_only=False)\n    arch = ck.get('arch', arch_default)\n    ck_res = int(ck.get('res', res_default))\n    bb = build_backbone(arch, pretrained=False)\n    model = RaptorClassifier(bb, F_dim=bb.num_features)\n    model.load_state_dict(ck['model'], strict=True)\n    model.eval().to(device)\n    del ck\n    gc.collect()\n    return (model, ck_res)\n\ndef load_refit_head(pt_path, feature_dim, device):\n    ck = torch.load(pt_path, map_location='cpu', weights_only=False)\n    state = ck.get('model', ck)\n    head = RaptorClassifier(nn.Identity(), F_dim=int(feature_dim))\n    head_state = {name: tensor for name, tensor in state.items()\n                  if not name.startswith('backbone.')}\n    head.load_state_dict(head_state, strict=True)\n    head.eval().to(device)\n    del ck, state, head_state\n    gc.collect()\n    return head\n\ndef _eval_centers(mask, D, k):\n    valid = np.where(mask > 0)[0]\n    if len(valid) < 3:\n        valid = np.arange(min(3, D))\n    lo, hi = (int(valid.min()), int(valid.max()))\n    cs = [c for c in range(lo + 1, hi) if c - 1 >= lo and c + 1 <= hi]\n    if not cs:\n        cs = [max(1, min((lo + hi) // 2, D - 2))]\n    idx = np.linspace(0, len(cs) - 1, k).round().astype(int)\n    return [cs[i] for i in idx]\n\ndef eval_windows(vol, mask, k, res, norm=NORM):\n    D = vol.shape[0]\n    cs = _eval_centers(mask, D, k)\n    wins = np.empty((len(cs), 3, res, res), np.float32)\n    for j, c in enumerate(cs):\n        c = max(1, min(c, D - 2))\n        tri = np.stack([vol[c - 1], vol[c], vol[c + 1]], 0).astype(np.float32) / 255.0\n        t = torch.from_numpy(tri)\n        if t.shape[-1] != res:\n            t = F.interpolate(t[None], size=(res, res), mode='bilinear', align_corners=False)[0]\n        wins[j] = t.numpy()\n    x = torch.from_numpy(wins)\n    if norm == 'imagenet':\n        x = (x - _MEAN) / _STD\n    return x\n\n@torch.no_grad()\ndef infer_probs(model, xwins, device):\n    x = xwins.unsqueeze(0).to(device)\n    use_cuda = device != 'cpu' and str(device).startswith('cuda')\n    if use_cuda:\n        try:\n            with torch.autocast('cuda', dtype=torch.float16):\n                o = torch.sigmoid(model(x).float())\n            return o[0].cpu().numpy()\n        except RuntimeError:\n            torch.cuda.empty_cache()\n            o = torch.sigmoid(model(x).float())\n            return o[0].cpu().numpy()\n    o = torch.sigmoid(model(x).float())\n    return o[0].cpu().numpy()\n\n@torch.no_grad()\ndef infer_probs_two_heads(model, refit_head, xwins, device):\n    x = xwins.unsqueeze(0).to(device)\n    use_cuda = device != 'cpu' and str(device).startswith('cuda')\n    def forward_heads():\n        features = model.encode(x)\n        original = torch.sigmoid(model.head(features).float())\n        refitted = torch.sigmoid(refit_head.head(features).float())\n        return (original[0].cpu().numpy(), refitted[0].cpu().numpy())\n    if use_cuda:\n        try:\n            with torch.autocast('cuda', dtype=torch.float16):\n                return forward_heads()\n        except RuntimeError:\n            torch.cuda.empty_cache()\n            return forward_heads()\n    return forward_heads()\n\ndef rankpct(x):\n    order = x.argsort(0).argsort(0).astype(np.float64)\n    return order / max(1, x.shape[0] - 1)\n\ndef _make_reader():\n    import pydicom, cv2\n    from pydicom.pixel_data_handlers.util import apply_modality_lut\n\n    def order_and_meta(sdir):\n        fs = glob.glob(sdir + '/*.dcm')\n        recs = []\n        ps_list = []\n        for f in fs:\n            try:\n                h = pydicom.dcmread(f, stop_before_pixels=True)\n                iop = getattr(h, 'ImageOrientationPatient', None)\n                ipp = getattr(h, 'ImagePositionPatient', None)\n                if iop is not None and ipp is not None and (len(iop) == 6):\n                    r = np.array(iop[:3], float)\n                    c = np.array(iop[3:], float)\n                    n = np.cross(r, c)\n                    pos = float(np.dot(np.array(ipp, float), n))\n                else:\n                    pos = float(getattr(h, 'InstanceNumber', 0) or 0)\n                ps = getattr(h, 'PixelSpacing', None)\n                ps = float(ps[0]) if ps is not None else 0.5\n                ps_list.append(ps)\n                recs.append((pos, f, ps))\n            except Exception:\n                recs.append((0.0, f, 0.5))\n        recs.sort(key=lambda x: x[0])\n        med_ps = float(np.median(ps_list)) if ps_list else 0.5\n        return ([(f, ps) for _, f, ps in recs], med_ps)\n\n    def read_px(f):\n        d = pydicom.dcmread(f)\n        a = apply_modality_lut(d.pixel_array, d).astype(np.float32)\n        if str(getattr(d, 'PhotometricInterpretation', '')) == 'MONOCHROME1':\n            a = a.max() - a\n        return a\n\n    def mm_crop_resize(a, ps):\n        h, w = a.shape\n        cpx = int(round(CROP_MM / max(ps, 0.001)))\n        cpx = min(cpx, min(h, w))\n        y0 = (h - cpx) // 2\n        x0 = (w - cpx) // 2\n        a = a[y0:y0 + cpx, x0:x0 + cpx]\n        return cv2.resize(a, (IMG, IMG), interpolation=cv2.INTER_AREA)\n    return (order_and_meta, read_px, mm_crop_resize)\n\ndef _pick_series_for_slot(rows, plane, fluid, used):\n    cands = [r for r in rows if r['Anatomical_Plane'] == plane and r['SeriesInstanceUID'] not in used]\n    if fluid in (0, 1):\n        pref = [r for r in cands if int(r.get('Fluid_Sensitive', 0) or 0) == fluid]\n        if pref:\n            return pref[0]\n    return cands[0] if cands else None\n\ndef build_study(sid, ser_records, tsdir, reader):\n    order_and_meta, read_px, mm_crop_resize = reader\n    rows = ser_records.get(sid, [])\n    vol = np.zeros((MAXS, IMG, IMG), np.uint8)\n    idx = 0\n    used = set()\n    for plane, fluid, k in SLOTS:\n        r = _pick_series_for_slot(rows, plane, fluid, used)\n        if r is None:\n            idx += k\n            continue\n        used.add(r['SeriesInstanceUID'])\n        files, med_ps = order_and_meta(f\"{tsdir}/{sid}/{r['SeriesInstanceUID']}\")\n        if not files:\n            idx += k\n            continue\n        n = len(files)\n        lo, hi = (int(n * SPAN_LO), int(n * SPAN_HI) - 1)\n        hi = max(hi, lo)\n        picks = np.linspace(lo, hi, k).round().astype(int) if n > 1 else [0] * k\n        arrs = []\n        pss = []\n        for p in picks:\n            fp, ps = files[min(p, n - 1)]\n            try:\n                arrs.append(read_px(fp))\n                pss.append(ps)\n            except Exception:\n                arrs.append(None)\n                pss.append(med_ps)\n        valid = [a for a in arrs if a is not None]\n        if valid:\n            allpx = np.concatenate([a.ravel() for a in valid])\n            loq, hiq = np.percentile(allpx, [2.0, 98.0])\n        else:\n            loq, hiq = (0.0, 1.0)\n        for a, ps in zip(arrs, pss):\n            if idx >= MAXS:\n                break\n            if a is None:\n                idx += 1\n                continue\n            aw = np.clip((a - loq) / (hiq - loq + 1e-06), 0, 1)\n            aw = mm_crop_resize(aw, ps if ps > 0 else med_ps)\n            vol[idx] = (aw * 255).astype(np.uint8)\n            idx += 1\n        if idx >= MAXS:\n            break\n    mask = (vol.reshape(MAXS, -1).sum(1) > 0).astype(np.uint8)\n    return (vol, mask)\n\ndef find_test_root():\n    cands = ['/kaggle/input/competitions/rsna-knee-abnormality-detection', '/kaggle/input/rsna-knee-abnormality-detection']\n    for b in cands:\n        if os.path.exists(b + '/test.csv'):\n            return b\n    for d, _, f in os.walk('/kaggle/input'):\n        if 'test.csv' in f and (os.path.isdir(d + '/test_series') or os.path.isdir(d + '/test_images')):\n            return d\n    for d, _, f in os.walk('/kaggle/input'):\n        if 'test.csv' in f:\n            return d\n    raise RuntimeError('no test root under /kaggle/input')\n\ndef find_weight_file(fname):\n    direct = [f'/kaggle/input/raptor-knee-maxspan/{fname}', f'/kaggle/input/raptor-knee-native384dense/{fname}', f'/kaggle/input/raptor-knee-native384/{fname}', f'/kaggle/input/raptor-knee-arms/{fname}', f'/kaggle/input/raptor-knee-arms/1/{fname}', f'/kaggle/input/raptor-cnn336/{fname}']\n    for p in direct:\n        if os.path.exists(p):\n            return p\n    for d in sorted(glob.glob('/kaggle/input/*/')):\n        if 'competition' in d.lower():\n            continue\n        hits = glob.glob(os.path.join(d, '**', fname), recursive=True)\n        if hits:\n            return hits[0]\n    raise RuntimeError(f'{fname} not found under /kaggle/input')\n\ndef find_optional_verified_weight(fname, expected_sha256, root='/kaggle/input'):\n    hits = []\n    for directory in sorted(glob.glob(os.path.join(root, '*/'))):\n        if 'competition' in directory.lower():\n            continue\n        hits.extend(glob.glob(os.path.join(directory, '**', fname), recursive=True))\n    for path in sorted(set(hits)):\n        digest = hashlib.sha256()\n        with open(path, 'rb') as stream:\n            for chunk in iter(lambda: stream.read(8 * 1024 * 1024), b''):\n                digest.update(chunk)\n        if digest.hexdigest() == expected_sha256:\n            return path\n        print(f'[head-refit] ignored hash-mismatched optional checkpoint: {path}', flush=True)\n    return None\n\ndef main():\n    import pandas as pd\n    t0 = time.time()\n    dev = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n    print(f\"device {dev} | gpus {torch.cuda.device_count()} | torch {torch.__version__}\", flush=True)\n    root = find_test_root()\n    tsdir = root + \"/test_series\"\n    if not os.path.isdir(tsdir):\n        tsdir = root + \"/test_images\"\n    print(\"test root:\", root, \"| series dir:\", tsdir, flush=True)\n    test = pd.read_csv(root + \"/test.csv\")\n    test[\"StudyInstanceUID\"] = test[\"StudyInstanceUID\"].astype(str)\n    test_ids = test[\"StudyInstanceUID\"].tolist()\n    tser = pd.read_csv(root + \"/test_series.csv\")\n    tser[\"StudyInstanceUID\"] = tser[\"StudyInstanceUID\"].astype(str)\n    tser[\"SeriesInstanceUID\"] = tser[\"SeriesInstanceUID\"].astype(str)\n    series = {key: frame.to_dict(\"records\") for key, frame in tser.groupby(\"StudyInstanceUID\")}\n    print(f\"test studies {len(test_ids)} | test series {len(tser)}\", flush=True)\n    sub_cols = [\"StudyInstanceUID\"] + LAB\n    sample = os.path.join(root, \"sample_submission.csv\")\n    if os.path.exists(sample):\n        sub_cols = list(pd.read_csv(sample, nrows=1).columns)\n    reader = _make_reader()\n    n_study, n_arm = len(test_ids), len(ARMS)\n    arm_probs = [np.full((n_study, len(LAB)), 0.5, np.float32) for _ in range(n_arm)]\n    for arm_index, arm in enumerate(ARMS):\n        globals()[\"IMG\"] = int(arm[\"img\"])\n        globals()[\"SLOTS\"] = list(arm[\"slots\"])\n        globals()[\"MAXS\"] = sum(slot[2] for slot in SLOTS)\n        globals()[\"SPAN_LO\"], globals()[\"SPAN_HI\"] = map(float, arm[\"span\"])\n        globals()[\"K_EVAL\"] = int(arm[\"k_eval\"])\n        weight_path = find_weight_file(arm[\"file\"])\n        model, resolution = load_model(weight_path, arm[\"arch\"], arm[\"res\"], dev)\n        print(f\"[arm {arm_index}] {arm['name']} | img {IMG} | slices {MAXS} | \"\n              f\"span {SPAN_LO:.2f}-{SPAN_HI:.2f} | windows {K_EVAL} | \"\n              f\"res {resolution} | {time.time() - t0:.0f}s\", flush=True)\n        for study_index, study_uid in enumerate(test_ids):\n            try:\n                volume, mask = build_study(study_uid, series, tsdir, reader)\n                windows = eval_windows(volume, mask, k=K_EVAL, res=resolution, norm=NORM)\n                if bool(arm.get(\"reverse\", False)):\n                    windows = windows.flip(1).contiguous()\n                arm_probs[arm_index][study_index] = infer_probs(model, windows, dev)\n                del volume, mask, windows\n            except Exception as error:\n                print(f\"  [arm {arm_index}] study {study_index} {study_uid[:16]} FALLBACK \"\n                      f\"({type(error).__name__}: {error})\", flush=True)\n            if (study_index + 1) % 100 == 0 or study_index + 1 == n_study:\n                print(f\"  [arm {arm_index}] {study_index + 1}/{n_study} | \"\n                      f\"{time.time() - t0:.0f}s\", flush=True)\n        del model\n        gc.collect()\n        if str(dev).startswith(\"cuda\"):\n            torch.cuda.empty_cache()\n        print(f\"[arm {arm_index}] done + freed | {time.time() - t0:.0f}s\", flush=True)\n    weights = np.array([float(arm.get(\"w\", 1.0)) for arm in ARMS], dtype=np.float64)\n    weights /= weights.sum()\n    print(f\"[blend] global probability mean w=\"\n          f\"{dict(zip([arm['name'] for arm in ARMS], weights.round(4)))}\", flush=True)\n    probability_blend = np.tensordot(\n        weights, np.stack([np.clip(values, 0, 1) for values in arm_probs]), axes=(0, 0))\n    ranks = rankpct(probability_blend)\n    if not np.isfinite(ranks).all():\n        ranks[~np.isfinite(ranks)] = 0.5\n    submission = pd.DataFrame(ranks.astype(np.float32), columns=LAB)\n    submission.insert(0, \"StudyInstanceUID\", test_ids)\n    submission = submission[sub_cols]\n    assert submission[\"StudyInstanceUID\"].tolist() == test_ids\n    assert np.isfinite(submission[LAB].values).all()\n    out = \"/kaggle/working/_raptor.csv\"\n    submission.to_csv(out, index=False)\n    print(\"wrote\", out, \"|\", len(submission), \"rows x\", len(submission.columns), \"cols\", flush=True)\n    print(submission.head().to_string(index=False), flush=True)\n    print(f\"DONE {time.time() - t0:.0f}s\", flush=True)\n"},{"cell_type":"code","metadata":{},"execution_count":null,"outputs":[],"source":"# Import the Raptor arm as a real module (its own globals, exactly as the\n# parent's exec-into-a-fresh-dict provided) and install the raw-output guards.\nimport sys as _ke_sys\nif '/kaggle/working' not in _ke_sys.path:\n    _ke_sys.path.insert(0, '/kaggle/working')\nimport raptor_arm as _ke_raptor\n\n_rep_raptor_checked_predictions = 0\n_rep_raptor_checked_ranks = 0\n\ndef _rep_wrap_raptor(module):\n    \"\"\"Audit wrapper: count and range-check raw outputs without altering them.\"\"\"\n    global _rep_raptor_guard_installed\n    import numpy as _np\n    _orig_predict, _orig_rank = module.infer_probs, module.rankpct\n\n    def checked_predict(*a, **k):\n        global _rep_raptor_checked_predictions\n        out = _orig_predict(*a, **k)\n        assert _np.asarray(out).shape == (12,), 'Raptor raw prediction shape'\n        assert _np.isfinite(out).all(), 'Raptor raw nonfinite predictions'\n        _rep_raptor_checked_predictions += 1\n        return out\n\n    def checked_rank(values):\n        global _rep_raptor_checked_ranks\n        assert _np.isfinite(values).all(), 'Raptor raw nonfinite blend'\n        _rep_raptor_checked_ranks += 1\n        return _orig_rank(values)\n\n    module.infer_probs, module.rankpct = checked_predict, checked_rank\n    _rep_raptor_guard_installed = True\n\n_rep_wrap_raptor(_ke_raptor)\n_ke_raptor.main()\n"},{"cell_type":"code","metadata":{},"execution_count":null,"outputs":[],"source":"import numpy as _ke_np\nimport pandas as _ke_pd\nfrom pathlib import Path as _KePath\n\n_ke_primary = _KePath('/kaggle/working/submission.csv')\n_ke_ours = _ke_pd.read_csv(_ke_primary, dtype={'StudyInstanceUID': str})\n_KE_LAB = [c for c in _ke_ours.columns if c != 'StudyInstanceUID']\n\ndef _coat_substitute():\n    import hashlib as _h, os as _o, subprocess as _sp, sys as _sy\n    from pathlib import Path as _P\n    import pandas as _pd\n\n    MAN_SHA = '98511a8fdeb9da0e6e70c78d013ff636e1476f31c80b5dc134d294b18c3f284e'\n    WHL_SHA = '236c8df54a90f4d02076e6f9c1cc763d794542e886c576a6fee46ec8ff75a7a9'\n    raptor = _P('/kaggle/working/_raptor.csv')\n\n    def sha(p):\n        d = _h.sha256()\n        with _P(p).open('rb') as f:\n            for b in iter(lambda: f.read(8 << 20), b''):\n                d.update(b)\n        return d.hexdigest()\n\n    def find(name, want):\n        root = _P('/kaggle/input')\n        if not root.is_dir():\n            return None\n        for base in sorted(root.iterdir()):\n            if base.name in ('competitions', 'train_series', 'test_series'):\n                continue\n            for p in sorted(base.rglob(name)):\n                if p.is_file() and sha(p) == want:\n                    return p\n        return None\n\n    man = find('coat_resgated_ep10_top3_manifest.json', MAN_SHA)\n    if man is None:\n        raise RuntimeError('coat manifest absent or hash mismatch')\n    art = man.parent\n    whl = find('opencv_python_headless-4.12.0.88-*.whl', WHL_SHA)\n    if whl is None:\n        for c in sorted(_P('/kaggle/input').rglob('opencv_python_headless-4.12.0.88-*.whl')):\n            if sha(c) == WHL_SHA:\n                whl = c\n                break\n    if whl is None:\n        raise RuntimeError('pinned opencv wheel absent or hash mismatch')\n\n    envd = _P('/kaggle/working/_coat_env')\n    _sp.run([_sy.executable, '-m', 'pip', 'install', '--no-deps', '--quiet',\n             '--target', str(envd), str(whl)], check=True)\n\n    out = _P('/kaggle/working/_coat_arm.csv')\n    child = (\n        \"import sys, json\\n\"\n        f\"sys.path.insert(0, {str(envd)!r})\\n\"\n        f\"sys.path.insert(0, {str(art)!r})\\n\"\n        \"import cv2; assert cv2.__version__ == '4.12.0', cv2.__version__\\n\"\n        \"import torch; assert torch.cuda.device_count() == 2\\n\"\n        \"import coatnet_resgated_ep10_top3_inference as rt\\n\"\n        \"assert rt.base.cv2.__version__ == '4.12.0'\\n\"\n        \"from pathlib import Path\\n\"\n        \"r = rt.run_submission(competition_root=rt.base.find_competition_root(),\\n\"\n        f\"    artifact_root=Path({str(art)!r}), output_path=Path({str(out)!r}),\\n\"\n        \"    gpu_batch_studies=2, backbone_micro_images=8)\\n\"\n        \"assert r['status'] == rt.SUBMISSION_STATUS\\n\"\n        \"assert r['models'] == 3\\n\"\n        \"assert [i['epoch'] for i in r['checkpoints']] == [4, 6, 8]\\n\"\n        \"assert r['fallback_studies'] == 0, r['failures']\\n\"\n        \"Path('/kaggle/working/_coat_arm_receipt.json').write_text(json.dumps(r, indent=2))\\n\")\n    env = dict(_o.environ)\n    env['PYTHONPATH'] = f\"{envd}:{art}:\" + env.get('PYTHONPATH', '')\n    proc = _sp.run([_sy.executable, '-c', child], env=env, capture_output=True, text=True)\n    if proc.returncode != 0:\n        raise RuntimeError(f'coat child failed: {proc.stderr[-700:]}')\n\n    pub = _pd.read_csv(raptor, dtype={'StudyInstanceUID': str})\n    ours = _pd.read_csv(out, dtype={'StudyInstanceUID': str})\n    if list(ours.columns) != list(pub.columns):\n        raise RuntimeError('coat arm column drift')\n    ours = ours.set_index('StudyInstanceUID').reindex(\n        pub.StudyInstanceUID.astype(str).tolist()).reset_index()\n    lab = [c for c in pub.columns if c != 'StudyInstanceUID']\n    if ours[lab].isna().any().any():\n        raise RuntimeError('coat arm does not cover every study')\n    import json as _j\n    import numpy as _np\n    private_alpha = 0.40000000000000002\n    public_rank = pub[lab].rank(method='average', pct=True)\n    private_rank = ours[lab].rank(method='average', pct=True)\n    hybrid = pub.copy()\n    hybrid[lab] = (\n        (1.0 - private_alpha) * public_rank\n        + private_alpha * private_rank\n    )\n    if not _np.isfinite(hybrid[lab].to_numpy(_np.float64)).all():\n        raise RuntimeError('CoAt/Raptor hybrid contains non-finite values')\n    tmp = raptor.with_name('.raptor_coat_hybrid.csv')\n    hybrid.to_csv(tmp, index=False)\n    _o.replace(tmp, raptor)\n    raptor.with_name('_coat_raptor_blend_receipt.json').write_text(_j.dumps({\n        'contract': 'public_raptor_private_residual_coat_global_rank_blend_v1',\n        'private_alpha': private_alpha,\n        'public_raptor_alpha': 1.0 - private_alpha,\n        'study_count': len(ours),\n        'finding_specific_weights': False,\n    }, indent=2, sort_keys=True) + '\\n')\n    return len(ours)\n\n\n_coat_public_path = _KePath('/kaggle/working/_raptor.csv')\n_coat_n = _coat_substitute()\nprint(f'[coat-arm] blended OUR resgated e4/e6/e8 into the public Raptor arm '\n      f'(private alpha 0.400; {_coat_n} studies)', flush=True)\n\n_ke_theirs = _ke_pd.read_csv('/kaggle/working/_raptor.csv',\n                             dtype={'StudyInstanceUID': str})\nassert list(_ke_theirs.columns) == list(_ke_ours.columns), 'column drift'\n_ke_theirs = _ke_theirs.set_index('StudyInstanceUID').reindex(\n    _ke_ours['StudyInstanceUID']).reset_index()\nassert _ke_theirs[_KE_LAB].notna().all().all(), 'study identity drift'\n\n\n_ke_tr = _ke_ours[_KE_LAB].rank(method='average', pct=True)\n_ke_cr = _ke_theirs[_KE_LAB].copy()\n_blend_transformer = _ke_ours.copy()\n_blend_coatnet = _ke_theirs.copy()\n_blend_labels = list(_KE_LAB)\n_blend_tr = _ke_tr.copy()\n_blend_cr = _ke_cr.copy()\n_coatnet_weight = {label: 0.60 for label in _blend_labels}\n# probe22 per-finding outer weights (maverickss26, Apache-2.0): raising these\n# five is what took their notebook from 0.939 to 0.941.\n_coatnet_weight.update({'ACL': 0.75, 'Medial Meniscus': 0.8, 'Lateral Meniscus': 1.0, 'Lateral OA': 0.75, 'Fracture': 0.75})\n_blend_output = _blend_transformer.copy()\nfor _blend_label in _blend_labels:\n    _blend_w = float(_coatnet_weight[_blend_label])\n    _blend_output[_blend_label] = (\n        (1.0 - _blend_w) * _blend_tr[_blend_label]\n        + _blend_w * _blend_cr[_blend_label]\n    )\n_blend_output[_blend_labels] = _blend_output[_blend_labels].rank(\n    method='average', pct=True\n)\nassert _ke_np.isfinite(\n    _blend_output[_blend_labels].to_numpy(_ke_np.float64)\n).all()\n_blend_output.to_csv(_ke_primary, index=False)\n"},{"cell_type":"markdown","metadata":{},"source":"## Sources, licenses, and limits\n\nThe assembled inference code is reproduced from Mattia Angeli's V27 notebook under\n**Apache 2.0**. Its code comments and embedded notices are retained. Original\nexplanations and verification cells were added for this release. The source\nnotebook file is authenticated by SHA-256\n`6b03919bbca5fe61e37c5ea6546609a771cd7db6503121f3e2dacdd8c9b6b052`.\n\n| Source | Role and attribution |\n|---|---|\n| [pilkwang's RSNA weights](https://www.kaggle.com/datasets/pilkwang/rsna-knee-weights) | Released DINO MRI ensemble; dataset CC0-1.0, original backbone terms retained |\n| [Mattia Angeli's fold weights](https://www.kaggle.com/datasets/mattiaangeli/knee-mri-fold-weights) | Five A5 folds and their saved inference configuration; dataset CC0-1.0, original backbone terms retained |\n| [Johnathan Wagner / DreadDevelopment's Max-Span](https://www.kaggle.com/datasets/dreaddevelopment/raptor-knee-maxspan), [Native384](https://www.kaggle.com/datasets/dreaddevelopment/raptor-knee-native384), and [Native384 Dense](https://www.kaggle.com/datasets/dreaddevelopment/raptor-knee-native384dense) | Raptor CoAtNet checkpoints and view contracts; all three datasets CC0-1.0 |\n| [Mattia Angeli's residual CoAtNet release](https://www.kaggle.com/datasets/mattiaangeli/rsna-knee-coat-resgated-ep10-top3) | Epochs 4/6/8 and authenticated runtime package; dataset CC0-1.0 |\n| [Antoine's reference Rad heads](https://www.kaggle.com/datasets/antoinegg1/rsna-knee-e9-radimagenet-heads-v15) | Operative reference-v15 heads; **CC-BY-NC-SA-4.0** for the generated model artifacts |\n| [Marwan's RadImageNet mirror](https://www.kaggle.com/datasets/marwanmath/resnet-50-radimagenet-marwan) | BMEII-AI / RadImageNet ResNet-50 encoder; Kaggle weight metadata **CC-BY-NC-SA-4.0** |\n| [Sofia Anjenje's E11](https://www.kaggle.com/code/sofiaanjenje/rsna-knee-e11-train) and [E13](https://www.kaggle.com/code/sofiaanjenje/rsna-knee-e13-train) | Public training implementations and E13 heads; notebook code Apache 2.0, with the inherited encoder's model terms retained |\n| [Meta DINOv2](https://www.kaggle.com/models/metaresearch/dinov2/PyTorch/small/1) | Attached DINOv2 backbone; retain its model license |\n| [Meta DINOv3](https://github.com/facebookresearch/dinov3) | A5 uses `vit_small_patch16_dinov3.lvd1689m`; the [DINOv3 License](https://github.com/facebookresearch/dinov3/blob/main/LICENSE.md) remains distinct from the fold dataset's CC0 grant |\n| [RSNA competition](https://www.kaggle.com/competitions/rsna-knee-abnormality-detection) | MRI studies and competition data; competition access and data terms apply |\n| [OpenCV Python 4.12.0.88](https://pypi.org/project/opencv-python-headless/4.12.0.88/) | Exact wheel SHA-256 matches the official release; Apache 2.0, with bundled third-party notices retained |\n\nCode licensing does not relicense pretrained weights. The RadImageNet components\ncarry noncommercial and share-alike conditions separately from Apache-licensed\nnotebook code. This is a Kaggle research checkpoint. Review the original model\nand dataset terms before redistributing weights or considering another use.\nLicense texts: [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0),\n[CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/), and\n[CC0 1.0](https://creativecommons.org/publicdomain/zero/1.0/).\nDependencies retained solely for exact source compatibility are listed in the\nattached-input manifest; attaching a resource does not make it an active model.\nOur historical V52 alternative Rad heads produce a diagnostic CSV; the final\nwriter uses reference-v15 and E13. They are not an extra final ensemble member.\n\nThe inherited checkpoints and calibrator used their authors' training and model\nselection procedures, including official labels in some releases. We have not\ncreated a new independent validation set by reproducing them. No new Gold-label\nevaluation or leaderboard-driven weight search occurs here. A public score on\nthis competition does not establish performance on other patients or scanners.\n\n## Verify the final artifact\n\nThe following checks inspect the predictions already computed. They do not alter\nthe ensemble. A missing branch, study-level model fallback, UID mismatch, or\nnonfinite output fails the run before it can qualify for submission.\n","id":"reproduction-033"},{"cell_type":"markdown","metadata":{},"source":"<details><summary>Full license: APACHE-2.0.txt</summary>\n\n\n                                 Apache License\n                           Version 2.0, January 2004\n                        http://www.apache.org/licenses/\n\n   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n   1. Definitions.\n\n      \"License\" shall mean the terms and conditions for use, reproduction,\n      and distribution as defined by Sections 1 through 9 of this document.\n\n      \"Licensor\" shall mean the copyright owner or entity authorized by\n      the copyright owner that is granting the License.\n\n      \"Legal Entity\" shall mean the union of the acting entity and all\n      other entities that control, are controlled by, or are under common\n      control with that entity. For the purposes of this definition,\n      \"control\" means (i) the power, direct or indirect, to cause the\n      direction or management of such entity, whether by contract or\n      otherwise, or (ii) ownership of fifty percent (50%) or more of the\n      outstanding shares, or (iii) beneficial ownership of such entity.\n\n      \"You\" (or \"Your\") shall mean an individual or Legal Entity\n      exercising permissions granted by this License.\n\n      \"Source\" form shall mean the preferred form for making modifications,\n      including but not limited to software source code, documentation\n      source, and configuration files.\n\n      \"Object\" form shall mean any form resulting from mechanical\n      transformation or translation of a Source form, including but\n      not limited to compiled object code, generated documentation,\n      and conversions to other media types.\n\n      \"Work\" shall mean the work of authorship, whether in Source or\n      Object form, made available under the License, as indicated by a\n      copyright notice that is included in or attached to the work\n      (an example is provided in the Appendix below).\n\n      \"Derivative Works\" shall mean any work, whether in Source or Object\n      form, that is based on (or derived from) the Work and for which the\n      editorial revisions, annotations, elaborations, or other modifications\n      represent, as a whole, an original work of authorship. For the purposes\n      of this License, Derivative Works shall not include works that remain\n      separable from, or merely link (or bind by name) to the interfaces of,\n      the Work and Derivative Works thereof.\n\n      \"Contribution\" shall mean any work of authorship, including\n      the original version of the Work and any modifications or additions\n      to that Work or Derivative Works thereof, that is intentionally\n      submitted to Licensor for inclusion in the Work by the copyright owner\n      or by an individual or Legal Entity authorized to submit on behalf of\n      the copyright owner. For the purposes of this definition, \"submitted\"\n      means any form of electronic, verbal, or written communication sent\n      to the Licensor or its representatives, including but not limited to\n      communication on electronic mailing lists, source code control systems,\n      and issue tracking systems that are managed by, or on behalf of, the\n      Licensor for the purpose of discussing and improving the Work, but\n      excluding communication that is conspicuously marked or otherwise\n      designated in writing by the copyright owner as \"Not a Contribution.\"\n\n      \"Contributor\" shall mean Licensor and any individual or Legal Entity\n      on behalf of whom a Contribution has been received by Licensor and\n      subsequently incorporated within the Work.\n\n   2. Grant of Copyright License. Subject to the terms and conditions of\n      this License, each Contributor hereby grants to You a perpetual,\n      worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n      copyright license to reproduce, prepare Derivative Works of,\n      publicly display, publicly perform, sublicense, and distribute the\n      Work and such Derivative Works in Source or Object form.\n\n   3. Grant of Patent License. Subject to the terms and conditions of\n      this License, each Contributor hereby grants to You a perpetual,\n      worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n      (except as stated in this section) patent license to make, have made,\n      use, offer to sell, sell, import, and otherwise transfer the Work,\n      where such license applies only to those patent claims licensable\n      by such Contributor that are necessarily infringed by their\n      Contribution(s) alone or by combination of their Contribution(s)\n      with the Work to which such Contribution(s) was submitted. If You\n      institute patent litigation against any entity (including a\n      cross-claim or counterclaim in a lawsuit) alleging that the Work\n      or a Contribution incorporated within the Work constitutes direct\n      or contributory patent infringement, then any patent licenses\n      granted to You under this License for that Work shall terminate\n      as of the date such litigation is filed.\n\n   4. Redistribution. You may reproduce and distribute copies of the\n      Work or Derivative Works thereof in any medium, with or without\n      modifications, and in Source or Object form, provided that You\n      meet the following conditions:\n\n      (a) You must give any other recipients of the Work or\n          Derivative Works a copy of this License; and\n\n      (b) You must cause any modified files to carry prominent notices\n          stating that You changed the files; and\n\n      (c) You must retain, in the Source form of any Derivative Works\n          that You distribute, all copyright, patent, trademark, and\n          attribution notices from the Source form of the Work,\n          excluding those notices that do not pertain to any part of\n          the Derivative Works; and\n\n      (d) If the Work includes a \"NOTICE\" text file as part of its\n          distribution, then any Derivative Works that You distribute must\n          include a readable copy of the attribution notices contained\n          within such NOTICE file, excluding those notices that do not\n          pertain to any part of the Derivative Works, in at least one\n          of the following places: within a NOTICE text file distributed\n          as part of the Derivative Works; within the Source form or\n          documentation, if provided along with the Derivative Works; or,\n          within a display generated by the Derivative Works, if and\n          wherever such third-party notices normally appear. The contents\n          of the NOTICE file are for informational purposes only and\n          do not modify the License. You may add Your own attribution\n          notices within Derivative Works that You distribute, alongside\n          or as an addendum to the NOTICE text from the Work, provided\n          that such additional attribution notices cannot be construed\n          as modifying the License.\n\n      You may add Your own copyright statement to Your modifications and\n      may provide additional or different license terms and conditions\n      for use, reproduction, or distribution of Your modifications, or\n      for any such Derivative Works as a whole, provided Your use,\n      reproduction, and distribution of the Work otherwise complies with\n      the conditions stated in this License.\n\n   5. Submission of Contributions. Unless You explicitly state otherwise,\n      any Contribution intentionally submitted for inclusion in the Work\n      by You to the Licensor shall be under the terms and conditions of\n      this License, without any additional terms or conditions.\n      Notwithstanding the above, nothing herein shall supersede or modify\n      the terms of any separate license agreement you may have executed\n      with Licensor regarding such Contributions.\n\n   6. Trademarks. This License does not grant permission to use the trade\n      names, trademarks, service marks, or product names of the Licensor,\n      except as required for reasonable and customary use in describing the\n      origin of the Work and reproducing the content of the NOTICE file.\n\n   7. Disclaimer of Warranty. Unless required by applicable law or\n      agreed to in writing, Licensor provides the Work (and each\n      Contributor provides its Contributions) on an \"AS IS\" BASIS,\n      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n      implied, including, without limitation, any warranties or conditions\n      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n      PARTICULAR PURPOSE. You are solely responsible for determining the\n      appropriateness of using or redistributing the Work and assume any\n      risks associated with Your exercise of permissions under this License.\n\n   8. Limitation of Liability. In no event and under no legal theory,\n      whether in tort (including negligence), contract, or otherwise,\n      unless required by applicable law (such as deliberate and grossly\n      negligent acts) or agreed to in writing, shall any Contributor be\n      liable to You for damages, including any direct, indirect, special,\n      incidental, or consequential damages of any character arising as a\n      result of this License or out of the use or inability to use the\n      Work (including but not limited to damages for loss of goodwill,\n      work stoppage, computer failure or malfunction, or any and all\n      other commercial damages or losses), even if such Contributor\n      has been advised of the possibility of such damages.\n\n   9. Accepting Warranty or Additional Liability. While redistributing\n      the Work or Derivative Works thereof, You may choose to offer,\n      and charge a fee for, acceptance of support, warranty, indemnity,\n      or other liability obligations and/or rights consistent with this\n      License. However, in accepting such obligations, You may act only\n      on Your own behalf and on Your sole responsibility, not on behalf\n      of any other Contributor, and only if You agree to indemnify,\n      defend, and hold each Contributor harmless for any liability\n      incurred by, or claims asserted against, such Contributor by reason\n      of your accepting any such warranty or additional liability.\n\n   END OF TERMS AND CONDITIONS\n\n   APPENDIX: How to apply the Apache License to your work.\n\n      To apply the Apache License to your work, attach the following\n      boilerplate notice, with the fields enclosed by brackets \"[]\"\n      replaced with your own identifying information. (Don't include\n      the brackets!)  The text should be enclosed in the appropriate\n      comment syntax for the file format. We also recommend that a\n      file or class name and description of purpose be included on the\n      same \"printed page\" as the copyright notice for easier\n      identification within third-party archives.\n\n   Copyright [yyyy] [name of copyright owner]\n\n   Licensed under the Apache License, Version 2.0 (the \"License\");\n   you may not use this file except in compliance with the License.\n   You may obtain a copy of the License at\n\n       http://www.apache.org/licenses/LICENSE-2.0\n\n   Unless required by applicable law or agreed to in writing, software\n   distributed under the License is distributed on an \"AS IS\" BASIS,\n   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n   See the License for the specific language governing permissions and\n   limitations under the License.\n\n\n</details>\n","id":"reproduction-034"},{"cell_type":"markdown","metadata":{},"source":"<details><summary>Full license: DINOv3-LICENSE.md</summary>\n\n# DINOv3 License\n\n*Last Updated: August 19, 2025*\n\n**“Agreement”** means the terms and conditions for use, reproduction, distribution and modification of the DINO Materials set forth herein.\n\n**“DINO Materials”** means, collectively, Documentation and the models, software and algorithms, including machine-learning model code, trained model weights, inference-enabling code, training-enabling code, fine-tuning enabling code, and other elements of the foregoing distributed by Meta and made available under this Agreement.\n\n**“Documentation”** means the specifications, manuals and documentation accompanying\nDINO Materials distributed by Meta.\n\n**“Licensee”** or **“you”** means you, or your employer or any other person or entity (if you are entering into this Agreement on such person or entity’s behalf), of the age required under applicable laws, rules or regulations to provide legal consent and that has legal authority to bind your employer or such other person or entity if you are entering in this Agreement on their behalf.\n\n**“Meta”** or **“we”** means Meta Platforms Ireland Limited (if you are located in or, if you are an entity, your principal place of business is in the EEA or Switzerland) or Meta Platforms, Inc. (if you are located outside of the EEA or Switzerland).\n\n**“Sanctions”** means any economic or trade sanctions or restrictions administered or enforced by the United States (including the Office of Foreign Assets Control of the U.S. Department of the Treasury (“OFAC”), the U.S. Department of State and the U.S. Department of Commerce), the United Nations, the European Union, or the United Kingdom.\n\n**“Trade Controls”** means any of the following: Sanctions and applicable export and import controls.\n\nBy clicking “I Accept” below or by using or distributing any portion or element of the DINO Materials, you agree to be bound by this Agreement.\n\n## 1. License Rights and Redistribution.\n\na. <ins>Grant of Rights</ins>. You are granted a non-exclusive, worldwide, non-transferable and royalty-free limited license under Meta’s intellectual property or other rights owned by Meta embodied in the DINO Materials to use, reproduce, distribute, copy, create derivative works of, and make modifications to the DINO Materials.\n\nb. <ins>Redistribution and Use</ins>.\n\ni. Distribution of DINO Materials, and any derivative works thereof, are subject to the terms of this Agreement. If you distribute or make the DINO Materials, or any derivative works thereof, available to a third party, you may only do so under the terms of this Agreement and you shall provide a copy of this Agreement with any such DINO Materials.\n\nii.  If you submit for publication the results of research you perform on, using, or otherwise in connection with DINO Materials, you must acknowledge the use of DINO Materials in your publication.\n\niii. Your use of the DINO Materials must comply with applicable laws and regulations, including Trade Control Laws and applicable privacy and data protection laws.\n\niv. Your use of the DINO Materials will not involve or encourage others to reverse engineer, decompile or discover the underlying components of the DINO Materials.\n\nv. You are not the target of Trade Controls and your use of DINO Materials must comply with Trade Controls. You agree not to use, or permit others to use, DINO Materials for any activities subject to the International Traffic in Arms Regulations (ITAR) or end uses prohibited by Trade Controls, including those related to military or warfare purposes, nuclear industries or applications, espionage, or the development or use of guns or illegal weapons.\n\n## 2. User Support.\n\nYour use of the DINO Materials is done at your own discretion; Meta does not process any information nor provide any service in relation to such use.  Meta is under no obligation to provide any support services for the DINO Materials. Any support provided is “as is”, “with all faults”, and without warranty of any kind.\n\n## 3. Disclaimer of Warranty.\n\nUNLESS REQUIRED BY APPLICABLE LAW, THE DINO MATERIALS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON AN “AS IS” BASIS, WITHOUT WARRANTIES OF ANY KIND, AND META DISCLAIMS ALL WARRANTIES OF ANY KIND, BOTH EXPRESS AND IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OF USING OR REDISTRIBUTING THE DINO MATERIALS AND ASSUME ANY RISKS ASSOCIATED WITH YOUR USE OF THE DINO MATERIALS AND ANY OUTPUT AND RESULTS.\n\n## 4. Limitation of Liability.\n\nIN NO EVENT WILL META OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, ARISING OUT OF THIS AGREEMENT, FOR ANY LOST PROFITS OR ANY DIRECT OR INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY OR PUNITIVE DAMAGES, EVEN IF META OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.\n\n## 5. Intellectual Property.\n\na. Subject to Meta’s ownership of DINO Materials and derivatives made by or for Meta, with respect to any derivative works and modifications of the DINO Materials that are made by you, as between you and Meta, you are and will be the owner of such derivative works and modifications.\n\nb. If you institute litigation or other proceedings against Meta or any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the DINO Materials, outputs or results, or any portion of any of the foregoing, constitutes infringement of intellectual property or other rights owned or licensable by you, then any licenses granted to you under this Agreement shall terminate as of the date such litigation or claim is filed or instituted. You will indemnify and hold harmless Meta from and against any claim by any third party arising out of or related to your use or distribution of the DINO Materials.\n\n## 6. Term and Termination.\n\nThe term of this Agreement will commence upon your acceptance of this Agreement or access to the DINO Materials and will continue in full force and effect until terminated in accordance with the terms and conditions herein. Meta may terminate this Agreement if you are in breach of any term or condition of this Agreement. Upon termination of this Agreement, you shall delete and cease use of the DINO Materials. Sections 3, 4 and 7 shall survive the termination of this Agreement.\n\n## 7. Governing Law and Jurisdiction.\n\nThis Agreement will be governed and construed under the laws of the State of California without regard to choice of law principles, and the UN Convention on Contracts for the International Sale of Goods does not apply to this Agreement. The courts of California shall have exclusive jurisdiction of any dispute arising out of this Agreement.\n\n## 8. Modifications and Amendments.\n\nMeta may modify this Agreement from time to time; provided that they are similar in spirit to the current version of the Agreement, but may differ in detail to address new problems or concerns. All such changes will be effective immediately. Your continued use of the DINO Materials after any modification to this Agreement constitutes your agreement to such modification. Except as provided in this Agreement, no modification or addition to any provision of this Agreement will be binding unless it is in writing and signed by an authorized representative of both you and Meta.\n\n\n</details>\n","id":"reproduction-035"},{"cell_type":"markdown","metadata":{},"source":"### Pinned attached inputs\n\n- `dreaddevelopment/raptor-knee-maxspan/1`\n- `dreaddevelopment/raptor-knee-native384/1`\n- `dreaddevelopment/raptor-knee-native384dense/1`\n- `mattiaangeli/knee-mri-fold-weights/2`\n- `mattiaangeli/opencv-python-headless-4120088-x86/1`\n- `marwanmath/resnet-50-radimagenet-marwan/1`\n- `mattiaangeli/rsna-knee-coat-resgated-ep10-top3/3`\n- `antoinegg1/rsna-knee-e11-diverse-heads-v20/1`\n- `antoinegg1/rsna-knee-e9-radimagenet-heads-v15/3`\n- `pilkwang/rsna-knee-llm-labels/1`\n- `prvsiyan/rsna-knee-v52-radimagenet-heads-20260812/1`\n- `pilkwang/rsna-knee-weights/1`\n- `sofiaanjenje/rsna-knee-e11-train/2`\n- `sofiaanjenje/rsna-knee-e13-train/1`\n- `metaresearch/dinov2/PyTorch/small/1`\n","id":"reproduction-036"},{"cell_type":"code","metadata":{},"source":"\"\"\"Validate the executed source graph and bind the actual final CSV.\"\"\"\nimport numpy as _rep_np\nimport pandas as _rep_pd\n\n_rep_snapshot('final')\n_rep_work = _RepPath('/kaggle/working')\n_rep_log = ''.join(_rep_messages)\n_rep_failures = _rep_re.findall(\n    r'^.*(?:\\[arm \\d+\\].*FALLBACK|study failed:|degenerate predictions; not banked|'\n    r'public-frontier vote omitted|surrendering \\d+ member|probe FAIL).*$',\n    _rep_log, flags=_rep_re.MULTILINE)\nassert not _rep_failures, _rep_failures\nassert 'rank mean of 20 member(s)' in _rep_log, 'DINO member coverage'\nassert sum(('loaded m_f' + str(i) + '.pt') in _rep_log for i in range(5)) == 5, 'A5 folds'\nassert all(('[arm ' + str(i) + '] done + freed') in _rep_log for i in range(4)), 'Raptor views'\nassert V18_CALIBRATOR_APPLIED is True, 'source calibrator did not run'\nassert _rep_dino_guard_installed and _rep_raptor_guard_installed, 'raw guards'\nassert _rep_dino_raw_members_checked == 20, 'DINO raw coverage'\nassert _rep_rad_guard_installed and _rep_rad_rank_checks > 0, 'Rad raw rank guards'\nassert _rep_raptor_checked_predictions == 4 * _rep_stages['final']['rows'], 'raw Raptor coverage'\nassert _rep_raptor_checked_ranks == 1, 'raw Raptor blend coverage'\n\n_rep_coat = _rep_json.loads((_rep_work / '_coat_arm_receipt.json').read_text())\nassert _rep_coat['status'] == 'VALID_COAT_RESGATED_EP10_TOP3_RANK_SUBMISSION'\nassert _rep_coat['models'] == 3\nassert [c['epoch'] for c in _rep_coat['checkpoints']] == [4, 6, 8]\nassert _rep_coat['fallback_studies'] == 0 and not _rep_coat['failures']\nassert _rep_coat['output_sha256'] == _rep_sha(_rep_work / '_coat_arm.csv')\n_rep_blend = _rep_json.loads((_rep_work / '_coat_raptor_blend_receipt.json').read_text())\nassert _rep_blend['private_alpha'] == 0.4\nassert _rep_blend['finding_specific_weights'] is False\n_rep_rad = _rep_json.loads((_rep_work / 'v50_v2_repro_receipt.json').read_text())\nassert _rep_rad['submission_sha256'] == _rep_stages['rad']['sha256']\n\n# Recompute the final fusion independently from saved intermediate outputs.\n_rep_parent = _rep_pd.read_csv(_rep_work / 'reproduction_rad.csv', dtype={'StudyInstanceUID': str})\n_rep_hybrid = _rep_pd.read_csv(_rep_work / '_raptor.csv', dtype={'StudyInstanceUID': str})\n_rep_final = _rep_pd.read_csv(_rep_work / 'submission.csv', dtype={'StudyInstanceUID': str})\n_rep_labels = list(_rep_final.columns[1:])\nassert _rep_hybrid.StudyInstanceUID.tolist() == _rep_final.StudyInstanceUID.tolist()\n_rep_parent_rank = _rep_parent[_rep_labels].rank(method='average', pct=True)\n_rep_rebuilt = _rep_parent_rank.copy()\nfor _rep_label in _rep_labels:\n    _rep_weight = {'ACL': 0.75, 'Medial Meniscus': 0.8, 'Lateral Meniscus': 1.0, 'Lateral OA': 0.75, 'Fracture': 0.75}.get(_rep_label, 0.6)\n    _rep_rebuilt[_rep_label] = ((1.0 - _rep_weight) * _rep_parent_rank[_rep_label]\n                              + _rep_weight * _rep_hybrid[_rep_label])\n_rep_rebuilt = _rep_rebuilt.rank(method='average', pct=True)\nassert _rep_np.allclose(_rep_final[_rep_labels], _rep_rebuilt, rtol=0, atol=1e-15)\nassert _rep_coat['studies'] == len(_rep_final)\nassert _rep_rad['test_studies'] == len(_rep_final)\nassert _rep_blend['study_count'] == len(_rep_final)\n\n_rep_receipt = {\n    'contract': 'exact_public_mattia_v27_reproduction_v1',\n    'status': 'PASS',\n    'source_notebook_sha256': _rep_source_sha256,\n    'source_version': 27,\n    'source_script_version_id': 348195103,\n    'prediction_source_modified': False,\n    'inference_only': True,\n    'new_gold_evaluation': False,\n    'gpu_names': _rep_gpu_names,\n    'rows': len(_rep_final),\n    'columns': list(_rep_final.columns),\n    'uid_sha256': _rep_hashlib.sha256(('\\n'.join(_rep_final.StudyInstanceUID) + '\\n').encode()).hexdigest(),\n    'stages': _rep_stages,\n    'final_submission_sha256': _rep_sha(_rep_work / 'submission.csv'),\n    'coat_receipt_sha256': _rep_sha(_rep_work / '_coat_arm_receipt.json'),\n    'rad_receipt_sha256': _rep_sha(_rep_work / 'v50_v2_repro_receipt.json'),\n    'final_fusion_reconstruction_passed': True,\n    'execution_fallbacks': _rep_failures,\n    'raptor_raw_predictions_checked': _rep_raptor_checked_predictions,\n    'dino_raw_members_checked': _rep_dino_raw_members_checked,\n    'rad_raw_rank_checks': _rep_rad_rank_checks,\n    'elapsed_seconds': _rep_time.time() - _rep_started,\n    'score': None,\n    'score_note': 'Only a completed competition submissions API row establishes our score.',\n}\n(_rep_work / 'reproduction_receipt.json').write_text(_rep_json.dumps(_rep_receipt, indent=2, sort_keys=True) + '\\n')\nprint('[reproduction] PASS', _rep_receipt['final_submission_sha256'], flush=True)\n","execution_count":null,"outputs":[],"id":"reproduction-037"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.11.13"}},"nbformat":4,"nbformat_minor":5}