{"cells":[{"cell_type":"markdown","id":"v41-overview","metadata":{},"source":"# V41: DINOv3 + E10 alpha 0.60\n\nThis version changes only the precommitted E10 configuration from uniform 0.35 to uniform 0.60. Grouped nested selection chose 0.60 in all five public outer folds; an independent OOF source chose 0.70 in all five. No visible-test tuning or identifier is used.\n"},{"cell_type":"markdown","id":"v40-overview","metadata":{},"source":"# V40: DINOv3 cross-series + E10 RadImageNet hybrid\n\nThis hidden-test-safe candidate retains Mattia Angeli's cross-series DINOv3 member, removes its older Rad15 stage, and applies fishface's hash-pinned E10 uniform-0.35 RadImageNet correction to the resulting dynamic DINOv2+DINOv3 parent. No visible test identifier or prediction is embedded, and no test-time weight selection is performed.\n"},{"cell_type":"markdown","id":"9eb86ac2","metadata":{},"source":"# Bend the Knee Ensemble\n\nA community pipeline, with one more model added.\n\n## Thank you\n\nThis is built on work other people did first and shared:\n\n- **pilkwang** — twenty trained models that are part of this ensemble, and a\n  set of labels read from the reports.\n- **stevenleehans** and **lixin73** — two more sets of labels read from the\n  reports, so one reading could be checked against another.\n- **tonylica** — four more trained models that join the ensemble.\n- **marwanmath** — the official RadImageNet ResNet-50 weights.\n- **prvsiyan** — the notebook this was forked from (Apache 2.0). Most of its\n  later stages have been removed here.\n- **cf696666** — for leaving two findings out of the RadImageNet blend, which\n  is done here too."},{"cell_type":"markdown","id":"013f5cdb","metadata":{},"source":"### The added member\n\nThe submission this forks from is a rank mean of twenty **DINOv2** models. Added\nhere is one **DINOv3 ViT-S/16** — self-supervised and pretrained without labels,\nthen fine-tuned on this competition's knee MRI — rank-blended into that\nfamily.\n\nAbove the encoder, each series carries a learned type embedding — plane crossed\nwith fat suppression — added to all of its tokens. The tokens of every series in\na study are then concatenated into one key/value sequence, and twelve learned\nqueries, one per finding, cross-attend over it with multi-head attention. A\nfinding therefore draws evidence from any series in the study at once, rather\nthan from per-series summaries combined afterwards. Each query's output is\nconcatenated with the mean and the max of the per-series CLS embeddings before\nthe classifier. Series a study does not contain are removed by the key-padding\nmask, so nothing is imputed for them.\n\n**RadImageNet** joins the vote at the end — a ResNet-50 pretrained on\nradiology images rather than natural ones. Its heads were retrained here\nrather than used as published.\n\nIts five folds are combined on ranks rather than by averaging probabilities, matching\nhow the other two families are already combined — the metric is macro ROC-AUC, so only\nordering matters. That correction is due to **romantamrazov** (*RSNA Knee | DINOsaur V2*)."},{"cell_type":"code","execution_count":null,"id":"c8cdcc5c","metadata":{},"outputs":[],"source":"from __future__ import annotations\nimport re\nimport unicodedata\nTARGETS = ['ACL', 'MCL', 'Medial Meniscus', 'Lateral Meniscus', 'Medial OA', 'Lateral OA', 'PF OA', 'Effusion', 'Synovitis', \"Baker's\", 'Contusion', 'Fracture']\n_PRE = str.maketrans({'ı': 'i', 'İ': 'i', 'I': 'i', 'ß': 'ss', 'đ': 'd', 'Đ': 'd', 'ø': 'o', 'Ø': 'o', 'æ': 'ae', 'Æ': 'ae'})\n\ndef normalize(text: str) -> str:\n    if not isinstance(text, str):\n        return ''\n    text = text.translate(_PRE).lower()\n    text = unicodedata.normalize('NFKD', text)\n    text = ''.join((ch for ch in text if not unicodedata.combining(ch)))\n    text = text.replace('\\xad', '')\n    text = re.sub('[_\\\\-/\\\\\\\\]+', ' ', text)\n    text = re.sub('[ \\\\t]+', ' ', text)\n    return text\n_SENT_SPLIT = re.compile('(?<=[.;!?])\\\\s+|\\\\n+')\n\ndef unwrap(text: str) -> str:\n    if not isinstance(text, str):\n        return ''\n    out = []\n    for line in text.split('\\n'):\n        s = line.strip()\n        if out and out[-1] and (not re.search('[.;:!?>*•]$', out[-1])) and (len(out[-1].split()) >= 4) and s and (not s[:1].isupper()):\n            out[-1] = out[-1] + ' ' + s\n        else:\n            out.append(s)\n    return '\\n'.join(out)\n\ndef clauses(text: str):\n    norm = normalize(unwrap(text) if FEATURES['unwrap'] else text)\n    raw = [c.strip() for c in _SENT_SPLIT.split(norm) if c and c.strip()]\n    merged = []\n    for i, c in enumerate(raw):\n        if c.endswith(':') and len(c.split()) <= 14 and (i + 1 < len(raw)):\n            merged.append(c + ' ' + raw[i + 1])\n        merged.append(c)\n    out = []\n    for c in merged:\n        out.append(c)\n        if len(c.split()) > 25:\n            out.extend((p.strip() for p in c.split(',') if len(p.split()) > 2))\n    return out\nFEATURES = {'unwrap': True, 'directional_negation': True, 'oa_inherit': True, 'graded_pathology': True, 'synovitis_backoff': True}\n\ndef _rx(*alts: str) -> re.Pattern:\n    return re.compile('|'.join(alts))\nPRE_NEG = _rx('\\\\bno\\\\b', '\\\\bnot\\\\b', '\\\\bwithout\\\\b', '\\\\bnegative for\\\\b', '\\\\babsence\\\\b', '\\\\bno evidence\\\\b', '\\\\bfree of\\\\b', '\\\\bnone\\\\b', '\\\\bneither\\\\b', '\\\\bnor\\\\b', '\\\\bsin\\\\b', '\\\\bno hay\\\\b', '\\\\bausencia\\\\b', '\\\\bausentes?\\\\b', '\\\\bno se\\\\b', '\\\\bpas de\\\\b', '\\\\bsans\\\\b', '\\\\baucune?\\\\b', '\\\\bgeen\\\\b', '\\\\bzonder\\\\b', '\\\\bniet\\\\b', '\\\\bkeine?[nmrs]?\\\\b', '\\\\bohne\\\\b', '\\\\bnicht\\\\b', '\\\\bkein\\\\b', '\\\\bnema\\\\b', '\\\\bbez\\\\b', '\\\\bnisu\\\\b', '\\\\bnije\\\\b', '\\\\bδεν\\\\b', '\\\\bχωρις\\\\b', 'ουδεν', '\\\\bουτε\\\\b', '\\\\bбез\\\\b', '\\\\bне\\\\b', 'липсва', '\\\\bняма\\\\b')\nPOST_NEG = _rx('\\\\byok\\\\b', '\\\\byoktur\\\\b', 'izlenmemekte', 'saptanmadi', '\\\\bdegil\\\\b', 'gozlenmemekte', 'mevcut degil', 'eslik etmiyor', '\\\\bizlenmedi\\\\b', 'izlenmemistir', 'saptanmamistir', 'gorulmemistir', '\\\\bnema znakova\\\\b', 'bez znakova')\nNEGATION = _rx(PRE_NEG.pattern, POST_NEG.pattern, '\\\\bunremarkable\\\\b')\nNEG_WINDOW = 90\n\ndef _negated(clause: str, start: int, end: int) -> bool:\n    for m in PRE_NEG.finditer(clause):\n        if m.end() <= start and start - m.end() <= NEG_WINDOW:\n            if not re.search('\\\\b(but|however|ancak|fakat|pero|maar|aber|no i|ali|ωστοσο|αλλα|но)\\\\b', clause[m.end():start]):\n                return True\n    for m in POST_NEG.finditer(clause):\n        if m.start() >= end and m.start() - end <= NEG_WINDOW:\n            return True\n    return False\nNORMALITY = _rx('\\\\bnormal', '\\\\bintact\\\\b', '\\\\bpreserved\\\\b', '\\\\bwithin normal limits\\\\b', 'limites normales', '\\\\bconservad', '\\\\bintegr', '\\\\bnormales\\\\b', '\\\\bdoga(l|ll)\\\\b', 'korunmus', '\\\\bnormaldir\\\\b', 'olagan', '\\\\buredn', '\\\\bocuvan', '\\\\bodrzan', '\\\\bintakt', '\\\\bprimjeren', '\\\\bodrzanog kontinuiteta', '\\\\bodržan', 'φυσιολογικ', 'ακεραι', 'δεν παρατηρουνται', 'δεν σημειωνονται', 'unauffallig', 'regelrecht', '\\\\bo\\\\.?b\\\\.?\\\\b', 'нормал', 'запазен', 'съхранен', '\\\\bбез особености\\\\b', 'интактн', '\\\\bgaaf\\\\b', '\\\\bnormaal\\\\b')\nNORMAL_PHRASE = _rx('\\\\bsin alteracion', '\\\\bsin cambios\\\\b', '\\\\bsin particularidad', '\\\\bsin hallazgos\\\\b', '\\\\bsin lesion', '\\\\bsin signos de (rotura|lesion)', '\\\\bcontinu[oa]s?\\\\b', '\\\\bcontinuidad conservada\\\\b', '\\\\bno abnormalit', '\\\\bno significant abnormalit', '\\\\bunremarkable\\\\b', '\\\\bno evidence of (tear|injury|abnormalit)', '\\\\bohne auffalligkeit', '\\\\bkein nachweis\\\\b', '\\\\bohne befund\\\\b', '\\\\bgeen afwijking', '\\\\bzonder afwijking', '\\\\bsans anomalie', \"\\\\bpas d[e']anomalie\", '\\\\bbez osobitosti\\\\b', '\\\\bbez znakova (rupture|lezije)\\\\b', '\\\\bbez patoloskih\\\\b', 'χωρις αλλοιωσ', 'χωρις παθολογ', 'δεν παρατηρουνται (αξιολογα|παθολογ)', '\\\\bбез особености\\\\b', '\\\\bбез патологич', '\\\\bбез данни за\\\\b', '\\\\bozel bir ozellik yok', '\\\\bpatolojik bulgu (yok|izlenmemis)')\nUNCERTAIN = _rx('\\\\bpossible\\\\b', '\\\\bprobable\\\\b', '\\\\bsuspicious\\\\b', '\\\\bsuspected?\\\\b', 'cannot (be )?exclude', '\\\\bmay\\\\b', '\\\\bquestionable\\\\b', '\\\\bequivocal\\\\b', '\\\\br/o\\\\b', '\\\\bdd\\\\b', '\\\\blikely\\\\b', '\\\\bsuggest', '\\\\bcompatible with\\\\b', '\\\\bposible\\\\b', 'sin criterios categoricos', '\\\\bdudos', '\\\\bsugier', '\\\\bmuhtemel\\\\b', '\\\\bolasi\\\\b', '\\\\bsupheli\\\\b', '\\\\bizlenim', '\\\\bdusundur', '\\\\bmoguce\\\\b', '\\\\bvjerojatno\\\\b', '\\\\bsumnja\\\\b', '\\\\bmoze odgovarati\\\\b', 'πιθαν', 'υποπτ', '\\\\bmoglich', '\\\\bverdachtig', '\\\\bfraglich', '\\\\bv\\\\.?a\\\\.?\\\\b', '\\\\bwohl\\\\b', '\\\\bвъзможно\\\\b', '\\\\bвероятно\\\\b', 'суспект', '\\\\bmogelijk\\\\b', '\\\\bverdacht\\\\b')"},{"cell_type":"code","execution_count":null,"id":"3f3e4239","metadata":{},"outputs":[],"source":"TEAR = _rx('\\\\btear', '\\\\btorn\\\\b', '\\\\brupture', '\\\\bdisruption\\\\b', 'discontinuit', '\\\\bavuls', '\\\\bmacerat', '\\\\bbuckethandle\\\\b', 'bucket handle', '\\\\brotura\\\\b', '\\\\broturas\\\\b', '\\\\bruptura', '\\\\bdesgarro', '\\\\broto\\\\b', '\\\\bdechirure', '\\\\bdechire', '\\\\bscheur', '\\\\bruptuur', 'gescheurd', '\\\\briss\\\\b', 'einriss', '\\\\bruptur', 'zerreiss', '\\\\blasion', '\\\\bausriss', '\\\\byirtik', '\\\\byirtig', '\\\\bkopma\\\\b', 'butunluk kaybi', '\\\\brupturu\\\\b', 'devamsizlik', '\\\\brupture\\\\b', '\\\\bdevamliligi secilememis', '\\\\bpuknuce', '\\\\bprekid\\\\b', '\\\\bpukotin', '\\\\bruptur', 'ρηξη', 'ρηξις', 'ρηγμα', 'ασυνεχεια', 'руптура', 'разкъсв', 'разрив', 'скъсв', '\\\\bлезия\\\\b')\nDEGEN = _rx('degenerat', '\\\\bmucoid\\\\b', '\\\\bmyxoid\\\\b', '\\\\bfray', '\\\\bfissur', 'dejeneratif', '\\\\bmukoid\\\\b', 'degenerativn', 'εκφυλ', 'дегенерат', '\\\\bμυξοειδ', '\\\\bμυξωδ', '\\\\bmeniskopat', '\\\\bmeniscopath', '\\\\bmuco ?ide\\\\b', 'aufgefasert', '\\\\bdejenerasyon\\\\b')\nINJURY = _rx('\\\\binjur', '\\\\bsprain', '\\\\blesion', '\\\\blasion', '\\\\bedema\\\\b', '\\\\boedema\\\\b', '\\\\bodem\\\\b', '\\\\bedem\\\\b', '\\\\bοιδημα', '\\\\bодем', '\\\\bедем', '\\\\bstrain\\\\b', '\\\\bhigh signal\\\\b', '\\\\bsignal alteration\\\\b', '\\\\bhiperintens', '\\\\bhyperintens', 'aumento de senal', 'alteracion de senal', 'cambio de senal', '\\\\bsignalanhebung', '\\\\bsignalalteration', 'verhoogd signaal', 'sinyal artis', 'αυξημενο σημα', 'повишен сигнал', '\\\\besguince\\\\b', '\\\\bthicken', '\\\\bzadebljanje\\\\b', '\\\\bverdikking\\\\b', '\\\\bdistenzij', '\\\\blaksite\\\\b', '\\\\blaxity\\\\b', '\\\\bpartial\\\\b', '\\\\bparcijaln', '\\\\bparcial', '\\\\bpartiel', '\\\\bpartiell')\n_GRADE_RX = re.compile('(?:grade|grad|grado|grau|derece|stupnja|stupanj|βαθμ|степен|icrs|outerbridge)[\\\\s:]*(?:grade\\\\s*)?([1-4]|iv|iii|ii|i)\\\\b')\n_ROMAN = {'i': 1, 'ii': 2, 'iii': 3, 'iv': 4}\n\ndef _grade_of(clause: str):\n    best = None\n    for m in _GRADE_RX.finditer(clause):\n        v = m.group(1)\n        n = _ROMAN.get(v, None) if not v.isdigit() else int(v)\n        if n is not None and (best is None or n > best):\n            best = n\n    return best\nANAT = {'ACL': _rx('anterior cruciate', '\\\\bacl\\\\b', 'cruzado anterior', '\\\\blca\\\\b', 'croise anterieur', 'voorste kruisband', '\\\\bvkb\\\\b', 'vorderes kreuzband', 'vorderen kreuzband', 'vordere kreuzband', 'on capraz', '\\\\bocb\\\\b', 'anterior capraz', 'prednji krizni', 'prednjeg krizn', 'προσθι[οα][^ ]* χιαστ', 'προσθιου χιαστου', 'χιαστο[^ ]* συνδεσμ', '\\\\bχιαστ\\\\w*', 'предна кръстна', 'предната кръстна', 'предна кръста', 'cruciate ligaments', 'ligamentos cruzados', 'ligaments croises', 'kruisbanden', 'kreuzbander', 'capraz baglar', 'krizn[a-z]* ligament[a-z]*', 'χιαστοι συνδεσμ', 'χιαστων συνδεσμ', 'кръстните връзки', 'кръстни връзки'), 'MCL': _rx('medial collateral', '\\\\bmcl\\\\b', 'tibial collateral', 'colateral medial', 'colateral interno', '\\\\blcm\\\\b', 'collateral medial', 'collateral interne', 'mediale collaterale', 'binnenband', '\\\\b(mediale|laterale) banden\\\\b', '\\\\bcollaterale banden\\\\b', 'innenband', 'mediales? kollateral', '\\\\bic yan bag', 'medial kollateral', '\\\\biyb\\\\b', 'medyal kollateral', 'medijalni kolateraln', 'medijalnog kolateraln', 'εσω πλαγι', 'εσωτερικο πλαγι', '\\\\bπλαγι\\\\w* συνδεσμ', '\\\\bπλαγιοι\\\\b', 'медиален колатерал', 'вътрешна странична', '\\\\bколатерал\\\\w*', '\\\\bcolaterales\\\\b', '\\\\bcollateraux\\\\b', '\\\\bcollateralen\\\\b', '\\\\bkolateralni\\\\b', 'collateral ligaments', 'ligamentos colaterales', 'ligaments collateraux', 'collaterale banden', 'kollateralbander', 'seitenbander', 'yan baglar', 'kolateraln[a-z]* ligament[a-z]*', 'πλαγιοι συνδεσμ', 'πλαγιων συνδεσμ', 'колатерални връзки', 'страничните връзки'), 'Medial Meniscus': _rx('medial meniscus', '\\\\bmm\\\\b(?= tear)', 'medial menisc', 'menisco medial', 'menisco interno', 'menisque medial', 'menisque interne', 'mediale meniscus', 'binnenmeniscus', 'innenmeniskus', 'medialen? meniskus', 'innenmeniskushinterhorn', 'medyal menisk', '\\\\bic menisk', 'medijalni meniskus', 'medijalnog meniskusa', 'medijalnom meniskusu', 'medijaln\\\\w* menisk\\\\w*', '\\\\bmedijalnog meniska\\\\b', 'medijalni menisk', 'εσω μηνισκ', 'μηνισκ[^ ]* του εσω', 'εσω διαμερισμα[^.]{0,40}μηνισκ', 'медиалния менискус', 'медиален менискус', 'вътрешния менискус', 'oba meniska', 'both menisci', 'ambos meniscos', 'beide menisci', 'her iki menisku', 'amfoteroi\\\\w* mhnisk', 'αμφοτερ\\\\w* μηνισκ', 'двата менискуса', 'medial (and|&) lateral menisc'), 'Lateral Meniscus': _rx('lateral meniscus', 'lateral menisc', 'menisco lateral', 'menisco externo', 'menisque lateral', 'menisque externe', 'laterale meniscus', 'buitenmeniscus', 'aussenmeniskus', 'lateralen? meniskus', 'aussenmeniskushinterhorn', 'lateral menisk', '\\\\bdis menisk', 'lateralni meniskus', 'lateralnog meniskusa', 'lateralnom meniskusu', 'lateraln\\\\w* menisk\\\\w*', '\\\\blateralnog meniska\\\\b', 'εξω μηνισκ', 'μηνισκ[^ ]* του εξω', 'εξω διαμερισμα[^.]{0,40}μηνισκ', 'латералния менискус', 'латерален менискус', 'външния менискус', 'oba meniska', 'both menisci', 'ambos meniscos', 'beide menisci', 'her iki menisku', 'αμφοτερ\\\\w* μηνισκ', 'двата менискуса', 'medial (and|&) lateral menisc')}\nOA_EVIDENCE = _rx('osteoarthrit', '\\\\barthros', '\\\\bgonarthros', '\\\\bosteoarthros', 'chondropath', 'chondromalac', 'condropat', 'condromalac', '\\\\bchondros', '\\\\bchondrosis\\\\b', 'chondral (loss|defect|ulcer|thinning|injury|fissur|wear)', 'cartilage (loss|thinning|defect|fissur|wear|damage|heterogeneity|irregularit)', '(loss|thinning|fissur|defect|ulcer|erosion|denudation) of[^.]{0,20}cartilage', 'articular cartilage[^.]{0,30}(loss|thin|fissur|defect|erosion|wear|irregular)', 'osteophyt', 'osteofit', 'osteofyt', 'osteofito', 'osteophyten', 'spurring', 'joint space narrowing', 'pinzamiento articular', 'reduced joint space', 'kikirdak kayb', 'kikirdak incelme', 'kondropati', 'kondral', 'kikirdak dejener', 'eklem aralig\\\\w* daral', 'eklem mesafesi daral', 'kikirdak kalinlig\\\\w* azal', 'kraakbeen', 'gonartrose', 'artrose', '\\\\bknorpel', 'arthrose', 'gonarthrose', 'hrskavic', 'hondromalac', 'artroz', 'osteoartrit', 'artrotsk', 'artrotick', '\\\\boa promjen', '\\\\boa\\\\b', 'degenerativne promjene hrskav', 'χονδρ[^ ]*παθ', 'αρθριτ', 'αρθρωσ', 'οστεοφυτ', 'χονδρομαλακ', 'αρθρικου χονδρου', 'εξαλειψη του αρθρικου χονδρου', 'διαβρωση του αρθρικου χονδρ', 'λεπτυνση[^.]{0,30}χονδρ', 'φθορα[^.]{0,20}χονδρ', 'артроз', 'хондропат', 'остеофит', 'хрущял[^.]{0,40}(изтън|увред|дефект|липс)', 'изтъняване[^.]{0,30}хрущял', 'хондромалац', 'ulcera[s]? condral', 'cartilago[^.]{0,25}(perdida|adelgaz)', 'icrs grade', 'icrs\\\\b', 'outerbridge', '\\\\bdenudation\\\\b', 'denudacij', 'erozivne promjene', '\\\\berosion of[^.]{0,20}cartilage', 'kraakbeenlijden', 'kraakbeenverlies')\nTF_SITE = _rx('compartment', 'compartimento', 'compartiment', 'kompartman', 'kompartiment', 'kompartment', 'odjelj', 'διαμερισμα', 'компартм', '\\\\bотдел', 'femorotibial', 'tibiofemoral', 'femoro tibial', 'femorotibiaal', 'femorotibijaln', 'феморотибиал', '\\\\bft zglob', 'tibiofemoraln', 'condyle', 'condilo', 'kondyl', 'kondil', 'condyl', 'κονδυλ', 'кондил', '\\\\bplateau', '\\\\bplato\\\\b', 'platillo', 'meseta', 'плато', 'tibiaplateau', 'tibijaln\\\\w* plato', 'tibyal plato', 'tibia plato', 'κνημιαι', 'μηριαι', 'weightbearing', 'weightbaring', 'zona de carga', 'dragende deel', 'agirlik tasiyan', '\\\\bfemur\\\\b', '\\\\btibia\\\\b', '\\\\bfemoral\\\\b', '\\\\btibial\\\\b', '\\\\bfemura\\\\b', '\\\\btibije\\\\b', '\\\\bmesarthrio\\\\b', 'μεσαρθριο')\nPF_SITE = _rx('patellofemoral', 'femoropatellar', 'femoropatelar', 'patelofemoral', 'retropatellar', 'retrorotulian', 'trochlea', 'troclea', 'troklea', 'trochlear', 'trohlej', 'τροχιλ', '\\\\bpatella', '\\\\bpatellar', 'rotulian', '\\\\brotula\\\\b', '\\\\bpatele\\\\b', 'patellofemoraal', 'femoropatellair', 'επιγονατιδ', 'μηροεπιγονατιδ', 'пател', 'феморопател', 'anterior compartment', 'compartimento anterior', 'prednj\\\\w* odjeljk', '\\\\bfp zglob', '\\\\bpf zglob', '\\\\bfaset', '\\\\bfacet', 'patellofemoraln')\nSIDE_MEDIAL = _rx('\\\\bmedial\\\\w*', '\\\\bmedyal\\\\w*', '\\\\bmedijaln\\\\w*', '\\\\bmediaal\\\\w*', '\\\\bmediale\\\\w*', '\\\\binterno\\\\b', '\\\\binterna\\\\b', '\\\\binternos\\\\b', '\\\\binterne\\\\b', '\\\\binnen\\\\w*', '\\\\bic\\\\b', '\\\\bunutarnj\\\\w*', '\\\\bεσω\\\\w*', '\\\\bεσωτερικ\\\\w*', '\\\\bмедиал\\\\w*', '\\\\bвътреш\\\\w*', '\\\\bbinnen\\\\w*', '\\\\bmediaal\\\\b', '\\\\bmediales?\\\\b')\nSIDE_LATERAL = _rx('\\\\blateral\\\\w*', '\\\\bexterno\\\\b', '\\\\bexterna\\\\b', '\\\\bexternos\\\\b', '\\\\bexterne\\\\b', '\\\\bdis\\\\b', '\\\\blateraln\\\\w*', '\\\\baussen\\\\w*', '\\\\bbuiten\\\\w*', '\\\\bεξω\\\\w*', '\\\\bεξωτερικ\\\\w*', '\\\\bлатерал\\\\w*', '\\\\bвъншн\\\\w*', '\\\\bvanjsk\\\\w*')\nSIDE_ANTERIOR = _rx('\\\\banterior\\\\w*', '\\\\bant\\\\b', '\\\\bon\\\\b', '\\\\bprednj\\\\w*', '\\\\bvorder\\\\w*', '\\\\bvoorste\\\\b', '\\\\bπροσθι\\\\w*', '\\\\bпредн\\\\w*', '\\\\banteriyor\\\\w*', '\\\\bavant\\\\b', '\\\\banterieur\\\\w*')\nGLOBAL_OA = _rx('tri ?compartment', 'all three compartment', 'global(ised)? (oa|osteoarthrit)', '\\\\bgonarthros', '\\\\bgonartros', '\\\\bgonarthrose', '\\\\bgonartrose', 'gonartro', 'goanrtrot', 'gonartrot', 'osteoarthritis of the knee', 'artrosis (de |)(la )?rodilla', 'knee osteoarthrit', '\\\\bdiz osteoartrit', '\\\\bgonartroz', 'artroza koljena', 'οστεοαρθριτιδα', 'αρθριτιδα του γονατος', 'εκφυλιστικη οστεοαρθριτ', 'артроза на колянната', 'гонартроз', 'degenerative joint disease', '\\\\bdjd\\\\b', 'three compartments', 'compartmens', 'compartments')\nDIRECT = {'Effusion': _rx('\\\\beffusion', 'joint fluid', 'intra ?articular fluid', '\\\\bhydrops\\\\b', '\\\\bhemarthros', '\\\\bhaemarthros', 'derrame articular', '\\\\bderrame\\\\b', 'liquido articular', 'hemartrosis', 'epanchement', 'gewrichtsvocht', '\\\\bvocht\\\\b', 'gewrichtseffusie', 'opzetting van suprapatell', 'gelenkerguss', '\\\\berguss\\\\b', 'gelenksergu', 'gelenksflussigkeit', 'eklem\\\\w* ic\\\\w* sivi', 'efuzyon', 'eklem sivisi', 'eklem mesafesinde sivi', 'sivi (miktari|artisi|birikimi)', 'sivi artis', '\\\\bsivi\\\\b[^.]{0,25}artmis', '\\\\bizljev', '\\\\bizliv', 'zglobn[^ ]* tekucin', '\\\\bhidrops\\\\b', 'αρθρικ[^ ]* υγρ', 'υγρου ενδαρθρικα', 'ενδαρθρικ[^ ]* υγρ', 'ποσοτητα υγρου', 'ενδαρθρικ', 'αρθρικη συλλογη', 'υγρο στην αρθρωση', 'υγρου στην αρθρωση', 'συλλογη υγρου', 'ενθαρθρικ', 'ставен излив', 'излив', 'ставна течност', 'синовиална течност'), 'Synovitis': _rx('synovit', 'sinovit', 'synovial (thickening|proliferation|hypertroph)', 'thicken\\\\w* synovial', 'hypertroph\\\\w* of the synovium', 'synoviale? (verdikking|proliferatie)', 'verdikkingen van (het )?synovium', 'synovialitis', 'synovialis(verdickung|proliferation)', 'reizsynovial', 'sinovijalitis', 'sinovitis', 'zadebljanje sinovij', 'proliferacij\\\\w* sinovij', 'sinovijaln\\\\w* proliferacij', 'υμενιτιδα', 'συνοβιτιδα', 'υμενικ[^ ]* υπερτροφ', 'αρθρικου υμεν', 'παχυνση[^.]{0,20}υμεν', 'υμενα', 'синовит', 'синовиал[^ ]* (задебел|пролифер)', '\\\\bpannus\\\\b', '\\\\bhoffit', 'sinovyal\\\\w* (kalinlas|proliferas)', 'sinovyal hipertrof', '\\\\bartrit\\\\b', '\\\\barthritis\\\\b'), \"Baker's\": _rx('baker', 'popliteal cyst', 'quiste popliteo', 'quistes popliteos', 'kyste poplite', 'popliteale? cyst', 'poplitealzyste', 'bakerzyste', 'popliteal kist', '\\\\bbakerova\\\\b', 'poplitealn[^ ]* cist', 'popliteal\\\\w* cist', 'κυστη baker', 'πολυχωρη συνοβιακη κυστη', 'κυστη του baker', 'συνοβιακη κυστη', 'κυστη τυπου baker', 'киста на бейкър', 'бейкърова киста', 'поплитеална киста', 'бекеров', 'gastrocnemio ?semimembranos', 'gastrocnemius semimembranosus burs'), 'Contusion': _rx('\\\\bcontusion', 'bone bruise', 'bone marrow (o?edema|contusion)', 'marrow o?edema', '\\\\bkontuz', 'medular bone o?edema', 'osseous contusion', 'contusion osea', 'edema oseo', 'edema de medula osea', 'contusiones oseas', 'oedeme osseux', 'contusion osseuse', 'botcontusie', 'botoedeem', 'beenmergoedeem', 'botmergoedeem', 'knochenmarkodem', 'knochenodem', 'knochenmarksodem', 'kontusion', 'kemik kontuzyonu', 'kemik iligi odemi', 'kemik odemi', 'kemik iliginde odem', 'kontuzyonel kemik', 'kemik iligi odemleri', 'kostani edem', 'edem kosti', 'kontuzij', 'kostane srzi[^.]{0,20}edem', 'οστεομυελικ[^ ]* οιδημα', 'οστικο οιδημα', 'μυελικο οιδημα', 'οστικο μωλωπ', 'костномозъчен едем', 'костен едем', 'контузионен', 'костно мозъчен едем'), 'Fracture': _rx('\\\\bfractur', '\\\\bfract\\\\b', '\\\\bfractura', '\\\\bfracturas\\\\b', '\\\\bfractuur', '\\\\bbreuk\\\\b', '\\\\bfraktur', '\\\\bbruch\\\\b', '\\\\bkirik\\\\b', '\\\\bkirigi\\\\b', '\\\\bkiri[kg]\\\\w*', '\\\\bprijelom', 'impresijsk[^ ]* fraktur', 'impaktcij', 'καταγμα', 'καταγματ', 'фрактур', 'счупван', 'фисур', 'insufficiency fracture', 'stress fracture', 'avulsion fracture', 'subchondral fracture', 'subkondral kiri', 'impaction (fracture|injury)', 'osteochondral (fracture|impaction)', '\\\\bsegond\\\\b', 'impactiefractuur', 'subchondrale impression', 'subchondraler? impress')}\nDECOY = {'Fracture': _rx('microfractur', '\\\\bfracture (risk|prophyla)'), \"Baker's\": _rx('meniscal cyst', 'quiste meniscal', 'parameniscal')}\nPAIRED = {'ACL', 'MCL', 'Medial Meniscus', 'Lateral Meniscus'}\nOA_TARGETS = ['Medial OA', 'Lateral OA', 'PF OA']\nPLURAL_MENISCI = _rx('\\\\bmenisci\\\\b', '\\\\bmeniscos\\\\b', '\\\\bmenisques\\\\b', '\\\\bmenisken\\\\b', '\\\\bmeniskusi\\\\b', '\\\\bmenisk\\\\w*ler\\\\b', '\\\\bμηνισκοι\\\\b', '\\\\bμηνισκων\\\\b', '\\\\bменискуси\\\\b', '\\\\bменискусите\\\\b', '\\\\bmenisci\\\\w*\\\\b')\nANY_SIDE = _rx(SIDE_MEDIAL.pattern, SIDE_LATERAL.pattern)\nSTEM_MENISCUS = _rx('menisc\\\\w*', 'menisk\\\\w*', 'μηνισκ\\\\w*', 'мениск\\\\w*')\nSTEM_CRUCIATE = _rx('cruciate', 'cruzado', 'croise', 'kruisband', 'kreuzband', 'capraz bag\\\\w*', 'krizn\\\\w*', 'χιαστ\\\\w*', 'кръстн\\\\w*', '\\\\bacl\\\\b', '\\\\blca\\\\b', '\\\\bvkb\\\\b', '\\\\bocb\\\\b', '\\\\bacb\\\\b')\nSTEM_COLLATERAL = _rx('collateral\\\\w*', 'colateral\\\\w*', 'kollateral\\\\w*', 'collaterale\\\\w*', 'kolateraln\\\\w*', 'yan bag\\\\w*', 'πλαγι\\\\w*', 'колатерал\\\\w*', 'странич\\\\w*', 'innenband\\\\w*', 'binnenband\\\\w*', '\\\\bmcl\\\\b', '\\\\blcm\\\\b', '\\\\biyb\\\\b')\nSTEM_FRACTURE = _rx('fractur\\\\w*', 'fraktur\\\\w*', 'fractuur\\\\w*', '\\\\bfract\\\\b', 'kiri[kgğ]\\\\w*', 'prijelom\\\\w*', 'lom kosti', '\\\\bbreuk\\\\w*', '\\\\bbruch\\\\w*', 'καταγμα\\\\w*', 'καταγματ\\\\w*', 'фрактур\\\\w*', 'счупван\\\\w*', 'fisur\\\\w* (osea|oseas|kost)', 'fissur\\\\w* kost')\nPOSTERIOR_ONLY = _rx('\\\\bpcl\\\\b', '\\\\blcp\\\\b', '\\\\bhkb\\\\b', '\\\\bacb\\\\b', 'posterior cruciate', 'cruzado posterior', 'croise posterieur', 'achterste kruisband', 'hinteres kreuzband', 'arka capraz', 'straznji krizn', 'οπισθι[οα]\\\\w* χιαστ', 'задна кръстн', 'задната кръстн')\nLATERAL_COLL_ONLY = _rx('\\\\blcl\\\\b', '\\\\bfcl\\\\b', 'lateral collateral', 'fibular collateral', 'colateral lateral', 'colateral externo', 'buitenband', 'aussenband', 'dis yan bag', 'lateralni kolateraln', 'εξω πλαγι', 'латерален колатерал')"},{"cell_type":"code","execution_count":null,"id":"febe605a","metadata":{},"outputs":[],"source":"def _near(clause: str, stem_rx: re.Pattern, qual_rx: re.Pattern, window: int=55):\n    for m in stem_rx.finditer(clause):\n        lo = max(0, m.start() - window)\n        hi = min(len(clause), m.end() + window)\n        if qual_rx.search(clause[lo:hi]):\n            return True\n    return False\nSTEM_RULES = {'ACL': (STEM_CRUCIATE, SIDE_ANTERIOR), 'MCL': (STEM_COLLATERAL, SIDE_MEDIAL), 'Medial Meniscus': (STEM_MENISCUS, SIDE_MEDIAL), 'Lateral Meniscus': (STEM_MENISCUS, SIDE_LATERAL)}\n\nclass _Matcher:\n\n    def __init__(self, phrase_rx, stem=None, side=None, window=55):\n        self.phrase_rx = phrase_rx\n        self.stem = stem\n        self.side = side\n        self.window = window\n\n    def search(self, clause):\n        m = self.phrase_rx.search(clause)\n        if m is not None:\n            return m\n        if self.stem is not None and _near(clause, self.stem, self.side, self.window):\n            return self.stem.search(clause)\n        return None\nANAT_MATCH = {t: _Matcher(ANAT[t], *STEM_RULES[t]) for t in PAIRED}\nDIRECT_MATCH = {t: _Matcher(_rx(rx.pattern, STEM_FRACTURE.pattern) if t == 'Fracture' else rx) for t, rx in DIRECT.items()}\nSEV_LOW = _rx('\\\\bsmall\\\\b', '\\\\bminimal\\\\b', '\\\\btrace\\\\b', '\\\\bmild\\\\b', '\\\\bslight\\\\b', '\\\\btiny\\\\b', '\\\\bscant\\\\b', '\\\\bdiscrete\\\\b', '\\\\blow ?grade\\\\b', '\\\\bincipient\\\\b', '\\\\bleve\\\\b', '\\\\bminim', '\\\\bpeque', '\\\\bfina\\\\b', '\\\\bfino\\\\b', '\\\\bligero\\\\b', '\\\\bescaso\\\\b', '\\\\bdiscreto\\\\b', '\\\\bhafif\\\\b', '\\\\baz miktarda\\\\b', '\\\\bsilik\\\\b', '\\\\bmanj\\\\w*', '\\\\bblago\\\\b', '\\\\bdiskretn', '\\\\bmalo\\\\b', '\\\\bpocetn', '\\\\bgering', '\\\\bdiskret', '\\\\bkleine?r?\\\\b', '\\\\bwenig\\\\b', '\\\\bzarte?\\\\b', '\\\\bbeperkte?\\\\b', '\\\\bgeringe\\\\b', '\\\\bweinig\\\\b', '\\\\blichte?\\\\b', '\\\\blicht\\\\b', '\\\\bηπι', '\\\\bμικρ', '\\\\bελαχιστ', '\\\\bαρχομεν', '\\\\bминимал', '\\\\bлек', '\\\\bмалк', '\\\\bнеголям')\nSEV_HIGH = _rx('\\\\blarge\\\\b', '\\\\bmarked\\\\b', '\\\\bmassive\\\\b', '\\\\bsevere\\\\b', '\\\\bextensive\\\\b', '\\\\bmoderate\\\\b', '\\\\bgross\\\\b', '\\\\bsignificant\\\\b', '\\\\babundant\\\\b', '\\\\btense\\\\b', '\\\\bcomplete\\\\b', '\\\\bfull ?thickness\\\\b', '\\\\bhigh ?grade\\\\b', '\\\\badvanced\\\\b', '\\\\bmoderad', '\\\\bimportante\\\\b', '\\\\bsevera?\\\\b', '\\\\bmarcad', '\\\\bcuantios', '\\\\bespesor total\\\\b', '\\\\bcompleta?\\\\b', '\\\\bbelirgin\\\\b', '\\\\byaygin\\\\b', '\\\\bileri\\\\b', '\\\\bciddi\\\\b', '\\\\bbol\\\\b', '\\\\bkomplet', '\\\\bopsezan\\\\b', '\\\\bveliki\\\\b', '\\\\bizrazit', '\\\\bznacajn', '\\\\bumjeren', '\\\\buznapredoval', '\\\\bpotpun', '\\\\bkompleksn', '\\\\bausgepragt', '\\\\bdeutlich', '\\\\bmassiv', '\\\\bmassig', '\\\\bgross', '\\\\buitgebreid', '\\\\bgevorderd', '\\\\bveel\\\\b', '\\\\bmatige?\\\\b', '\\\\bvolledig', '\\\\bμετρι', '\\\\bμεγαλ', '\\\\bεκτεταμεν', '\\\\bευμεγεθ', '\\\\bσοβαρ', '\\\\bπληρη', '\\\\bголям', '\\\\bизразен', '\\\\bзначим', '\\\\bумерен', '\\\\bобилен', '\\\\bпълн')\nGRADE_HIGH = re.compile('grade?[ao]?\\\\s*(3|4|iii|iv)\\\\b|icrs grade (iii|iv|3|4)|stupnja iv|stupnja iii|\\\\bgrado (3|4)\\\\b|\\\\bgrad (3|4)\\\\b|\\\\bgrade (3|4)\\\\b')\nDEGENERATIVE_MARROW = _rx('subchondral', 'subcondral', 'subkondral', 'supkondraln', 'subchondraln', 'υποχονδρι', 'υπαρθρικ', 'субхондрал', 'subchondrale?', 'subartikuler', '\\\\bcyst', '\\\\bquist', '\\\\bzyste\\\\b', '\\\\bcistic', 'reactive', 'reactivo', 'degenerative', 'degenerativ', 'reaktiv', '\\\\bcisti\\\\b')\nTRAUMA = _rx('\\\\bbruise\\\\b', '\\\\bcontusion', '\\\\bkontuz', '\\\\btrauma', '\\\\bimpaction\\\\b', '\\\\bpivot shift\\\\b', '\\\\bkissing\\\\b', '\\\\bacute\\\\b', '\\\\bagudo\\\\b', '\\\\bakut', '\\\\bpivot kaymasi\\\\b', '\\\\bcontusion osseuse\\\\b', '\\\\bbone bruise\\\\b', '\\\\bbotcontusie\\\\b', '\\\\bконтузион', '\\\\bμωλωπ', '\\\\bkontuzij', '\\\\bimpaktcij', '\\\\bimpakcij', '\\\\bfall\\\\b', '\\\\binjury\\\\b', '\\\\bimpression\\\\b')\nSYNOVIAL_PROXY = _rx('bursit', 'burzit', '\\\\bbursa\\\\b[^.]{0,30}(fluid|distend|sivi|tekucin|opzetting)', 'suprapatellar (bursitis|effusion|recess)', 'suprapatellar bursa', 'suprapatellar bursada', 'suprapatelarno', 'suprapatellaire recessus', 'hoffa', 'hoffit', 'plica', 'plika', 'πλικα', 'fat pad[^.]{0,20}(edema|oedema)', 'kapsul', 'capsul', 'καψ', 'капсул', '\\\\bpannus\\\\b', '\\\\bsinov', '\\\\bsynov')\n\ndef _polarity(clause: str, span=None) -> str:\n    if UNCERTAIN.search(clause):\n        return 'uncertain'\n    if span is None or not FEATURES['directional_negation']:\n        if NEGATION.search(clause):\n            return 'negative'\n    elif _negated(clause, span[0], span[1]):\n        return 'negative'\n    if NORMALITY.search(clause):\n        if TEAR.search(clause) or GRADE_HIGH.search(clause):\n            return 'positive'\n        return 'negative'\n    return 'positive'\n\ndef _severity(clause: str) -> float:\n    high = SEV_HIGH.search(clause) is not None\n    low = SEV_LOW.search(clause) is not None\n    if high and (not low):\n        return 1.0\n    if low and (not high):\n        return 0.45\n    if high and low:\n        return 0.8\n    return 0.75\n\ndef _grade(n_pos, n_neg, n_unc, best):\n    if n_pos or n_unc:\n        score = min(0.97, 0.5 + 0.45 * best + 0.015 * min(n_pos, 3))\n        conf = min(1.0, 0.55 + 0.15 * n_pos)\n    elif n_neg:\n        score = max(0.04, 0.2 - 0.04 * n_neg)\n        conf = min(0.9, 0.45 + 0.12 * n_neg)\n    else:\n        score, conf = (0.28, 0.05)\n    return (score, conf)\n\ndef _paired_weight(clause: str, meniscus: bool) -> float:\n    g = _grade_of(clause) if FEATURES['graded_pathology'] else None\n    tear = TEAR.search(clause) is not None\n    if meniscus:\n        if tear:\n            base = 1.0\n        elif g is not None:\n            base = 0.95 if g >= 3 else 0.3\n        elif DEGEN.search(clause):\n            base = 0.35\n        else:\n            base = 0.45\n    elif tear:\n        base = 1.0\n    elif g is not None:\n        base = 0.85 if g >= 2 else 0.3\n    elif DEGEN.search(clause):\n        base = 0.4\n    else:\n        base = 0.55\n    if SEV_HIGH.search(clause) and (not SEV_LOW.search(clause)):\n        base = min(1.0, base * 1.2)\n    elif SEV_LOW.search(clause) and (not SEV_HIGH.search(clause)):\n        base *= 0.7\n    return base\n\ndef _score_paired(cls, tgt):\n    anat_rx = ANAT_MATCH[tgt]\n    path_rx = _rx(TEAR.pattern, DEGEN.pattern, INJURY.pattern)\n    meniscus = 'Meniscus' in tgt\n    n_pos = n_neg = n_unc = 0\n    best = 0.0\n    for c in cls:\n        hit = anat_rx.search(c)\n        if hit is None and meniscus and PLURAL_MENISCI.search(c) and (not ANY_SIDE.search(c)):\n            hit = PLURAL_MENISCI.search(c)\n        if hit is None:\n            continue\n        pm = path_rx.search(c)\n        if pm is None and _grade_of(c) is None:\n            if NORMAL_PHRASE.search(c) or (NORMALITY.search(c) and (not NEGATION.search(c))):\n                n_neg += 1\n            continue\n        span = (pm.start(), pm.end()) if pm is not None else None\n        pol = _polarity(c, span)\n        if pol == 'positive':\n            n_pos += 1\n            best = max(best, _paired_weight(c, meniscus))\n        elif pol == 'negative':\n            n_neg += 1\n        else:\n            n_unc += 1\n            best = max(best, 0.45 * _paired_weight(c, meniscus))\n    s, cf = _grade(n_pos, n_neg, n_unc, best)\n    return (s, cf, n_pos, n_neg)\n\ndef _score_clauses(cls, anat_rx, path_rx=None, decoy_rx=None, context_penalty=None, context_bonus=None):\n    n_pos = n_neg = n_unc = 0\n    best = 0.0\n    for c in cls:\n        m = anat_rx.search(c)\n        if not m:\n            continue\n        if decoy_rx is not None and decoy_rx.search(c):\n            continue\n        if path_rx is not None and (not path_rx.search(c)):\n            if NORMAL_PHRASE.search(c) or (NORMALITY.search(c) and (not NEGATION.search(c))):\n                n_neg += 1\n            continue\n        pol = _polarity(c, (m.start(), m.end()))\n        if pol == 'positive':\n            n_pos += 1\n            w = _severity(c)\n            if context_penalty is not None and context_penalty.search(c):\n                w *= 0.45\n            if context_bonus is not None and context_bonus.search(c):\n                w = min(1.0, w * 1.35)\n            best = max(best, w)\n        elif pol == 'negative':\n            n_neg += 1\n        else:\n            n_unc += 1\n            best = max(best, 0.3)\n    s, c = _grade(n_pos, n_neg, n_unc, best)\n    return (s, c, n_pos, n_neg)\n\ndef _score_oa(cls):\n    acc = {t: {'pos': 0, 'neg': 0, 'unc': 0, 'best': 0.0} for t in OA_TARGETS}\n    g_pos, g_neg, g_best = (0, 0, 0.0)\n    for c in cls:\n        m = OA_EVIDENCE.search(c)\n        if not m:\n            continue\n        pol = _polarity(c, (m.start(), m.end()))\n        sev = _severity(c)\n        tf_med = _near(c, TF_SITE, SIDE_MEDIAL, 45)\n        tf_lat = _near(c, TF_SITE, SIDE_LATERAL, 45)\n        pf = PF_SITE.search(c) is not None\n        hits = []\n        if tf_med:\n            hits.append('Medial OA')\n        if tf_lat:\n            hits.append('Lateral OA')\n        if pf:\n            hits.append('PF OA')\n        if not hits:\n            if pol == 'positive':\n                g_pos += 1\n                g_best = max(g_best, sev if GLOBAL_OA.search(c) else sev * 0.7)\n            elif pol == 'negative':\n                g_neg += 1\n            continue\n        for t in hits:\n            if pol == 'positive':\n                acc[t]['pos'] += 1\n                acc[t]['best'] = max(acc[t]['best'], sev)\n            elif pol == 'negative':\n                acc[t]['neg'] += 1\n            else:\n                acc[t]['unc'] += 1\n                acc[t]['best'] = max(acc[t]['best'], 0.3)\n    out = {}\n    for t in OA_TARGETS:\n        a = acc[t]\n        pos, neg, unc, best = (a['pos'], a['neg'], a['unc'], a['best'])\n        if not (pos or unc) and g_pos and FEATURES['oa_inherit']:\n            if neg:\n                score, conf = _grade(0, neg, 0, 0.0)\n                score = max(score, 0.35)\n                conf *= 0.7\n            else:\n                score, conf = _grade(g_pos, 0, 0, g_best * 0.92)\n                conf *= 0.75\n        else:\n            score, conf = _grade(pos, neg + g_neg, unc, best)\n        out[t] = (score, conf, pos, neg)\n    return out\n\ndef extract(report: str) -> dict:\n    cls = clauses(report)\n    out = {}\n    for tgt in PAIRED:\n        s, c, npos, nneg = _score_paired(cls, tgt)\n        out[tgt] = s\n        out[tgt + '__conf'] = c\n        out[tgt + '__npos'] = npos\n        out[tgt + '__nneg'] = nneg\n    for tgt, (s, c, npos, nneg) in _score_oa(cls).items():\n        out[tgt] = s\n        out[tgt + '__conf'] = c\n        out[tgt + '__npos'] = npos\n        out[tgt + '__nneg'] = nneg\n    for tgt in ('Effusion', 'Synovitis', \"Baker's\", 'Contusion', 'Fracture'):\n        if tgt == 'Contusion':\n            s, c, npos, nneg = _score_clauses(cls, DIRECT_MATCH[tgt], None, DECOY.get(tgt), context_penalty=DEGENERATIVE_MARROW, context_bonus=TRAUMA)\n        else:\n            s, c, npos, nneg = _score_clauses(cls, DIRECT_MATCH[tgt], None, DECOY.get(tgt))\n        out[tgt] = s\n        out[tgt + '__conf'] = c\n        out[tgt + '__npos'] = npos\n        out[tgt + '__nneg'] = nneg\n    if FEATURES['synovitis_backoff'] and out['Synovitis__npos'] == 0 and (out['Synovitis__nneg'] == 0):\n        proxy = sum((1 for c in cls if SYNOVIAL_PROXY.search(c) and _polarity(c) == 'positive'))\n        eff = out['Effusion']\n        prior = 0.3 + 0.3 * max(0.0, (eff - 0.5) / 0.45) + 0.06 * min(proxy, 3)\n        out['Synovitis'] = min(0.72, prior)\n        out['Synovitis__conf'] = 0.18\n    return out"},{"cell_type":"code","execution_count":null,"id":"fd28e2c0","metadata":{},"outputs":[],"source":"import os\nimport time\nimport warnings\nfrom pathlib import Path\nimport numpy as np\nimport pandas as pd\nwarnings.filterwarnings('ignore')\nT0 = time.time()\n\ndef log(msg):\n    print(f'[{time.time() - T0:7.1f}s] {msg}', flush=True)\n\ndef find_root():\n    for c in [Path('/kaggle/input/rsna-knee-abnormality-detection'), Path('/kaggle/input/competitions/rsna-knee-abnormality-detection'), Path('data'), Path('.')]:\n        if (c / 'test.csv').is_file() and (c / 'test_series').is_dir():\n            return c\n    base = Path('/kaggle/input')\n    if base.is_dir():\n        for d1 in sorted((p for p in base.iterdir() if p.is_dir())):\n            for cand in [d1] + sorted((p for p in d1.iterdir() if p.is_dir())):\n                if (cand / 'test.csv').is_file():\n                    return cand\n    raise FileNotFoundError('competition mount not found')\nROOT = find_root()\nlog(f'input root: {ROOT}')\n_test_df = pd.read_csv(ROOT / 'test.csv')\n_bench = _test_df[['StudyInstanceUID']].copy()\nfor _c in TARGETS:\n    _bench[_c] = 0.5\n_bench.to_csv('submission.csv', index=False)\nlog(f'benchmark submission.csv written ({len(_bench)} rows)')\nSTAGE_OK = {}\n\ndef stage(name):\n\n    def deco(fn):\n\n        def run(*a, **k):\n            t = time.time()\n            try:\n                out = fn(*a, **k)\n                STAGE_OK[name] = True\n                log(f\"stage '{name}' ok in {time.time() - t:.1f}s\")\n                return out\n            except Exception:\n                import traceback\n                traceback.print_exc()\n                STAGE_OK[name] = False\n                log(f\"stage '{name}' FAILED after {time.time() - t:.1f}s\")\n                return None\n        return run\n    return deco\ntrain_df = pd.read_csv(ROOT / 'train.csv')\nlog(f'train {train_df.shape}  test {_test_df.shape}')\nt = time.time()\nLAB = pd.DataFrame([extract(r) for r in train_df['Report'].fillna('')])\nLAB['StudyInstanceUID'] = train_df['StudyInstanceUID'].values\nLAB = LAB.set_index('StudyInstanceUID')\nlog(f'read {len(LAB)} reports in {time.time() - t:.1f}s')\nGOLD = train_df.dropna(subset=TARGETS).set_index('StudyInstanceUID')[TARGETS]\nlog(f'{len(GOLD)} studies carry the twelve annotations')\npos = (LAB[TARGETS] > 0.5).mean()\nsil = pd.Series({t_: float(((LAB[t_ + '__npos'] == 0) & (LAB[t_ + '__nneg'] == 0)).mean()) for t_ in TARGETS})\nprint(pd.DataFrame({'derived positive rate': pos.round(3), 'silence rate': sil.round(3), 'annotated positive rate': GOLD.mean().round(3)}).to_string())"},{"cell_type":"code","execution_count":null,"id":"eaba7603","metadata":{},"outputs":[],"source":"import matplotlib.pyplot as plt\nfrom sklearn.metrics import roc_auc_score\nplt.rcParams.update({'figure.dpi': 120, 'font.size': 8, 'axes.grid': True, 'grid.alpha': 0.25, 'axes.spines.top': False, 'axes.spines.right': False})\nINK, ACC, WARN = ('#22303f', '#2b7a9b', '#c25a3d')\n\ndef agreement(lab, n_boot=2000, seed=0):\n    rng = np.random.default_rng(seed)\n    g = lab.loc[GOLD.index]\n    rows = []\n    for t_ in TARGETS:\n        y = GOLD[t_].values.astype(int)\n        p = g[t_].values\n        if len(set(y)) < 2:\n            rows.append((t_, np.nan, np.nan, np.nan, int(y.sum()), int((1 - y).sum())))\n            continue\n        a = roc_auc_score(y, p)\n        bs = []\n        for _ in range(n_boot):\n            i = rng.integers(0, len(y), len(y))\n            if len(set(y[i])) > 1:\n                bs.append(roc_auc_score(y[i], p[i]))\n        rows.append((t_, a, np.percentile(bs, 2.5), np.percentile(bs, 97.5), int(y.sum()), int((1 - y).sum())))\n    return pd.DataFrame(rows, columns=['target', 'auc', 'lo', 'hi', 'npos', 'nneg'])\nAGREE = agreement(LAB).dropna(subset=['auc'])\nprint(AGREE.round(3).to_string(index=False))\nprint(f'\\nmacro agreement AUC: {AGREE.auc.mean():.4f}   mean silence rate: {sil.mean() * 100:.1f}%')\nfig, ax = plt.subplots(1, 2, figsize=(10.5, 3.6), gridspec_kw={'width_ratios': [1.25, 1]})\no = AGREE.sort_values('auc')\ny = np.arange(len(o))\nax[0].hlines(y, o.lo, o.hi, color=ACC, lw=3, alpha=0.35)\nax[0].plot(o.auc, y, 'o', color=ACC, ms=5)\nax[0].axvline(0.5, color=INK, lw=0.8, ls=':')\nax[0].axvline(o.auc.mean(), color=WARN, lw=1, ls='--')\nax[0].text(o.auc.mean(), -0.9, f' macro {o.auc.mean():.3f}', color=WARN, fontsize=7)\nax[0].set_ylim(-1.4, len(o) - 0.4)\nax[0].set_yticks(y)\nax[0].set_yticklabels([f'{t_}  ({p}+/{n}-)' for t_, p, n in zip(o.target, o.npos, o.nneg)])\nax[0].set_xlim(0.35, 1.02)\nax[0].set_xlabel('AUC of the derived score against the annotation')\nax[0].set_title('gauge one: agreement, n = 58\\nbars are 95% bootstrap intervals — they are this wide on purpose', loc='left', fontsize=8)\no2 = sil.sort_values()\nax[1].barh(np.arange(len(o2)), o2.values * 100, color=INK, alpha=0.8, height=0.65)\nax[1].set_yticks(np.arange(len(o2)))\nax[1].set_yticklabels(o2.index)\nax[1].set_xlabel('% of studies where no rule fired at all')\nax[1].set_title('gauge two: coverage, n = 4 407\\nno labels needed, so it runs on the whole corpus', loc='left', fontsize=8)\nfor i, v in enumerate(o2.values * 100):\n    ax[1].text(v + 1, i, f'{v:.0f}', va='center', fontsize=6.5, color=INK)\nfig.tight_layout()\nplt.show()"},{"cell_type":"code","execution_count":null,"id":"3919f87b","metadata":{},"outputs":[],"source":"_SCRIPT = {'el': re.compile('[Ͱ-Ͽ]'), 'bg/ru': re.compile('[Ѐ-ӿ]')}\n_STOP = {'en': '\\\\b(the|and|is|with|there is|normal)\\\\b', 'es': '\\\\b(del|los|las|con|sin|senal|rodilla|hallazgos|tecnica|resultados|impresion|menisco|rotura)\\\\b', 'fr': '\\\\b(des|les|avec|sans|genou|aucune)\\\\b', 'nl': '\\\\b(van|het|een|geen|met|voorste|knie)\\\\b', 'de': '\\\\b(der|die|und|mit|ohne|kein|keine|nachweis)\\\\b', 'tr': '\\\\b(ve|ile|izlenmistir|mevcut|normaldir|diz|bulgular)\\\\b', 'hr': '\\\\b(se|te|uz|bez|prikaz|uredan|koljena|meniska)\\\\b'}\n_STOP = {k: re.compile(v) for k, v in _STOP.items()}\n\ndef guess_language(report):\n    n = normalize(report)\n    for tag, rx in _SCRIPT.items():\n        if rx.search(n):\n            return tag\n    score = {k: len(rx.findall(n)) for k, rx in _STOP.items()}\n    best = max(score, key=score.get)\n    return best if score[best] >= 2 else '?'\nLANG = pd.Series([guess_language(r) for r in train_df['Report'].fillna('')], index=train_df['StudyInstanceUID'])\nprint(LANG.value_counts().to_string())\nSIL = pd.DataFrame({t: ((LAB[t + '__npos'] == 0) & (LAB[t + '__nneg'] == 0)).values for t in TARGETS}, index=LAB.index)\nby_lang = SIL.groupby(LANG.reindex(SIL.index).values).mean() * 100\nby_lang = by_lang.loc[LANG.value_counts().index.intersection(by_lang.index)]\nfig, ax = plt.subplots(1, 2, figsize=(12, 3.6), gridspec_kw={'width_ratios': [1.7, 1]})\nim = ax[0].imshow(by_lang.values, cmap='RdYlBu_r', vmin=0, vmax=100, aspect='auto')\nax[0].set_xticks(range(len(TARGETS)))\nax[0].set_xticklabels(TARGETS, rotation=55, ha='right', fontsize=6.5)\nax[0].set_yticks(range(len(by_lang)))\nax[0].set_yticklabels([f'{l}  (n={int((LANG == l).sum())})' for l in by_lang.index], fontsize=7)\nax[0].grid(False)\nfor i in range(by_lang.shape[0]):\n    for j in range(by_lang.shape[1]):\n        v = by_lang.values[i, j]\n        ax[0].text(j, i, f'{v:.0f}', ha='center', va='center', fontsize=5.5, color='white' if v > 62 or v < 12 else INK)\nax[0].set_title('gauge two, broken down: % of studies where no rule fired\\na common finding silent in one language and not another is a lexicon gap — or a reporting style', loc='left', fontsize=8)\nfig.colorbar(im, ax=ax[0], fraction=0.02, pad=0.01)\nCLAUSES = {u: clauses(r) for u, r in zip(train_df['StudyInstanceUID'], train_df['Report'].fillna(''))}\n\ndef names_it(uid, target):\n    cs = CLAUSES[uid]\n    if any((ANAT_MATCH[target].search(c) for c in cs)):\n        return True\n    if 'Meniscus' in target:\n        return any((PLURAL_MENISCI.search(c) and (not ANY_SIDE.search(c)) for c in cs))\n    return False\nrows = []\nfor t_ in ['ACL', 'MCL', 'Medial Meniscus', 'Lateral Meniscus']:\n    sel = SIL[t_].values\n    if not sel.sum():\n        continue\n    named = np.array([names_it(u, t_) for u in SIL.index[sel]])\n    rows.append((t_, int(sel.sum()), 100 * named.mean()))\nD = pd.DataFrame(rows, columns=['target', 'silent', 'names it anyway'])\ny = np.arange(len(D))\nax[1].barh(y, 100 - D['names it anyway'], color='#8a97a3', label='never mentioned')\nax[1].barh(y, D['names it anyway'], left=100 - D['names it anyway'], color=WARN, label='mentioned, missed')\nax[1].set_yticks(y)\nax[1].set_yticklabels([f'{t_}\\n({n} silent)' for t_, n in zip(D.target, D.silent)], fontsize=6.5)\nax[1].set_xlabel('% of the silent studies')\nax[1].legend(fontsize=6.5, frameon=False, loc='lower right')\nax[1].set_title('why it was silent\\nonly the orange half is a lexicon gap', loc='left', fontsize=8)\nfig.tight_layout()\nplt.show()\nprint(D.round(1).to_string(index=False))"},{"cell_type":"code","execution_count":null,"id":"a79229e6","metadata":{},"outputs":[],"source":"from __future__ import annotations\nimport os\nfor _v in ('OMP_NUM_THREADS', 'OPENBLAS_NUM_THREADS', 'MKL_NUM_THREADS'):\n    os.environ.setdefault(_v, '4')\nimport gc\nimport hashlib\nimport json\nimport re\nimport time\nimport traceback\nimport threading\nfrom concurrent.futures import ThreadPoolExecutor\nfrom pathlib import Path\nimport numpy as np\nimport pandas as pd\nimport pydicom\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\ndef _cuda_execution_probe(index):\n    dev = torch.device(f'cuda:{index}')\n    try:\n        major, minor = torch.cuda.get_device_capability(index)\n        probe = nn.Conv2d(3, 4, kernel_size=3, padding=1).eval().to(dev)\n        with torch.inference_mode():\n            out = probe(torch.zeros((1, 3, 16, 16), device=dev))\n            if tuple(out.shape) != (1, 4, 16, 16):\n                raise RuntimeError(f'unexpected CUDA probe shape {tuple(out.shape)}')\n        torch.cuda.synchronize(index)\n        print(f'cuda:{index} probe PASS (compute {major}.{minor})')\n        del probe, out\n        torch.cuda.empty_cache()\n        return True\n    except Exception as exc:\n        print(f'cuda:{index} probe FAIL ({type(exc).__name__}: {exc}); using CPU fallback')\n        try:\n            torch.cuda.empty_cache()\n        except Exception:\n            pass\n        return False\nDEVS = []\nif torch.cuda.is_available():\n    DEVS = [torch.device(f'cuda:{i}') for i in range(torch.cuda.device_count()) if _cuda_execution_probe(i)]\nif not DEVS:\n    DEVS = [torch.device('cpu')]\nprint(f'devices: {[str(d) for d in DEVS]}')\nT0 = time.time()\nSEED = 2026\nnp.random.seed(SEED)\ntorch.manual_seed(SEED)\nTARGETS = ['ACL', 'MCL', 'Medial Meniscus', 'Lateral Meniscus', 'Medial OA', 'Lateral OA', 'PF OA', 'Effusion', 'Synovitis', \"Baker's\", 'Contusion', 'Fracture']\nCROP_MM = 130.0\nCACHE_IMG = 336\nGROUP = 3\nN_GROUP_MAX = 1\nCACHE_FRACTION = 0.45\nCACHE_BUDGET_MAX_GB = 24.0\nCACHE_BUDGET_GB = 12.0\nTEST_SHARE = 0.3\nHDR_THREADS = 16\nPIX_THREADS = 12\nORDER_THREADS = 32\nORDER_BUDGET_S = 5400\nRUNS = [{'name': 'r224', 'img': 224}, {'name': 'r336', 'img': 336}]\nEPOCHS = 10\nBATCH_STUDIES = 8\nAUG_ROT_DEG = 8.0\nAUG_SCALE = 0.08\nAUG_SHIFT = 0.05\nAUG_INTENSITY = 0.1\nLAT_MIN_OFFSET_MM = 20.0\nSLICE_BAND = (0.2, 0.8)\nRULES_NATIVE = {'order': 'normal', 'lat': 'centre', 'slot_fallback': False, 'decode_fill': 'nearest'}\nRULES_LEGACY = {'order': 'dominant_axis', 'lat': 'corner_x', 'slot_fallback': True, 'decode_fill': 'zero'}\nRULES = dict(RULES_NATIVE)\nLEGACY_LAT_OFFSET_MM = 5.0\nLR_HEAD = 0.001\nLR_BACKBONE = 8e-06\nUNFREEZE_LAST = 6\nWEIGHT_DECAY = 0.02\nEVAL_BATCH = 8\nTIME_BUDGET = 8.0 * 3600\nSLOTS_RECOVERED = [('SAG_FLUID_FS', 'Sagittal', True, True), ('COR_FLUID_FS', 'Coronal', True, True), ('AX_FLUID_FS', 'Axial', True, True), ('SAG_FLUID_NOFS', 'Sagittal', True, False), ('COR_T1', 'Coronal', False, False), ('SAG_T1', 'Sagittal', False, False)]\nSLOTS_PUBLIC = [('SAG_FLUID', 'Sagittal', None, True), ('COR_FLUID', 'Coronal', None, True), ('AX_FLUID', 'Axial', None, True), ('SAG_STRUCT', 'Sagittal', None, False), ('COR_STRUCT', 'Coronal', None, False), ('AX_STRUCT', 'Axial', None, False)]\nSLOT_SCHEME = os.environ.get('SLOT_SCHEME', 'recovered')\nSLOTS = SLOTS_PUBLIC if SLOT_SCHEME == 'public' else SLOTS_RECOVERED\nN_SLOT = len(SLOTS)\nPOOL_PARTS = {'cls_mean': 2, 'cls_mean_focal': 3}\nSLOT_PRIOR_TABLE = {'ACL': (0, 3, 5), 'MCL': (1, 4), 'Medial Meniscus': (0, 1, 3, 4), 'Lateral Meniscus': (0, 1, 3, 4), 'Medial OA': (1, 4, 5), 'Lateral OA': (1, 4, 5), 'PF OA': (0, 2, 5), 'Effusion': (0, 2), 'Synovitis': (0, 2), \"Baker's\": (0,), 'Contusion': (0, 1, 2), 'Fracture': (0, 1, 2, 4, 5)}\nSLOT_PRIOR_STRENGTH = 0.55\nFATSAT_OPTS = {'FS', 'FATSAT', 'FAT_SAT', 'FSAT'}\n_SEP = re.compile('[_\\\\-.]')\n_FATSAT_RX = re.compile('\\\\bfs\\\\b|fatsat|fat sat|\\\\bstir\\\\b|\\\\bspair\\\\b|\\\\bspir\\\\b|\\\\bwe\\\\b|water excit|\\\\btirm\\\\b|\\\\bsting\\\\b|\\\\bfatsup\\\\b')\n_T1_RX = re.compile('\\\\bt1\\\\b|\\\\bt1w\\\\b')\n_T2_RX = re.compile('\\\\bt2\\\\b|\\\\bt2w\\\\b')\n_PD_RX = re.compile('\\\\bpd\\\\b|\\\\bpdw\\\\b|proton|\\\\bdp\\\\b|dens')"},{"cell_type":"code","execution_count":null,"id":"357ec3a5","metadata":{},"outputs":[],"source":"def log(msg):\n    print(f'[{time.time() - T0:7.1f}s] {msg}', flush=True)\n\ndef find_root():\n    for c in [Path('/kaggle/input/competitions/rsna-knee-abnormality-detection'), Path('/kaggle/input/rsna-knee-abnormality-detection'), Path('data'), Path('.')]:\n        if (c / 'test.csv').is_file() and (c / 'test_series').is_dir():\n            return c\n    base = Path('/kaggle/input')\n    if base.is_dir():\n        for depth1 in sorted((p for p in base.iterdir() if p.is_dir())):\n            for cand in [depth1] + sorted((p for p in depth1.iterdir() if p.is_dir())):\n                if (cand / 'test.csv').is_file():\n                    return cand\n    raise FileNotFoundError(f'competition mount not found (cwd {Path.cwd()}); expected a directory holding test.csv and test_series/')\n\ndef find_dinov2(variant='small'):\n    base = Path('/kaggle/input')\n    if not base.is_dir():\n        return None\n    hits = []\n    for root, dirs, files in os.walk(base):\n        dirs[:] = [d for d in dirs if d not in ('train_series', 'test_series')]\n        if 'config.json' in files and 'dinov2' in root.lower():\n            hits.append(Path(root))\n    for h in hits:\n        if variant in str(h).lower():\n            return h\n    return hits[0] if hits else None\nLABEL_COLS = TARGETS + [t + '__conf' for t in TARGETS]\n\nclass LabelSourceError(RuntimeError):\n    pass\n\ndef find_label_table():\n    base = Path('/kaggle/input')\n    cands = []\n    if base.is_dir():\n        for root, dirs, files in os.walk(base):\n            dirs[:] = [d for d in dirs if d not in ('train_series', 'test_series')]\n            cands += [Path(root) / f for f in files if f.startswith('report_labels') and f.endswith('.csv')]\n    cands += [p for p in (Path('data/derived/report_labels_v2.csv'),) if p.is_file()]\n    for c in cands:\n        try:\n            head = pd.read_csv(c, nrows=1)\n        except Exception:\n            continue\n        if 'StudyInstanceUID' in head.columns and all((t in head.columns for t in TARGETS)):\n            return c\n    return None\n\ndef label_mount_attached():\n    base = Path('/kaggle/input')\n    if not base.is_dir():\n        return False\n    return any(('label' in p.name.lower() for p in base.iterdir() if p.is_dir()))\n\ndef read_labels(train_df):\n    n = len(train_df)\n    lab = pd.DataFrame([extract(r) for r in train_df['Report'].fillna('')])\n    lab['StudyInstanceUID'] = train_df['StudyInstanceUID'].values\n    lab = lab.set_index('StudyInstanceUID')\n    src = find_label_table()\n    if src is None:\n        if label_mount_attached():\n            raise LabelSourceError('LABEL SOURCE: a label dataset is mounted but no usable table was found in it. Falling back to the lexicon here would train on the weaker labels and say so only in a log line, so the run stops instead.')\n        log(f'LABEL SOURCE: lexicon, {n} studies (no table mounted)')\n        return lab\n    tab = pd.read_csv(src).set_index('StudyInstanceUID')\n    missing = [c for c in LABEL_COLS if c not in tab.columns]\n    if missing:\n        raise LabelSourceError(f'LABEL SOURCE: {src} is missing {len(missing)} expected columns (first: {missing[0]!r}). Refusing to fall back silently.')\n    hit = lab.index.intersection(tab.index)\n    if not len(hit):\n        raise LabelSourceError(f'LABEL SOURCE: {src} shares no StudyInstanceUID with train.csv.')\n    log(f'LABEL SOURCE: {src.name} covers {len(hit)} of {n} studies, lexicon for the remaining {n - len(hit)}')\n    lab.loc[hit, LABEL_COLS] = tab.loc[hit, LABEL_COLS].values\n    return lab\nROOT = find_root()\nlog(f'input root: {ROOT}')\nIMG = CACHE_IMG\n\ndef available_gb():\n    try:\n        with open('/proc/meminfo') as fh:\n            info = {k.strip(): v for k, v in (l.split(':', 1) for l in fh if ':' in l)}\n        return int(info['MemAvailable'].split()[0]) / 1024 ** 2\n    except Exception:\n        return CACHE_BUDGET_GB / CACHE_FRACTION\n\ndef plan_cache(n_study, n_test=0):\n    avail = available_gb()\n    budget = min(avail * CACHE_FRACTION, CACHE_BUDGET_MAX_GB)\n    n_total = n_study + max(n_test, int(TEST_SHARE * n_study))\n    per_slice = n_total * N_SLOT * IMG * IMG\n    afford = int(budget * 1024 ** 3 // max(per_slice, 1))\n    groups = max(1, min(N_GROUP_MAX, afford // GROUP))\n    log(f'memory: {avail:.1f} GB available, {budget:.1f} GB to the cache; sizing for {n_study} train + {n_total - n_study} test studies -> {groups} group(s) of {GROUP} = {groups * GROUP} slices per slot' + (f' (wanted {N_GROUP_MAX})' if groups < N_GROUP_MAX else ''))\n    return groups\nN_GROUP = plan_cache(len(pd.read_csv(ROOT / 'train.csv')), len(pd.read_csv(ROOT / 'test.csv')))\nCACHE_SLICES = GROUP * N_GROUP\nlog(f'cache layout: {N_GROUP} groups x {GROUP} slices = {CACHE_SLICES} per slot')"},{"cell_type":"code","execution_count":null,"id":"5e97efae","metadata":{},"outputs":[],"source":"HDR_TAGS = ['SeriesDescription', 'SequenceName', 'ScanOptions', 'ScanningSequence', 'RepetitionTime', 'EchoTime', 'Laterality', 'PixelSpacing', 'Rows', 'Columns', 'RescaleSlope', 'RescaleIntercept', 'ImagePositionPatient', 'ImageOrientationPatient']\n\ndef _hdr_vec(s, n):\n    if not isinstance(s, str):\n        return None\n    try:\n        v = [float(x) for x in s.split('|')]\n    except ValueError:\n        return None\n    return np.array(v) if len(v) >= n else None\n\ndef side_from_geometry(h):\n    cx = {}\n    for r in h.itertuples(index=False):\n        ipp = _hdr_vec(getattr(r, 'ImagePositionPatient', None), 3)\n        iop = _hdr_vec(getattr(r, 'ImageOrientationPatient', None), 6)\n        ps = _hdr_vec(getattr(r, 'PixelSpacing', None), 2)\n        rows, cols = (getattr(r, 'Rows', None), getattr(r, 'Columns', None))\n        if ipp is None or iop is None or ps is None or (not rows) or (not cols):\n            continue\n        try:\n            c = ipp[:3] + iop[:3] * ps[1] * float(cols) / 2 + iop[3:6] * ps[0] * float(rows) / 2\n        except (TypeError, ValueError):\n            continue\n        cx.setdefault(r.StudyInstanceUID, []).append(float(c[0]))\n    out = {}\n    for st, xs in cx.items():\n        m = float(np.median(xs))\n        out[st] = None if abs(m) < LAT_MIN_OFFSET_MM else 'R' if m < 0 else 'L'\n    return out\n\ndef side_from_corner_x(h):\n    out = {}\n    for st, g in h.groupby('StudyInstanceUID'):\n        xs = []\n        for r in g.itertuples(index=False):\n            ipp = _hdr_vec(getattr(r, 'ImagePositionPatient', None), 3)\n            if ipp is not None and np.isfinite(ipp).all():\n                xs.append(float(ipp[0]))\n        if not xs:\n            out[st] = None\n            continue\n        x = float(np.median(xs))\n        out[st] = None if abs(x) < LEGACY_LAT_OFFSET_MM else 'R' if x < 0 else 'L'\n    return out\n\ndef lat_of(h, tag=''):\n    geo = side_from_corner_x(h) if RULES['lat'] == 'corner_x' else side_from_geometry(h)\n    d, n_tag, n_geo, n_none, n_disagree = ({}, 0, 0, 0, 0)\n    for st, g in h.groupby('StudyInstanceUID'):\n        v = [str(x).strip().upper() for x in g['Laterality'].dropna()]\n        if RULES['lat'] == 'corner_x' and 'ImageLaterality' in g.columns:\n            v += [str(x).strip().upper() for x in g['ImageLaterality'].dropna()]\n        v = [x[0] for x in v if x and x[0] in ('L', 'R')]\n        side = v[0] if v else None\n        if side is not None:\n            n_tag += 1\n            if geo.get(st) is not None and geo[st] != side:\n                n_disagree += 1\n        else:\n            side = geo.get(st)\n            n_geo += side is not None\n            n_none += side is None\n        d[st] = side\n    log(f'{tag}laterality: {n_tag} from the tag, {n_geo} from geometry, {n_none} unresolved; tag and geometry disagree on {n_disagree} ({n_disagree / max(n_tag, 1):.1%} of the tagged)')\n    return d\n\ndef probe(item):\n    split, study, series, path = item\n    row = {'split': split, 'StudyInstanceUID': study, 'SeriesInstanceUID': series, 'dir': path}\n    try:\n        files = sorted((e.name for e in os.scandir(path) if e.name.endswith('.dcm')))\n        row['files'] = files\n        row['n_slices'] = len(files)\n        if not files:\n            return row\n        ds = pydicom.dcmread(os.path.join(path, files[len(files) // 2]), stop_before_pixels=True, force=True)\n        for t in HDR_TAGS:\n            v = getattr(ds, t, None)\n            if v is None:\n                row[t] = None\n            elif isinstance(v, (list, tuple)) or type(v).__name__ == 'MultiValue':\n                row[t] = '|'.join((str(x) for x in v))\n            else:\n                row[t] = str(v)\n    except Exception as exc:\n        row['err'] = str(exc)[:120]\n    return row\n\ndef walk(split):\n    base = ROOT / split\n    items = []\n    if not base.is_dir():\n        return pd.DataFrame(columns=['split', 'StudyInstanceUID', 'SeriesInstanceUID', 'dir', 'files', 'n_slices'] + HDR_TAGS)\n    for study in os.scandir(base):\n        if study.is_dir():\n            for series in os.scandir(study.path):\n                if series.is_dir():\n                    items.append((split, study.name, series.name, series.path))\n    with ThreadPoolExecutor(max_workers=HDR_THREADS) as pool:\n        rows = list(pool.map(probe, items))\n    return pd.DataFrame(rows)\n\ndef annotate(df):\n    desc = df['SeriesDescription'].fillna('') + ' ' + df['SequenceName'].fillna('')\n    desc = desc.str.lower().str.replace(_SEP, ' ', regex=True)\n    opts = df['ScanOptions'].fillna('').str.upper().str.split('|')\n    opts_fs = opts.apply(lambda ts: any((t.strip() in FATSAT_OPTS for t in ts)))\n    df['fatsat'] = desc.str.contains(_FATSAT_RX) | opts_fs\n    tr = pd.to_numeric(df['RepetitionTime'], errors='coerce')\n    te = pd.to_numeric(df['EchoTime'], errors='coerce')\n    gre = df['ScanningSequence'].fillna('').str.upper().str.contains('GR')\n    t1, t2, pdw = (desc.str.contains(_T1_RX), desc.str.contains(_T2_RX), desc.str.contains(_PD_RX))\n    df['weight'] = np.where(t1 & ~t2 & ~pdw, 'T1', np.where(t2 & ~pdw, 'T2', np.where(pdw, 'PD', np.where(gre, 'GRE', np.where(tr < 800, 'T1', np.where(te > 60, 'T2', np.where(tr >= 800, 'PD', 'UNK')))))))\n    df['fluid'] = np.isin(df['weight'], ['PD', 'T2'])\n    df['px'] = pd.to_numeric(df['PixelSpacing'].fillna('').str.split('|').str[0].replace('', np.nan), errors='coerce')\n    return df"},{"cell_type":"code","execution_count":null,"id":"583fef66","metadata":{},"outputs":[],"source":"def pick_slots(series_df, plane_map):\n    series_df = series_df.copy()\n    series_df['plane'] = series_df['SeriesInstanceUID'].map(plane_map)\n    out = {}\n    for study, g in series_df.groupby('StudyInstanceUID'):\n        chosen = {}\n        for name, plane, fluid, fs in SLOTS:\n            sel = (g['plane'] == plane) & (g['fatsat'] == fs)\n            if fluid is not None:\n                sel &= g['fluid'] == fluid\n            cand = g[sel]\n            if len(cand) == 0 and RULES['slot_fallback'] and (fluid is False):\n                cand = g[(g['plane'] == plane) & ~g['fatsat']]\n            if len(cand):\n                chosen[name] = cand.sort_values('n_slices', ascending=False).iloc[0]\n        out[study] = chosen\n    return out"},{"cell_type":"code","execution_count":null,"id":"44c0310b","metadata":{},"outputs":[],"source":"ORDER_TAGS = [(32, 50), (32, 55), (32, 19)]\nDECODE_FAILED = []\n\ndef cache_tag(rules=None):\n    r = dict(RULES if rules is None else rules)\n    t = f'{CACHE_IMG}px_{CACHE_SLICES}sl_{int(CROP_MM)}mm_{SLICE_BAND[0]:.2f}-{SLICE_BAND[1]:.2f}'\n    if {k: r.get(k, v) for k, v in RULES_NATIVE.items()} != RULES_NATIVE:\n        t += '_' + hashlib.md5(json.dumps(r, sort_keys=True).encode()).hexdigest()[:6]\n    return t\n\ndef _natural_key(name):\n    return tuple((int(x) if x.isdigit() else x.lower() for x in re.split('(\\\\d+)', str(name))))\n\ndef _order_dominant_axis(rec):\n    files, d = (rec['files'], rec['dir'])\n    rows = []\n    for pos, f in enumerate(files):\n        ipp = inst = None\n        try:\n            ds = pydicom.dcmread(os.path.join(d, f), force=True, stop_before_pixels=True, specific_tags=['ImagePositionPatient', 'InstanceNumber'])\n            raw = getattr(ds, 'ImagePositionPatient', None)\n            if raw is not None and len(raw) >= 3:\n                c = np.asarray(raw[:3], dtype=np.float64)\n                if np.isfinite(c).all():\n                    ipp = c\n            n = getattr(ds, 'InstanceNumber', None)\n            if n is not None:\n                inst = float(n)\n        except Exception:\n            pass\n        rows.append((f, ipp, inst, pos))\n    placed = [r for r in rows if r[1] is not None]\n    need = max(2, int(0.8 * len(rows)))\n    if len(placed) >= need:\n        xyz = np.stack([r[1] for r in placed])\n        axis = int(np.argmax(np.ptp(xyz, axis=0)))\n        spare = float(np.nanmedian(xyz[:, axis]))\n        rows.sort(key=lambda r: (float(r[1][axis]) if r[1] is not None else spare, r[2] if r[2] is not None else float('inf'), r[3]))\n    elif sum((r[2] is not None for r in rows)) >= need:\n        rows.sort(key=lambda r: (r[2] if r[2] is not None else float('inf'), r[3]))\n    else:\n        rows.sort(key=lambda r: _natural_key(r[0]))\n    return ([r[0] for r in rows], True)\n\ndef order_slices(rec):\n    if RULES['order'] == 'dominant_axis':\n        return _order_dominant_axis(rec)\n    files, d = (rec['files'], rec['dir'])\n    keyed = []\n    for f in files:\n        k = None\n        try:\n            ds = pydicom.dcmread(os.path.join(d, f), force=True, stop_before_pixels=True, specific_tags=ORDER_TAGS)\n            iop = np.asarray(ds.ImageOrientationPatient, dtype=float)\n            ipp = np.asarray(ds.ImagePositionPatient, dtype=float)\n            k = float(np.dot(ipp, np.cross(iop[:3], iop[3:])))\n        except Exception:\n            try:\n                k = float(ds.InstanceNumber)\n            except Exception:\n                k = None\n        keyed.append((k, f))\n    if any((k is None for k, _ in keyed)):\n        return (files, False)\n    return ([f for _, f in sorted(keyed, key=lambda t: t[0])], True)\n\ndef read_slot(rec, n_slice=None, out_size=None):\n    n_slice = GROUP if n_slice is None else n_slice\n    out_size = IMG if out_size is None else out_size\n    files, d, px = (rec.get('ordered') or rec['files'], rec['dir'], rec['px'])\n    n = len(files)\n    if n == 0:\n        return None\n    lo, hi = (int(SLICE_BAND[0] * (n - 1)), int(SLICE_BAND[1] * (n - 1)))\n    idx = np.unique(np.linspace(lo, hi, n_slice).astype(int)) if hi > lo else np.array([n // 2])\n    while len(idx) < n_slice:\n        idx = np.append(idx, idx[-1])\n    planes = []\n    for i in idx[:n_slice]:\n        try:\n            ds = pydicom.dcmread(os.path.join(d, files[int(i)]), force=True)\n            a = ds.pixel_array.astype(np.float32)\n            sl = float(getattr(ds, 'RescaleSlope', 1) or 1)\n            ic = float(getattr(ds, 'RescaleIntercept', 0) or 0)\n            a = a * sl + ic\n        except Exception:\n            a = None\n        planes.append(a)\n    got = [k for k, p in enumerate(planes) if p is not None]\n    if RULES['decode_fill'] == 'zero':\n        if not got:\n            DECODE_FAILED.append(rec.get('SeriesInstanceUID', d))\n        planes = [np.zeros((out_size, out_size), np.float32) if p is None else p for p in planes]\n        got = list(range(len(planes)))\n    if not got:\n        DECODE_FAILED.append(rec.get('SeriesInstanceUID', d))\n        return None\n    if len(got) < len(planes):\n        DECODE_FAILED.append(rec.get('SeriesInstanceUID', d))\n        for k, p in enumerate(planes):\n            if p is None:\n                planes[k] = planes[min(got, key=lambda j: abs(j - k))]\n    shp = planes[0].shape\n    planes = [p if p.shape == shp else np.zeros(shp, np.float32) for p in planes]\n    vol = np.stack(planes)\n    if px and np.isfinite(px) and (px > 0):\n        want = int(round(CROP_MM / px))\n        h, w = shp\n        if 16 < want < min(h, w):\n            cy, cx = (h // 2, w // 2)\n            half = want // 2\n            vol = vol[:, max(0, cy - half):cy + half, max(0, cx - half):cx + half]\n    lo_v, hi_v = np.percentile(vol, [1, 99])\n    vol = np.clip((vol - lo_v) / max(hi_v - lo_v, 1e-06), 0, 1)\n    t = torch.from_numpy(np.ascontiguousarray(vol)).unsqueeze(0)\n    t = F.interpolate(t, size=(out_size, out_size), mode='bilinear', align_corners=False)\n    return (t.squeeze(0) * 255).round().clamp(0, 255).to(torch.uint8)"},{"cell_type":"code","execution_count":null,"id":"47f4d136","metadata":{},"outputs":[],"source":"def normalise_laterality(img, plane, lat):\n    if lat != 'R':\n        return img\n    if plane in ('Coronal', 'Axial'):\n        return torch.flip(img, dims=[-1])\n    return torch.flip(img, dims=[0])"},{"cell_type":"code","execution_count":null,"id":"5b3e7bec","metadata":{},"outputs":[],"source":"ORDER_CACHE = os.environ.get('RSNA_ORDER_CACHE') or None\n\ndef build_cache(slot_map, plane_map, lat_map, tag):\n    studies = sorted(slot_map)\n    sidx = {s: i for i, s in enumerate(studies)}\n    cache = np.zeros((len(studies), N_SLOT, CACHE_SLICES, IMG, IMG), np.uint8)\n    mask = np.zeros((len(studies), N_SLOT), np.float32)\n    log(f'{tag}: cache {cache.shape} = {cache.nbytes / 1024 ** 3:.1f} GB')\n    jobs = [(st, k, plane, slot_map[st][name]) for st in studies for k, (name, plane, _, _) in enumerate(SLOTS) if name in slot_map[st]]\n    n_job = len(jobs)\n    t_ord = time.time()\n    n_slice_total = sum((len(j[3]['files']) for j in jobs))\n    log(f'{tag}: ordering {len(jobs)} slot-series ({n_slice_total} slice headers)')\n    ok = done = 0\n    CHUNK_O = 1024\n    seen = {}\n    if ORDER_CACHE and Path(ORDER_CACHE).is_file():\n        try:\n            import json as _json\n            seen = _json.loads(Path(ORDER_CACHE).read_text())\n        except (OSError, ValueError):\n            seen = {}\n        hit = 0\n        for _, _, _, rec in jobs:\n            e = seen.get(rec['SeriesInstanceUID'])\n            if e and len(e['files']) == len(rec['files']):\n                rec['ordered'] = e['files']\n                ok += int(e['good'])\n                hit += 1\n        jobs = [j for j in jobs if 'ordered' not in j[3]]\n        log(f'{tag}: {hit} slot-series ordered from {ORDER_CACHE}, {len(jobs)} to read')\n    with ThreadPoolExecutor(max_workers=ORDER_THREADS) as pool:\n        for c0 in range(0, len(jobs), CHUNK_O):\n            block = jobs[c0:c0 + CHUNK_O]\n            for (_, _, _, rec), (files, good) in zip(block, pool.map(lambda j: order_slices(j[3]), block)):\n                rec['ordered'] = files\n                ok += int(good)\n                done += 1\n                if ORDER_CACHE:\n                    seen[rec['SeriesInstanceUID']] = {'files': files, 'good': bool(good)}\n            budget = min(ORDER_BUDGET_S, max(60.0, (TIME_BUDGET - (time.time() - T0)) * 0.35))\n            if time.time() - t_ord > budget:\n                log(f'{tag}: ordering budget spent at {done}/{len(jobs)}; the rest keep file order')\n                break\n    if ORDER_CACHE and done:\n        import json as _json\n        _t = Path(ORDER_CACHE).with_suffix('.tmp')\n        _t.write_text(_json.dumps(seen))\n        _t.replace(Path(ORDER_CACHE))\n    log(f'{tag}: ordered {ok}/{n_job} by geometry ({n_job - ok} kept arbitrary) in {time.time() - t_ord:.0f}s')\n    jobs = [(st, k, plane, slot_map[st][name]) for st in studies for k, (name, plane, _, _) in enumerate(SLOTS) if name in slot_map[st]]\n    log(f'{tag}: decoding {len(jobs)} slot-series')\n    n_failed_before = len(DECODE_FAILED)\n    CHUNK = 512\n    done = 0\n    with ThreadPoolExecutor(max_workers=PIX_THREADS) as pool:\n        for c0 in range(0, len(jobs), CHUNK):\n            block = jobs[c0:c0 + CHUNK]\n            for (st, k, plane, _), img in zip(block, pool.map(lambda j: read_slot(j[3], CACHE_SLICES, IMG), block)):\n                done += 1\n                if img is None:\n                    continue\n                cache[sidx[st], k] = normalise_laterality(img, plane, lat_map.get(st)).numpy()\n                mask[sidx[st], k] = 1.0\n            if done % 4096 < CHUNK:\n                log(f'  {tag} {done}/{len(jobs)}')\n            if time.time() - T0 > TIME_BUDGET:\n                log(f'  {tag}: time budget reached during decode')\n                break\n    n_failed = len(DECODE_FAILED) - n_failed_before\n    log(f'{tag}: {int(mask.sum())}/{len(jobs)} slots filled' + (f'; {n_failed} series had a slice that would not decode' if n_failed else ''))\n    gc.collect()\n    return (studies, cache, mask)"},{"cell_type":"code","execution_count":null,"id":"7b1c3ca6","metadata":{},"outputs":[],"source":"class SlotHead(nn.Module):\n\n    def __init__(self, dim, n_slot, n_out, hidden=256, p=0.2, prior=False):\n        super().__init__()\n        self.proj = nn.Sequential(nn.LayerNorm(dim), nn.Linear(dim, hidden), nn.GELU())\n        self.slot_emb = nn.Parameter(torch.randn(n_slot, hidden) * 0.02)\n        self.query = nn.Parameter(torch.randn(n_out, hidden) * 0.02)\n        self.drop = nn.Dropout(p)\n        self.out = nn.Linear(hidden, n_out)\n        self.hidden = hidden\n        p_ = torch.zeros(n_out, n_slot)\n        if prior and n_slot == len(SLOTS) and (n_out == len(TARGETS)):\n            for t, slots in SLOT_PRIOR_TABLE.items():\n                if t in TARGETS:\n                    p_[TARGETS.index(t), list(slots)] = SLOT_PRIOR_STRENGTH\n        self.prior = prior\n        if prior:\n            self.register_buffer('slot_prior', p_)\n\n    def forward(self, x, mask):\n        h = self.proj(x) + self.slot_emb\n        att = torch.einsum('bsh,oh->bos', h, self.query) / self.hidden ** 0.5\n        if self.prior:\n            att = att + self.slot_prior.unsqueeze(0)\n        att = att.masked_fill(mask.unsqueeze(1) < 0.5, -10000.0).softmax(-1)\n        ctx = self.drop(torch.einsum('bos,bsh->boh', att, h))\n        return (ctx * self.out.weight.unsqueeze(0)).sum(-1) + self.out.bias"},{"cell_type":"code","execution_count":null,"id":"4707508f","metadata":{},"outputs":[],"source":"class Model(nn.Module):\n\n    def __init__(self, backbone, dim, pool='cls_mean', prior=False):\n        super().__init__()\n        self.backbone = backbone\n        self.pool = pool\n        self.head = SlotHead(dim * POOL_PARTS[pool], N_SLOT, len(TARGETS), prior=prior)\n        self.register_buffer('mean', torch.tensor([0.485, 0.456, 0.406]).view(1, 3, 1, 1))\n        self.register_buffer('std', torch.tensor([0.229, 0.224, 0.225]).view(1, 3, 1, 1))\n\n    def forward(self, imgs, mask, img_size=None):\n        B, S = imgs.shape[:2]\n        x = imgs.reshape(B * S, *imgs.shape[2:]).float().div_(255.0)\n        if img_size is not None and img_size != x.shape[-1]:\n            x = F.interpolate(x, size=(img_size, img_size), mode='bilinear', align_corners=False)\n        x = (x - self.mean) / self.std\n        out = self.backbone(pixel_values=x).last_hidden_state\n        patch = out[:, 1:]\n        parts = [out[:, 0], patch.mean(1)]\n        if self.pool == 'cls_mean_focal':\n            k = max(1, patch.shape[1] // 8)\n            parts.append(patch.topk(k, dim=1).values.mean(1))\n        feat = torch.cat(parts, dim=1).reshape(B, S, -1)\n        return self.head(feat, mask)"},{"cell_type":"code","execution_count":null,"id":"338321b9","metadata":{},"outputs":[],"source":"def build_model(unfreeze_last, source=None, variant='small', pool='cls_mean', prior=False):\n    from transformers import AutoModel\n    p = source if source is not None else find_dinov2(variant)\n    if p is None:\n        raise FileNotFoundError('DINOv2 weights not attached')\n    bb = AutoModel.from_pretrained(str(p))\n    n_layer = len(bb.encoder.layer)\n    for prm in bb.parameters():\n        prm.requires_grad = False\n    for blk in bb.encoder.layer[max(0, n_layer - unfreeze_last):]:\n        for prm in blk.parameters():\n            prm.requires_grad = True\n    for prm in bb.layernorm.parameters():\n        prm.requires_grad = True\n    dim = bb.config.hidden_size\n    trainable = sum((p.numel() for p in bb.parameters() if p.requires_grad))\n    log(f'backbone: {n_layer} blocks, last {unfreeze_last} trainable ({trainable / 1000000.0:.1f}M params), feature dim {dim * POOL_PARTS[pool]}')\n    return Model(bb, dim, pool=pool, prior=prior)"},{"cell_type":"code","execution_count":null,"id":"bff7db08","metadata":{},"outputs":[],"source":"FINGERPRINT_TOL = 0.002\n\ndef fingerprint(model, dev, img_size, n_slot=None, group=None, seed=None):\n    n_slot = N_SLOT if n_slot is None else n_slot\n    group = GROUP if group is None else group\n    seed = SEED if seed is None else seed\n    g = torch.Generator().manual_seed(seed)\n    imgs = torch.randint(0, 256, (2, n_slot, group, img_size, img_size), generator=g, dtype=torch.uint8).to(dev)\n    mask = torch.ones(2, n_slot, device=dev)\n    mask[1, -1] = 0.0\n    was_training = model.training\n    model.eval()\n    with torch.no_grad():\n        out = model(imgs, mask, img_size).float().cpu().numpy()\n    if was_training:\n        model.train()\n    return out\n\ndef check_fingerprint(model, dev, img_size, expected, tol=FINGERPRINT_TOL, tag=''):\n    got = fingerprint(model, dev, img_size)\n    exp = np.asarray(expected, np.float32)\n    if got.shape != exp.shape:\n        raise WeightsError(f'{tag}fingerprint shape {got.shape} != stored {exp.shape}: the architecture is not the one these weights were fitted to')\n    d = float(np.abs(got - exp).max())\n    if d > tol:\n        raise WeightsError(f'{tag}fingerprint differs by {d:.4g} (tolerance {tol:g}). The weights load but do not compute what they computed when fitted - preprocessing, resolution or architecture has moved between the two runs.')\n    log(f'{tag}fingerprint matches within {d:.2g}')\n    return d\n\nclass WeightsError(RuntimeError):\n    pass\n\ndef find_weights(name='manifest.json'):\n    import json\n    base = Path('/kaggle/input')\n    if not base.is_dir():\n        return None\n    for root, dirs, files in os.walk(base):\n        dirs[:] = [d for d in dirs if d not in ('train_series', 'test_series')]\n        if name not in files:\n            continue\n        try:\n            man = json.loads((Path(root) / name).read_text())\n        except (OSError, ValueError):\n            continue\n        if isinstance(man.get('members'), list) and man['members']:\n            missing = [m['file'] for m in man['members'] if not (Path(root) / m['file']).is_file()]\n            if missing:\n                raise WeightsError(f\"{root} holds a manifest listing {len(man['members'])} members but {len(missing)} of their files are absent (first {missing[0]!r})\")\n            return Path(root)\n    return None\nTTA_OVERLAP = True\nTTA_POOL = 'prob'\nPUBLIC_FRONTIER_TARGET_POOL = {'Fracture': 'max', 'Contusion': 'max', 'Medial Meniscus': 'max', 'Lateral Meniscus': 'max', 'ACL': 'top2', 'MCL': 'top2', \"Baker's\": 'max'}\nTTA_TARGET_POOL = {**PUBLIC_FRONTIER_TARGET_POOL, 'Synovitis': 'original_mean'}\nLEGACY_MEMBER_WEIGHT_BY_TARGET = {'Lateral Meniscus': 15.0, 'Medial OA': 2.5, 'Lateral OA': 15.0, 'Contusion': 5.0}\n\ndef window_starts(n_slice, group, overlap=None):\n    overlap = TTA_OVERLAP if overlap is None else overlap\n    if overlap and n_slice >= group:\n        return list(range(n_slice - group + 1))\n    return [g * group for g in range(max(n_slice // group, 1))]\n\ndef apply_target_window_pool(values, probs, logits, original_probs, mapping, target_idx):\n    for target, mode in mapping.items():\n        j = target_idx[target]\n        if mode == 'max':\n            values[:, j] = probs[:, :, j].max(0).values\n        elif mode == 'mean':\n            values[:, j] = probs[:, :, j].mean(0)\n        elif mode == 'logit_mean':\n            values[:, j] = torch.sigmoid(logits[:, :, j].mean(0))\n        elif mode == 'original_mean':\n            values[:, j] = original_probs[:, :, j].mean(0)\n        elif mode in ('top2', 'top3'):\n            k = min(int(mode[3:]), probs.shape[0])\n            values[:, j] = probs[:, :, j].topk(k, dim=0).values.mean(0)\n        else:\n            raise ValueError(f'unknown TTA pooling mode for {target}: {mode}')\n    return values\n\n@torch.no_grad()\ndef predict_member(model, cache, mask, idx, dev, img_size, group=None, pool=None, starts=None, jitter=False, jitter_seed=SEED, return_public_frontier=False):\n    group = GROUP if group is None else group\n    pool = TTA_POOL if pool is None else pool\n    starts = window_starts(cache.shape[2], group) if starts is None else list(starts)\n    if not starts:\n        raise ValueError('predict_member was given no windows to average over')\n    target_idx = {t: j for j, t in enumerate(TARGETS)}\n    unknown = (set(TTA_TARGET_POOL) | set(PUBLIC_FRONTIER_TARGET_POOL)) - set(target_idx)\n    if unknown:\n        raise ValueError(f'unknown target(s) in TTA_TARGET_POOL: {unknown}')\n    jitter_gen = torch.Generator(device=dev)\n    jitter_gen.manual_seed(int(jitter_seed) % (2 ** 63 - 1))\n    model.eval()\n    out, public_frontier_out = ([], [])\n    for b in range(0, len(idx), EVAL_BATCH):\n        sel = idx[b:b + EVAL_BATCH]\n        m = torch.from_numpy(mask[sel]).to(dev)\n        win_probs, win_logits, win_original_probs = ([], [], [])\n        for st in starts:\n            rows = torch.from_numpy(np.ascontiguousarray(cache[sel, :, st:st + group])).to(dev)\n            views = [rows] + ([augment(rows, generator=jitter_gen)] if jitter else [])\n            view_probs, view_logits = ([], [])\n            for view in views:\n                with torch.autocast('cuda', enabled=dev.type == 'cuda'):\n                    z = model(view, m, img_size).float()\n                view_logits.append(z)\n                view_probs.append(torch.sigmoid(z))\n            win_logits.append(torch.stack(view_logits).mean(0))\n            win_probs.append(torch.stack(view_probs).mean(0))\n            win_original_probs.append(view_probs[0])\n        probs = torch.stack(win_probs)\n        logits = torch.stack(win_logits)\n        original_probs = torch.stack(win_original_probs)\n        v = torch.sigmoid(logits.mean(0)) if pool == 'logit' else probs.mean(0)\n        v = apply_target_window_pool(v, probs, logits, original_probs, TTA_TARGET_POOL, target_idx)\n        out.append(v.cpu().numpy())\n        if return_public_frontier:\n            public_v = apply_target_window_pool(original_probs.mean(0), original_probs, logits, original_probs, PUBLIC_FRONTIER_TARGET_POOL, target_idx)\n            public_frontier_out.append(public_v.cpu().numpy())\n    primary = np.concatenate(out) if out else np.zeros((0, len(TARGETS)), np.float32)\n    if not return_public_frontier:\n        return primary\n    public_frontier = np.concatenate(public_frontier_out) if public_frontier_out else np.zeros((0, len(TARGETS)), np.float32)\n    return (primary, public_frontier)\nBUILD_LOCK = threading.Lock()\nSTATE_LOCK = threading.Lock()\nLEGACY_BUNDLE_FILE = 'rsna_20260807_v1.pt'\nLEGACY_WEIGHT = 0.5\n\ndef find_legacy_bundle():\n    base = Path('/kaggle/input')\n    if not base.is_dir():\n        return None\n    for root, dirs, files in os.walk(base):\n        dirs[:] = [d for d in dirs if d not in ('train_series', 'test_series')]\n        if LEGACY_BUNDLE_FILE in files:\n            return Path(root) / LEGACY_BUNDLE_FILE\n    return None\n\ndef legacy_group_members():\n    p = find_legacy_bundle()\n    if p is None:\n        log('no legacy bundle attached; blending skipped')\n        return {}\n    try:\n        b = torch.load(p, map_location='cpu', weights_only=False)\n        folds = b.get('fold_states') or []\n        b_slots = [tuple(s)[0] for s in b.get('slots', SLOTS)]\n        if list(b.get('targets', TARGETS)) != TARGETS or b_slots != [s[0] for s in SLOTS]:\n            log(f'legacy bundle {p.name}: target/slot contract differs; blending skipped')\n            return {}\n        gr, n_gr = (int(b.get('group', 3)), int(b.get('n_group', 3)))\n        variant = str(b.get('model_variant', 'dinov2-small')).split('-')[-1]\n        key = json.dumps({'img': int(b.get('img', 224)), 'group': gr, 'slices': gr * n_gr, 'crop_mm': 160.0, 'band': [0.2, 0.8], 'rules': RULES_LEGACY, 'slots': [s[0] for s in SLOTS]}, sort_keys=True)\n        ms = [{'id': f\"legacy-f{f.get('fold', k)}\", 'fold': f.get('fold', k), 'state': f['state_dict'], 'holdout': None, 'weight': LEGACY_WEIGHT, 'target_weight': [LEGACY_MEMBER_WEIGHT_BY_TARGET.get(t, 0.0) for t in TARGETS], 'pixel_group': key, 'config': {'unfreeze_last': 6, 'variant': 'base' if variant == 'base' else 'small', 'pool': 'cls_mean_focal', 'prior': True}} for k, f in enumerate(folds)]\n        if ms:\n            active = sorted(set(LEGACY_MEMBER_WEIGHT_BY_TARGET.values()))\n            log(f'legacy bundle {p.name}: {len(ms)} fold(s) join with target-specific per-member weights {active}')\n        return {key: ms} if ms else {}\n    except Exception as exc:\n        log(f'legacy bundle unusable ({type(exc).__name__}: {exc}); blending skipped')\n        return {}\n\ndef _run_member(path, m, dev, Cte, Mte, idx, starts, jitter):\n    t0 = time.time()\n    with BUILD_LOCK:\n        if 'state' in m:\n            state, fp = (m['state'], None)\n        else:\n            ck = torch.load(Path(path) / m['file'], map_location='cpu', weights_only=False)\n            state, fp = (ck['model'], ck.get('fingerprint'))\n        model = build_model(int(m['config']['unfreeze_last']), variant=m['config']['variant'], pool=m['config'].get('pool', 'cls_mean'), prior=bool(m['config'].get('prior', False))).to(dev)\n        model.load_state_dict(state)\n        if fp is not None:\n            check_fingerprint(model, dev, IMG, fp, tag=f\"{m['id']}: \")\n        else:\n            log(f\"  {m['id']}: no stored fingerprint (legacy bundle) -- accepted at reduced weight\")\n    t_ready = time.time()\n    jitter_seed = SEED + int(hashlib.sha256(str(m['id']).encode()).hexdigest()[:8], 16)\n    public_member = 'state' not in m\n    predicted = predict_member(model, Cte, Mte, idx, dev, IMG, starts=starts, jitter=jitter, jitter_seed=jitter_seed, return_public_frontier=public_member)\n    if public_member:\n        p, public_p = predicted\n    else:\n        p, public_p = (predicted, None)\n    t_done = time.time()\n    del model, state\n    gc.collect()\n    if dev.type == 'cuda':\n        with torch.cuda.device(dev):\n            torch.cuda.empty_cache()\n    passes = len(starts) * (2 if jitter else 1)\n    return (p, public_p, (t_ready - t0, (t_done - t_ready) / max(passes, 1)))\n\ndef _combine(per_member):\n    all_ids = sorted({s for m in per_member for s in m['ids']})\n    pos = {s: i for i, s in enumerate(all_ids)}\n    acc = np.zeros((len(all_ids), len(TARGETS)), np.float64)\n    tot = np.zeros(len(TARGETS), np.float64)\n    for m in per_member:\n        target_weight = m.get('target_weight')\n        w = np.asarray(target_weight if target_weight is not None else [float(m.get('weight', 1.0))] * len(TARGETS), dtype=np.float64)\n        if w.shape != (len(TARGETS),) or np.any(w < 0):\n            raise ValueError(f\"invalid target weights for {m.get('id')}: {w}\")\n        r = pd.DataFrame(m['pred']).rank(pct=True).to_numpy()\n        acc[[pos[s] for s in m['ids']]] += r * w[None, :]\n        tot += w\n    if np.any(tot <= 0):\n        raise ValueError(f'at least one target has no ensemble vote: {tot}')\n    return (all_ids, acc / tot[None, :])\n\ndef infer_from_package(path, dev=None):\n    man = json.loads((Path(path) / 'manifest.json').read_text())\n    members = man['members']\n    log(f'weights package: {len(members)} member(s) from {path}; {len(DEVS)} device(s)')\n    test_df = pd.read_csv(ROOT / 'test.csv')\n    test_series = pd.read_csv(ROOT / 'test_series.csv')\n    plane_map = dict(zip(test_series['SeriesInstanceUID'], test_series['Anatomical_Plane']))\n    hte = annotate(walk('test_series'))\n    log(f'test header pass: {len(hte)} series')\n    groups = {}\n    for m in members:\n        groups.setdefault(m['pixel_group'], []).append(m)\n    groups.update(legacy_group_members())\n    per_member, public_frontier_members = ([], [])\n    est = {'fixed': None, 'win': None}\n\n    def bank(m, ids, pred, starts, jitter, public_pred=None):\n        if float(np.std(pred)) < 1e-09:\n            log(f\"  {m['id']}: degenerate predictions; not banked\")\n            return\n        with STATE_LOCK:\n            per_member.append({'id': m['id'], 'ids': ids, 'pred': pred, 'weight': m.get('weight', 1.0), 'target_weight': m.get('target_weight'), 'holdout': m.get('holdout')})\n            if public_pred is not None and len(starts) == len(starts_full):\n                if float(np.std(public_pred)) < 1e-09:\n                    raise WeightsError(f\"{m['id']}: degenerate public-frontier prediction\")\n                public_frontier_members.append({'id': m['id'], 'ids': ids, 'pred': public_pred})\n            elif public_pred is not None:\n                log(f\"  {m['id']}: public-frontier vote omitted because only {len(starts)} / {len(starts_full)} windows completed\")\n            all_ids, acc = _combine(per_member)\n            write_submission(acc, all_ids, test_df, 'submission.csv')\n            log(f\"  banked {m['id']} fold {m.get('fold', '?')} ({len(starts)} window(s){(', jitter' if jitter else '')}); submission.csv = weighted rank mean of {len(per_member)} member(s)\")\n    for gi, (key, gm) in enumerate(groups.items(), 1):\n        cfg = json.loads(key)\n        adopt_config_globals(cfg)\n        log(f\"decode group {gi}/{len(groups)}: {cfg['img']}px x {cfg['slices']} slices, crop {cfg['crop_mm']} mm -> {len(gm)} member(s)\")\n        st_te, Cte, Mte = build_cache(pick_slots(hte, plane_map), plane_map, lat_of(hte, 'test '), f'test g{gi}')\n        idx = np.arange(len(st_te))\n        starts_full = window_starts(Cte.shape[2], GROUP)\n        pending = sorted(gm, key=lambda m: -(m.get('holdout') or 0))\n        left_after = sum((len(g) for j, (_, g) in enumerate(groups.items(), 1) if j > gi))\n\n        def pop_next():\n            with STATE_LOCK:\n                if not pending:\n                    return (None, None, False)\n                left = TIME_BUDGET - (time.time() - T0)\n                remaining = len(pending) + left_after\n                slots_left = -(-remaining // len(DEVS))\n                starts, jit = (starts_full, False)\n                if est['fixed'] is not None and est['win'] is not None:\n                    afford = max(left * 0.9, 0.0)\n                    room = afford / max(slots_left, 1)\n                    if est['fixed'] + est['win'] > room:\n                        log(f'  {left / 60:.0f} min left: surrendering {len(pending)} member(s); not one more fits')\n                        pending.clear()\n                        return (None, None, False)\n                    jit = est['fixed'] + 2 * len(starts_full) * est['win'] <= room * 0.6\n                    per_win = est['win'] * (2 if jit else 1)\n                    n_win = int((room - est['fixed']) / per_win) if per_win > 0 else len(starts_full)\n                    n_win = max(1, min(len(starts_full), n_win))\n                    if n_win < len(starts_full):\n                        mid = (len(starts_full) - n_win) // 2\n                        starts = starts_full[mid:mid + n_win]\n                return (pending.pop(0), starts, jit)\n\n        def worker(dev):\n            others = [d for d in DEVS if d is not dev]\n            while True:\n                m, starts, jit = pop_next()\n                if m is None:\n                    return\n                for attempt, d in enumerate([dev] + others[:1]):\n                    try:\n                        p, public_p, (fs, ws) = _run_member(path, m, d, Cte, Mte, idx, starts, jit)\n                        with STATE_LOCK:\n                            est['fixed'], est['win'] = (fs, ws)\n                        bank(m, st_te, p, starts, jit, public_p)\n                        break\n                    except Exception as exc:\n                        log(f\"  MEMBER {m['id']} failed on {d} ({type(exc).__name__}: {exc}); \" + ('retrying on peer device' if attempt == 0 and others else 'dropped -- costs one vote, not the run'))\n                        if d.type == 'cuda':\n                            with torch.cuda.device(d):\n                                torch.cuda.empty_cache()\n        threads = [threading.Thread(target=worker, args=(d,)) for d in DEVS]\n        for t in threads:\n            t.start()\n        for t in threads:\n            t.join()\n        del Cte, Mte\n        gc.collect()\n    if not per_member:\n        raise WeightsError('no member produced predictions; submission stays at 0.5')\n    all_ids, acc = _combine(per_member)\n    sub = write_submission(acc, all_ids, test_df, 'submission.csv')\n    log(f'final submission.csv = weighted rank mean of {len(per_member)} member(s); {sub.shape}; nulls {int(sub[TARGETS].isna().sum().sum())}')\n    if len(public_frontier_members) == len(members):\n        frontier_ids, frontier_acc = _combine(public_frontier_members)\n        frontier_sub = write_submission(frontier_acc, frontier_ids, test_df, 'submission_public_0899.csv')\n        log(f'submission_public_0899.csv = exact no-jitter public-frontier rank mean of {len(public_frontier_members)} member(s); {frontier_sub.shape}; nulls {int(frontier_sub[TARGETS].isna().sum().sum())}')\n    else:\n        log(f'public-frontier fallback not emitted: {len(public_frontier_members)} / {len(members)} required public members completed')\n    return sub\n\ndef adopt_config_globals(cfg):\n    global IMG, CACHE_IMG, GROUP, CACHE_SLICES, N_GROUP, CROP_MM, SLICE_BAND, RULES\n    CACHE_IMG = IMG = int(cfg['img'])\n    GROUP = int(cfg['group'])\n    CACHE_SLICES = int(cfg['slices'])\n    N_GROUP = max(CACHE_SLICES // GROUP, 1)\n    CROP_MM = float(cfg['crop_mm'])\n    SLICE_BAND = tuple((float(x) for x in cfg['band']))\n    rules = cfg.get('rules') or RULES_NATIVE\n    unknown = {k: v for k, v in rules.items() if k not in RULES_NATIVE or v not in (RULES_NATIVE[k], RULES_LEGACY[k])}\n    if unknown:\n        raise WeightsError(f'the members record pixel rules this pipeline cannot reproduce: {unknown}')\n    RULES = {**RULES_NATIVE, **rules}\n    if [s[0] for s in SLOTS] != list(cfg['slots']):\n        raise WeightsError(f\"the members were fitted on slots {cfg['slots']} and this pipeline defines {[s[0] for s in SLOTS]}; a weight would be read against the wrong slot\")"},{"cell_type":"code","execution_count":null,"id":"27c516e9","metadata":{},"outputs":[],"source":"def take_group(cache_rows, g):\n    return cache_rows[:, :, g * GROUP:(g + 1) * GROUP]\n\ndef augment(imgs, generator=None):\n    lead = imgs.shape[:-3]\n    x = imgs.reshape(-1, *imgs.shape[-3:]).float()\n    n, dev = (x.shape[0], x.device)\n    rot = (torch.rand(n, device=dev, generator=generator) - 0.5) * 2 * (AUG_ROT_DEG * np.pi / 180)\n    sc = 1.0 + torch.rand(n, device=dev, generator=generator) * AUG_SCALE\n    tx = (torch.rand(n, device=dev, generator=generator) - 0.5) * 2 * AUG_SHIFT\n    ty = (torch.rand(n, device=dev, generator=generator) - 0.5) * 2 * AUG_SHIFT\n    cos, sin = (torch.cos(rot) / sc, torch.sin(rot) / sc)\n    theta = torch.zeros(n, 2, 3, device=dev, dtype=torch.float32)\n    theta[:, 0, 0], theta[:, 0, 1], theta[:, 0, 2] = (cos, -sin, tx)\n    theta[:, 1, 0], theta[:, 1, 1], theta[:, 1, 2] = (sin, cos, ty)\n    grid = F.affine_grid(theta, x.shape, align_corners=False)\n    x = F.grid_sample(x, grid, mode='bilinear', padding_mode='border', align_corners=False)\n    scale = 1.0 + (torch.rand(n, 1, 1, 1, device=dev, generator=generator) - 0.5) * 2 * AUG_INTENSITY\n    x = (x * scale).clamp(0, 255)\n    return x.reshape(*lead, *x.shape[-3:]).to(imgs.dtype)\n\n@torch.no_grad()\ndef predict(model, cache, mask, idx, dev, img_size=None):\n    model.eval()\n    out = []\n    for b in range(0, len(idx), EVAL_BATCH):\n        sel = idx[b:b + EVAL_BATCH]\n        m = torch.from_numpy(mask[sel]).to(dev)\n        acc = None\n        for g in range(N_GROUP):\n            rows = torch.from_numpy(np.ascontiguousarray(cache[sel, :, g * GROUP:(g + 1) * GROUP])).to(dev)\n            with torch.autocast('cuda', enabled=dev.type == 'cuda'):\n                z = model(rows, m, img_size).float()\n            acc = z if acc is None else acc + z\n        out.append(torch.sigmoid(acc / N_GROUP).cpu().numpy())\n    return np.concatenate(out) if out else np.zeros((0, len(TARGETS)), np.float32)\n\ndef macro_auc(y, p):\n    from sklearn.metrics import roc_auc_score\n    return float(np.nanmean([roc_auc_score(y[:, j], p[:, j]) if len(set(y[:, j])) > 1 else np.nan for j in range(y.shape[1])]))"},{"cell_type":"code","execution_count":null,"id":"d377c19d","metadata":{},"outputs":[],"source":"import math\nimport cv2\nTARGET_FAMILIES = ['acl', 'mcl', 'medial_meniscus', 'lateral_meniscus', 'medial_oa', 'lateral_oa', 'pf_oa', 'effusion', 'synovitis', 'baker', 'contusion', 'fracture']\nGROUP_NAMES = ['ligament', 'meniscus', 'oa', 'inflammation', 'bone', 'other']\n\ndef target_group_id(family):\n    if family in {'acl', 'mcl'}:\n        return 0\n    if family in {'medial_meniscus', 'lateral_meniscus'}:\n        return 1\n    if family in {'medial_oa', 'lateral_oa', 'pf_oa'}:\n        return 2\n    if family in {'effusion', 'synovitis', 'baker'}:\n        return 3\n    if family in {'contusion', 'fracture'}:\n        return 4\n    return 5\nTARGET_GROUP_IDS = torch.tensor([target_group_id(f) for f in TARGET_FAMILIES], dtype=torch.long)\n\nclass RTAHMIL(nn.Module):\n\n    def __init__(self, in_dim, hidden_dim, n_targets, n_slots, n_slices, dropout, series_dropout):\n        super().__init__()\n        self.n_targets = n_targets\n        self.n_slots = n_slots\n        self.n_slices = n_slices\n        self.series_dropout = float(series_dropout)\n        self.input_proj = nn.Sequential(nn.LayerNorm(in_dim), nn.Linear(in_dim, hidden_dim), nn.GELU(), nn.Dropout(dropout))\n        self.slice_pos_emb = nn.Parameter(torch.randn(n_slices, hidden_dim) / math.sqrt(hidden_dim))\n        slice_layer = nn.TransformerEncoderLayer(d_model=hidden_dim, nhead=8, dim_feedforward=hidden_dim * 4, dropout=dropout, activation='gelu', batch_first=True, norm_first=True)\n        self.slice_encoder = nn.TransformerEncoder(slice_layer, num_layers=1)\n        self.series_query = nn.Parameter(torch.randn(1, 1, hidden_dim) / math.sqrt(hidden_dim))\n        self.series_pool = nn.MultiheadAttention(hidden_dim, num_heads=8, dropout=dropout, batch_first=True)\n        self.slot_emb = nn.Embedding(n_slots, hidden_dim)\n        self.plane_emb = nn.Embedding(3, hidden_dim)\n        self.sequence_emb = nn.Embedding(2, hidden_dim)\n        study_layer = nn.TransformerEncoderLayer(d_model=hidden_dim, nhead=8, dim_feedforward=hidden_dim * 4, dropout=dropout, activation='gelu', batch_first=True, norm_first=True)\n        self.study_encoder = nn.TransformerEncoder(study_layer, num_layers=2)\n        self.target_queries = nn.Parameter(torch.randn(n_targets, hidden_dim) / math.sqrt(hidden_dim))\n        self.group_emb = nn.Embedding(len(GROUP_NAMES), hidden_dim)\n        self.target_cross_attn = nn.MultiheadAttention(hidden_dim, num_heads=8, dropout=dropout, batch_first=True)\n        self.target_fuse = nn.Sequential(nn.Linear(hidden_dim * 2, hidden_dim), nn.GELU(), nn.Dropout(dropout))\n        self.target_heads = nn.ModuleList([nn.Sequential(nn.LayerNorm(hidden_dim), nn.Linear(hidden_dim, hidden_dim // 2), nn.GELU(), nn.Dropout(dropout), nn.Linear(hidden_dim // 2, 1)) for _ in range(n_targets)])\n        slot_plane = [0, 0, 1, 1, 2, 2]\n        slot_sequence = [0, 1, 0, 1, 0, 1]\n        self.register_buffer('slot_plane_ids', torch.tensor(slot_plane, dtype=torch.long), persistent=False)\n        self.register_buffer('slot_sequence_ids', torch.tensor(slot_sequence, dtype=torch.long), persistent=False)\n        self.register_buffer('target_group_ids', TARGET_GROUP_IDS, persistent=False)\n\n    def stochastic_slot_mask(self, mask):\n        if not self.training or self.series_dropout <= 0:\n            return mask\n        keep = torch.rand(mask.shape, device=mask.device) > self.series_dropout\n        new_mask = mask & keep\n        all_missing = (~new_mask).all(dim=1)\n        if all_missing.any():\n            for row in torch.where(all_missing)[0]:\n                valid = torch.where(mask[row])[0]\n                if len(valid) > 0:\n                    new_mask[row, valid[0]] = True\n        return new_mask\n\n    def forward(self, x, slot_mask):\n        B, S, K, _ = x.shape\n        z = self.input_proj(x)\n        z = z + self.slice_pos_emb[None, None, :K, :]\n        z = z.reshape(B * S, K, -1)\n        z = self.slice_encoder(z)\n        q = self.series_query.expand(B * S, -1, -1)\n        series_token, _ = self.series_pool(q, z, z, need_weights=False)\n        series_token = series_token[:, 0].reshape(B, S, -1)\n        slot_ids = torch.arange(S, device=x.device)\n        plane_ids = self.slot_plane_ids[:S]\n        seq_ids = self.slot_sequence_ids[:S]\n        series_token = series_token + self.slot_emb(slot_ids)[None, :, :] + 0.35 * self.plane_emb(plane_ids)[None, :, :] + 0.35 * self.sequence_emb(seq_ids)[None, :, :]\n        effective_mask = self.stochastic_slot_mask(slot_mask)\n        no_valid_slot = (~effective_mask).all(dim=1)\n        if no_valid_slot.any():\n            effective_mask = effective_mask.clone()\n            series_token = series_token.clone()\n            effective_mask[no_valid_slot, 0] = True\n            series_token[no_valid_slot, 0] = 0.0\n        series_token = self.study_encoder(series_token, src_key_padding_mask=~effective_mask)\n        denom = effective_mask.sum(dim=1, keepdim=True).clamp_min(1).to(series_token.dtype)\n        study_global = (series_token * effective_mask.unsqueeze(-1)).sum(dim=1) / denom\n        target_q = self.target_queries + 0.25 * self.group_emb(self.target_group_ids)\n        target_q = target_q.unsqueeze(0).expand(B, -1, -1)\n        target_context, _ = self.target_cross_attn(target_q, series_token, series_token, key_padding_mask=~effective_mask, need_weights=False)\n        global_expand = study_global.unsqueeze(1).expand(-1, self.n_targets, -1)\n        fused = self.target_fuse(torch.cat([target_context, global_expand], dim=-1))\n        logits = []\n        for j, head in enumerate(self.target_heads):\n            logits.append(head(fused[:, j]))\n        return torch.cat(logits, dim=1)\n'Runtime helpers embedded into the V26 Kaggle notebook.\\n\\nThe exact RTAHMIL class from the public report-teacher notebook is prepended by the\\ncandidate builder. This file contains only hidden-test feature extraction, checkpoint\\ninference, and the fail-safe Synovitis blend.\\n'\nRT_START_CUTOFF_S = 5.9 * 3600\nRT_DEADLINE_S = 7.1 * 3600\nRT_IMG_SIZE = 336\nRT_TARGET_SPACING = 0.42\nRT_SLICES = 7\nRT_SYN_WEIGHT = 0.75\nRT_SEEDS = (2026, 3407)\n\ndef _rt_find_checkpoint_dir():\n    root = Path('/kaggle/input')\n    required = [f'rta_final_seed{seed}_fold{fold}.pth' for seed in RT_SEEDS for fold in range(4)]\n    for first in required[:1]:\n        for hit in root.glob(f'*/{first}'):\n            parent = hit.parent\n            if all(((parent / name).is_file() for name in required)):\n                return parent\n    raise FileNotFoundError('the complete eight-checkpoint report-teacher package is absent')\n\ndef _rt_find_dino_base():\n    direct = [Path('/kaggle/input/dinov2/pytorch/base/1'), Path('/kaggle/input/models/metaresearch/dinov2/pytorch/base/1')]\n    for path in direct:\n        if (path / 'config.json').is_file():\n            return path\n    for top in Path('/kaggle/input').iterdir():\n        if not top.is_dir() or 'dino' not in top.name.lower():\n            continue\n        for config in top.glob('**/config.json'):\n            try:\n                if 'dinov2' in config.read_text(errors='ignore').lower():\n                    model_type = json.loads(config.read_text()).get('model_type', '')\n                    if model_type == 'dinov2' and 'base' in str(config.parent).lower():\n                        return config.parent\n            except Exception:\n                continue\n    raise FileNotFoundError('offline DINOv2-base model is absent')\n\ndef _rt_binary_flag(value):\n    if pd.isna(value):\n        return 0\n    if isinstance(value, str):\n        return int(value.strip().lower() in {'1', 'true', 'yes', 'y'})\n    try:\n        return int(float(value) > 0)\n    except Exception:\n        return 0\n\ndef _rt_plane_id(value):\n    text = str(value).lower()\n    if 'sag' in text:\n        return 0\n    if 'cor' in text:\n        return 1\n    if 'axi' in text or 'trans' in text or 'tra' == text.strip():\n        return 2\n    return 3\n\ndef _rt_assign_slots(series_df):\n    x = series_df.copy()\n    x['StudyInstanceUID'] = x['StudyInstanceUID'].astype(str)\n    x['SeriesInstanceUID'] = x['SeriesInstanceUID'].astype(str)\n    x['_plane_id'] = x['Anatomical_Plane'].map(_rt_plane_id)\n    fluid = x['Fluid_Sensitive'].map(_rt_binary_flag)\n    fat = x['Fat_Suppression'].map(_rt_binary_flag)\n    x['_fluid_like'] = np.maximum(fluid.astype(int), fat.astype(int))\n    slot_defs = ((0, 0), (0, 1), (1, 0), (1, 1), (2, 0), (2, 1))\n    lookup = {}\n    for study_uid, group in x.groupby('StudyInstanceUID', sort=False):\n        group = group.sort_values(['SeriesInstanceUID']).copy()\n        used = set()\n        for slot_id, (plane, fluid_like) in enumerate(slot_defs):\n            desired = group[(group['_plane_id'] == plane) & (group['_fluid_like'] == fluid_like) & ~group['SeriesInstanceUID'].isin(used)]\n            if len(desired) == 0:\n                desired = group[(group['_plane_id'] == plane) & ~group['SeriesInstanceUID'].isin(used)]\n            if len(desired) == 0:\n                continue\n            series_uid = str(desired.iloc[0]['SeriesInstanceUID'])\n            used.add(series_uid)\n            lookup[str(study_uid), slot_id] = series_uid\n    return lookup\n\ndef _rt_locate_series_dir(study_uid, series_uid):\n    canonical = ROOT / 'test_series'\n    candidates = (canonical / str(series_uid), canonical / str(study_uid) / str(series_uid), ROOT / 'test' / str(study_uid) / str(series_uid), ROOT / 'test_images' / str(study_uid) / str(series_uid), ROOT / 'test_dicom' / str(study_uid) / str(series_uid), ROOT / 'test_dicoms' / str(study_uid) / str(series_uid), ROOT / 'images' / 'test' / str(study_uid) / str(series_uid))\n    for path in candidates:\n        if path.is_dir():\n            return path\n    raise FileNotFoundError(f'report-teacher series missing: study={study_uid}, series={series_uid}')\n\ndef _rt_sorted_dicom_files(series_dir):\n    files = list(Path(series_dir).glob('*.dcm'))\n    if not files:\n        files = [path for path in Path(series_dir).iterdir() if path.is_file()]\n    if not files:\n        raise FileNotFoundError(f'no DICOM files in {series_dir}')\n    simple_numeric = [path.stem.isdigit() and len(path.stem) <= 8 for path in files]\n    if np.mean(simple_numeric) >= 0.9:\n        number_re = re.compile('(\\\\d+)')\n\n        def key(path):\n            matches = number_re.findall(path.stem)\n            return int(matches[-1]) if matches else 10 ** 12\n        return sorted(files, key=key)\n    keyed = []\n    for index, path in enumerate(files):\n        try:\n            ds = pydicom.dcmread(str(path), stop_before_pixels=True, force=True)\n            if hasattr(ds, 'ImagePositionPatient') and len(ds.ImagePositionPatient) >= 3:\n                key = float(ds.ImagePositionPatient[2])\n            else:\n                key = float(getattr(ds, 'InstanceNumber', index))\n        except Exception:\n            key = float(index)\n        keyed.append((key, path))\n    return [path for _, path in sorted(keyed, key=lambda pair: pair[0])]\n\ndef _rt_robust_uint8(array):\n    array = np.asarray(array, dtype=np.float32)\n    finite = np.isfinite(array)\n    if not finite.any():\n        return np.zeros(array.shape, dtype=np.uint8)\n    values = array[finite]\n    lo, hi = np.percentile(values, [1.0, 99.0])\n    if hi <= lo:\n        lo, hi = (float(values.min()), float(values.max()) + 1e-06)\n    array = np.clip(array, lo, hi)\n    array = (array - lo) / max(hi - lo, 1e-06)\n    return np.clip(array * 255.0, 0, 255).astype(np.uint8)\n\ndef _rt_center_crop_or_pad(image):\n    height, width = image.shape[:2]\n    pad_y, pad_x = (max(0, RT_IMG_SIZE - height), max(0, RT_IMG_SIZE - width))\n    if pad_y or pad_x:\n        top, left = (pad_y // 2, pad_x // 2)\n        image = cv2.copyMakeBorder(image, top, pad_y - top, left, pad_x - left, borderType=cv2.BORDER_CONSTANT, value=0)\n    height, width = image.shape[:2]\n    y0, x0 = (max(0, (height - RT_IMG_SIZE) // 2), max(0, (width - RT_IMG_SIZE) // 2))\n    return image[y0:y0 + RT_IMG_SIZE, x0:x0 + RT_IMG_SIZE]\n\ndef _rt_read_dicom(path):\n    ds = pydicom.dcmread(str(path), force=True)\n    array = ds.pixel_array.astype(np.float32)\n    slope = float(getattr(ds, 'RescaleSlope', 1.0) or 1.0)\n    intercept = float(getattr(ds, 'RescaleIntercept', 0.0) or 0.0)\n    image = _rt_robust_uint8(array * slope + intercept)\n    if str(getattr(ds, 'PhotometricInterpretation', '')).upper() == 'MONOCHROME1':\n        image = 255 - image\n    spacing = getattr(ds, 'PixelSpacing', None)\n    if spacing is not None and len(spacing) >= 2:\n        try:\n            scale_y = np.clip(float(spacing[0]) / RT_TARGET_SPACING, 0.4, 3.0)\n            scale_x = np.clip(float(spacing[1]) / RT_TARGET_SPACING, 0.4, 3.0)\n            new_h = max(32, int(round(image.shape[0] * scale_y)))\n            new_w = max(32, int(round(image.shape[1] * scale_x)))\n            image = cv2.resize(image, (new_w, new_h), interpolation=cv2.INTER_LINEAR)\n            return _rt_center_crop_or_pad(image)\n        except Exception:\n            pass\n    return cv2.resize(image, (RT_IMG_SIZE, RT_IMG_SIZE), interpolation=cv2.INTER_AREA)\n\ndef _rt_load_series_25d(study_uid, series_uid):\n    files = _rt_sorted_dicom_files(_rt_locate_series_dir(study_uid, series_uid))\n    quantiles = np.array([0.08, 0.23, 0.38, 0.5, 0.62, 0.77, 0.92], np.float32)\n    centers = np.zeros(RT_SLICES, np.int64) if len(files) <= 1 else np.round(quantiles * (len(files) - 1)).astype(np.int64)\n    centers = np.clip(centers, 0, len(files) - 1)\n    views = []\n    for center in centers:\n        channels = []\n        for index in (max(0, center - 1), center, min(len(files) - 1, center + 1)):\n            try:\n                channels.append(_rt_read_dicom(files[index]))\n            except Exception:\n                channels.append(np.zeros((RT_IMG_SIZE, RT_IMG_SIZE), dtype=np.uint8))\n        views.append(np.stack(channels, axis=-1))\n    return np.stack(views, axis=0)\n\ndef _rt_try_attached_visible_features(checkpoint_dir, expected_uids):\n    uid_path = checkpoint_dir / 'rta_final_test_uids.txt'\n    feature_path = checkpoint_dir / 'rta_final_test_features.npy'\n    mask_path = checkpoint_dir / 'rta_final_test_slot_mask.npy'\n    if not (uid_path.is_file() and feature_path.is_file() and mask_path.is_file()):\n        return None\n    if uid_path.read_text().splitlines() != list(expected_uids):\n        return None\n    features = np.load(feature_path, mmap_mode='r')\n    mask = np.load(mask_path, mmap_mode='r')\n    if features.shape[:3] != (len(expected_uids), 6, 7) or mask.shape != (len(expected_uids), 6):\n        return None\n    log('report-teacher: exact attached visible-test features reused')\n    return (features, mask)\n\ndef _rt_extract_features(test_df, series_df, checkpoint_dir, dev):\n    from transformers import AutoModel\n    expected_uids = test_df['StudyInstanceUID'].astype(str).tolist()\n    attached = _rt_try_attached_visible_features(checkpoint_dir, expected_uids)\n    if attached is not None:\n        return attached\n    if time.time() - T0 > RT_START_CUTOFF_S:\n        raise TimeoutError('insufficient runtime reserve for report-teacher feature extraction')\n    dino_dir = _rt_find_dino_base()\n    log(f'report-teacher: DINOv2 base from {dino_dir}')\n    dino = AutoModel.from_pretrained(str(dino_dir), local_files_only=True).eval().to(dev)\n    for parameter in dino.parameters():\n        parameter.requires_grad_(False)\n    dino_dim = int(dino.config.hidden_size)\n    if dino_dim != 768:\n        raise AssertionError(f'expected DINOv2-base hidden size 768, got {dino_dim}')\n    mean = torch.tensor([0.485, 0.456, 0.406]).view(1, 3, 1, 1)\n    std = torch.tensor([0.229, 0.224, 0.225]).view(1, 3, 1, 1)\n    slot_lookup = _rt_assign_slots(series_df)\n    features = np.zeros((len(expected_uids), 6, RT_SLICES, dino_dim * 2), np.float16)\n    slot_mask = np.zeros((len(expected_uids), 6), bool)\n\n    @torch.inference_mode()\n    def encode(images):\n        tensor = torch.from_numpy(images).permute(0, 3, 1, 2).float() / 255.0\n        tensor = (tensor - mean) / std\n        parts = []\n        for start in range(0, len(tensor), 8):\n            batch = tensor[start:start + 8].to(dev, non_blocking=True)\n            with torch.autocast(device_type='cuda', dtype=torch.float16, enabled=dev.type == 'cuda'):\n                output = dino(pixel_values=batch, interpolate_pos_encoding=True)\n                tokens = output.last_hidden_state\n                part = torch.cat((tokens[:, 0], tokens[:, 1:].mean(dim=1)), dim=-1)\n            parts.append(part.float().cpu())\n        return torch.cat(parts, dim=0).numpy().astype(np.float32)\n    for row_index, study_uid in enumerate(expected_uids):\n        if time.time() - T0 > RT_DEADLINE_S:\n            raise TimeoutError('report-teacher deadline reached before submission overwrite')\n        jobs = [(slot_id, slot_lookup[study_uid, slot_id]) for slot_id in range(6) if (study_uid, slot_id) in slot_lookup]\n\n        def load_job(job):\n            slot_id, series_uid = job\n            return (slot_id, _rt_load_series_25d(study_uid, series_uid))\n        if jobs:\n            with ThreadPoolExecutor(max_workers=4) as executor:\n                loaded = list(executor.map(load_job, jobs))\n            encoded = encode(np.concatenate([views for _, views in loaded], axis=0))\n            cursor = 0\n            for slot_id, views in loaded:\n                count = len(views)\n                features[row_index, slot_id] = encoded[cursor:cursor + count].astype(np.float16)\n                slot_mask[row_index, slot_id] = True\n                cursor += count\n        if row_index == 0 or (row_index + 1) % 100 == 0 or row_index + 1 == len(expected_uids):\n            log(f'report-teacher features {row_index + 1}/{len(expected_uids)}')\n        if dev.type == 'cuda' and (row_index + 1) % 100 == 0:\n            torch.cuda.empty_cache()\n    del dino\n    gc.collect()\n    if dev.type == 'cuda':\n        torch.cuda.empty_cache()\n    return (features, slot_mask)\n\n@torch.inference_mode()\ndef _rt_predict_checkpoints(features, slot_mask, checkpoint_dir, dev):\n    syn_index = TARGETS.index('Synovitis')\n    seed_predictions = []\n    for seed in RT_SEEDS:\n        seed_prediction = np.zeros(len(features), np.float32)\n        for fold in range(4):\n            if time.time() - T0 > RT_DEADLINE_S:\n                raise TimeoutError('report-teacher deadline reached during checkpoint ensemble')\n            path = checkpoint_dir / f'rta_final_seed{seed}_fold{fold}.pth'\n            checkpoint = torch.load(path, map_location='cpu', weights_only=False)\n            if checkpoint.get('targets') != TARGETS:\n                raise AssertionError(f'target order mismatch in {path.name}')\n            cfg = checkpoint['cfg']\n            model = RTAHMIL(in_dim=int(checkpoint['slice_feat_dim']), hidden_dim=int(cfg['hidden_dim']), n_targets=len(TARGETS), n_slots=int(cfg['n_slots']), n_slices=int(cfg['slices_per_series']), dropout=float(cfg['dropout']), series_dropout=float(cfg['series_dropout']))\n            model.load_state_dict(checkpoint['state_dict'], strict=True)\n            model.eval().to(dev)\n            fold_prediction = []\n            for start in range(0, len(features), 48):\n                x = torch.from_numpy(np.asarray(features[start:start + 48])).float().to(dev)\n                mask = torch.from_numpy(np.asarray(slot_mask[start:start + 48])).bool().to(dev)\n                with torch.autocast(device_type='cuda', dtype=torch.float16, enabled=dev.type == 'cuda'):\n                    logits = model(x, mask)\n                fold_prediction.append(torch.sigmoid(logits[:, syn_index]).float().cpu().numpy())\n            seed_prediction += np.concatenate(fold_prediction) / 4.0\n            del model, checkpoint\n            gc.collect()\n            if dev.type == 'cuda':\n                torch.cuda.empty_cache()\n        seed_predictions.append(seed_prediction)\n    return np.mean(np.stack(seed_predictions, axis=0), axis=0)\n\ndef _rt_blend_synovitis(primary, teacher_synovitis, teacher_uids):\n    result = primary.copy()\n    primary_uids = result['StudyInstanceUID'].astype(str)\n    teacher = pd.Series(np.asarray(teacher_synovitis, dtype=np.float64), index=pd.Index([str(uid) for uid in teacher_uids], name='StudyInstanceUID'))\n    if teacher.index.has_duplicates or set(primary_uids) != set(teacher.index):\n        raise AssertionError('report-teacher and primary StudyInstanceUID sets differ')\n    teacher = teacher.reindex(primary_uids.values)\n    if not np.isfinite(teacher.values).all():\n        raise AssertionError('non-finite report-teacher prediction')\n    base_rank = result['Synovitis'].rank(pct=True).to_numpy(np.float64)\n    teacher_rank = teacher.rank(pct=True).to_numpy(np.float64)\n    result['Synovitis'] = (1.0 - RT_SYN_WEIGHT) * base_rank + RT_SYN_WEIGHT * teacher_rank\n    return result\n\ndef run_report_teacher_synovitis_specialist():\n    if time.time() - T0 > RT_START_CUTOFF_S:\n        log('report-teacher skipped: the primary ensemble used its runtime reserve')\n        return False\n    checkpoint_dir = _rt_find_checkpoint_dir()\n    primary_path = Path('submission.csv')\n    primary = pd.read_csv(primary_path, dtype={'StudyInstanceUID': str})\n    if primary.columns.tolist() != ['StudyInstanceUID'] + TARGETS:\n        raise AssertionError('primary submission schema mismatch')\n    test_df = pd.read_csv(ROOT / 'test.csv', dtype={'StudyInstanceUID': str})\n    series_df = pd.read_csv(ROOT / 'test_series.csv', dtype={'StudyInstanceUID': str, 'SeriesInstanceUID': str})\n    expected_uids = test_df['StudyInstanceUID'].astype(str).tolist()\n    dev = DEVS[0]\n    features, slot_mask = _rt_extract_features(test_df, series_df, checkpoint_dir, dev)\n    teacher_synovitis = _rt_predict_checkpoints(features, slot_mask, checkpoint_dir, dev)\n    result = _rt_blend_synovitis(primary, teacher_synovitis, expected_uids)\n    untouched = [target for target in TARGETS if target != 'Synovitis']\n    if not result[untouched].equals(primary[untouched]):\n        raise AssertionError('report-teacher changed a non-Synovitis target')\n    if result.shape != primary.shape or not np.isfinite(result[TARGETS].to_numpy()).all():\n        raise AssertionError('invalid report-teacher blend')\n    temp_path = Path('submission_v26_synovitis.tmp.csv')\n    result.to_csv(temp_path, index=False)\n    reread = pd.read_csv(temp_path)\n    if reread.shape != primary.shape or not np.isfinite(reread[TARGETS].to_numpy()).all():\n        raise AssertionError('serialized report-teacher blend is invalid')\n    temp_path.replace(primary_path)\n    log('report-teacher complete: 0.75 Synovitis rank blend; all other targets preserved')\n    return True"},{"cell_type":"code","execution_count":null,"id":"eb7f8204","metadata":{},"outputs":[],"source":"import base64\nimport gc\nimport hashlib\nimport io\nimport json\nimport math\nimport os\nimport random\nimport time\nimport zlib\nfrom concurrent.futures import ThreadPoolExecutor\nfrom functools import lru_cache\nfrom pathlib import Path\nimport cv2\nimport joblib\nimport numpy as np\nimport pandas as pd\nimport pydicom\nfrom scipy.stats import rankdata\nfrom sklearn.ensemble import ExtraTreesClassifier, HistGradientBoostingClassifier\nfrom sklearn.decomposition import PCA\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.preprocessing import StandardScaler\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom transformers import AutoModel\nHYB_PREFIX = 'v8_hybrid_dino224_6slot_5pos_radiomics'\nHYB_EXPECTED_TRAIN_ID_SHA256 = '21c1944bd15c3397290f0816de614ad4153f62e84c4bfb0e4d6147ac72084af8'\nHYB_TEACHER_PAYLOAD = 'eNrtXX3MnWdZf2dlNCNTiuNDx0fZnC2kA3RTSEvPmdoxoHN2M1W2lJQy2m2wvqvrRs8YTCBiKhuCkJB0ETMDhmUYdBETwkjWTBfBDEOEZAQzAkaTRUnbYEZcMNHze8/5nff3XO91P8/99cL+uN/l2jnnee7nfu6P6+N3fTxP9+ze8NMXL83+Xr70wy+9+Zr/m/89b+m8pZsP77/+luWjB5eP3n50/x2vWj5yx1lLz13ad/msPT8/cdXe39pz7VlL7166c8s7Dh69/tYt2zdvef2hS7ds27zl0C233nbrgeX9t9z6joM4/oYDNx89OD1+9MYDRw5Of2+99NLXvHbbK7Ztft/m3L9zPvfAK0egcx6/8PVK59727cX3zxzZOZ5/3/mZI+8baztc+3cfuGqsv0FsN+1nh/S5A/Scxy8cm3uttHnj996zcvzaH9y9Zjxsh75v2vCS8YkTP7/S9mOP3D7G7127do3/+Pjx14Mwdny+5pPvXTmGcfMcCH3g823b7hjrcUucC77fc9HDIx7ndz3P37x3H33gqvNX7os56DjYF8atY9N7gDBf/f2i3XeN2WeozdD8QFh/fsd64hzGwXUEffmhHYP9aj9Yjy8/dOcY+8RjmLf+/s4Ta8dq54M+ME899vinjyx+h/bS9oO5oC9vbXGcazI5dmwMvsZx8PJ8LjvY/vNXvHys1+Kc9mc/wdMP/vsLx7o2obUPEdaNa4D7YVz3XPQH098bV/j7a49dOeac2Z/ygeUjPUZeRB/YD91z0L6P/sZCxrSf48ffOP7m2e8ZKz+dOHFt555c19C9Y/jU479rvvjSTr+qQ7gfmD9+8zzW5sVPXjrmd0/PWJ2De1qdZ9uxf69fttfruI/kRRzDHnAPOe+h8aEN+rLH0S+/Y490f0DYXzs+rie+Y0/ZhscgE/gEz+m8Vefhu5W57Yc3j/v0jhL7uv6rr+6sj0e2X+07pA/QP85bOaQOwv26+mgmW1b3oB3WyM5D23Hsuh579z5/fMklRwdlHW3+44W7x2rHMC4Q9oHyOrvPQ6McudL5rdXf3eN2/p5uBb1i38+u8A3HgDYYr+i4kd6D7fCb+0n88In921b4jPbekwfw9vVf/esR23GPqQso/zyPT4xPZVRlBQTeufCCt3eOoT10OIh2APPCPTw9orhHz8NmUR9qW+oA8BTbaX+4j/ZpdR3ugT7xSZxl26jMcx3wSbsEUnyH4zoHjPGJJ9662CPwBIi8KPZ4ZGWA/MN+0ni0a5/1GNZLeVPlgrYopEO0r2dvP9wZ33SeI/An9QT4Wq8Fj1j+tzZ/ev0CC6u8qWzY63EM8wnZS9UF9ryVUYx906arp/L32uCa79r1DyPIHdpR1+A3ZYlr9/Bbbh336RhPp1PO/LWZjVf3hvwKvqN8sB/Kpdojz+6R6B+oXcax71/9xRHPqTwor5PuevAXxypDaP/Fl35oBKIN0XnrvlHncZ+hy3huNueZfuX82R/a8rzyDTFWP17cuBPXq/4FYW66F322gG24NqpLPbwTwihoD56x/qTuLXUOjoEshrN+qO4rCPsAuaTcKJ7h3lgchXmde9t1nWN2DjgP0t+8L+6jtohtQXP7NVKc5Y2bfes8ydt2THbPdV+p69SGAj/y+PxzBF/LyuaFF1zg6hjVn1av3H/ubeM+3yR0jGvx5Bd+fQ0Pgse9az3dR56nX6a+DTCp7RvtT59aXtFl+H7DoUOdOcOWx/rKSr99/o1jYjSNefC7hweInZUfLG972N47hv6/f/Vk7J3DuBhfsfidGIGY5+KnL3fXnnhIdSeO0Q8KyTwwhmJyjof3JiZBX1wjPW7ldOg7rqPORT/gJWIf9XHoK/P4PE6zQ3EXf1O++ZvX4BM6auYP3tk5z2OQD3stiNhNx0EfHd/pLyvu9rABrrV6H3qDc9r29OUjXsf4ivoGGvvAdbPYgW/PcZ+lpfePZuNZ9QXQP66d6YSuj0Dsgu+77/rllf2wshDyudXGWt7T48SnGC/wpupQ5Z/QfbQNdYbXzvKvHlfetfdj/zcc+qWx2ri+tn2+g2d7bQw2RIofXvahd3V4x/p41EUhrGZtkSW9l/bDuc78+o07OTdiJN1vxrXgY9Aean8ejp/ppTvHqb4DcO/evXs71wEDa0wxhrzYK/r43APvHFts9dhjj408DIaxdX2Ernzr3DVWzPXlvuKT2CRkk/v8IOwHbKL22eenefzLfdu0aVPHx7VyhHOM85O/tT2PcT60abZP5CaUz+yYqLsxt79/6ka3j5Bfw/Wjz8Uxe/gytN7gZU9nc780zsrr4dvpPg7FhtHe2lKNK6jNUyzKvdJ4QUgH8Rrawz7Mwmu5VsQTVm9xjnp/b36QU8Z9mZfQ/WJMIkRcb5t3GYovrvpi4ZxESD50zyjb6iuqf6jXkB+6mHSmpy2/AW949+e96dMzVhOSYb0X2+n4MaahPIIXk7X87cUOGZvmenCv7Dh07W2+TM+H4r4cB9akz9fz+Xl2PpSXtD4AY+m2zWen+yVxyg628PSWvf6+b7xhTAxvY0I8NvVNRjH+mfYR2qO+nBH254KpD3nThreO+/wn7/rD9/3cmLrpqa0vtnplp/GR1mBp8Bi/sw32dYa/r+34AMTqxNqKxTUuyj70XpBb+MmUYeoq1VlT/3Kh71axznULnKc6Wf09tLGxaW/PEa+yMWKLg0MYkvEuxkrYFraZ56mbsZ+nT72qI1dWZ1xyyd868YLV+GpIL1i96ekiGwvkdZbXvZytd5/5ni5428YK6Td6a8h1ZtxEMU1frLFnL3pxOmJ7jPvTLnM+6Jt8hT3UGBrmYG2qnadiAOUL9blhX+31nm6i/2v9EYsJRD4W/WiOhbjI84e0H82Ngjz8pvEVXk+/Vvv7x4/vHzNvqXUmmj/lGPlJ38TyL/mSuVr0BwxGP0/b4p6SD9gZ0o0cq83l4beHFULYALzj5e8sbrH40Y5JMe/s+6w/5iA1Tzmzq3868saltsXLWWi+27PXwAfcT7s2mKvKlsaaLI9yz5gvoj21cYA+P4V7aeN8+NT8gfKnyrWNUfN+j378ZWMbW7T7hPVA7tjabZvH5riYrxzK13r+K/vlHmCdiR8wT6tzGXtjG6036PJhmP/Jt4rf8J15AXzO7NVDI7/vrswN5bk9W+XhyiEfmnoHY7d5+T5ZD8VPVmtoZv1ojIL71JPH2akxCNpZ2G3NV1p/oC8v5OFpi7spI335JF1bi+V5Dp/w81b7W9U5WqNBfchx0M8Zrp+6tiM/qstsDo++XV9ew9OhXl7UttXaN8bCKVOH7/udNT6mrq3nB82OhfX9ECbXPi3mCuGuFF/B4yG1Bdb+WB+wT4ZwLbAEeUT5Tfmd/MN6FK43Mb93H7UHqmdpXyy21FjH/XNfj33bfAPjR16uxYs3q49x/rx+DbpQ56z+js7J+qgWV6nd7MOR3m+uM+0ZeNur+wV/2ryNnacXxwDvq3+r+CQm9jKc8/b9aNuXtU2+jVnNKdo1DvkauubE9+pTYEyaO6ZOwvc7z1t2eYc1ABpLtXlJ5iNDNRSWd/AdvqTudQwRS9g14JiI3+in6hg1f8k+9DxzKxaPcb9sPsPiT1uLotiLet6r4evq5x1jrQ160e5RRxZjdG6Aj0ceb9GnCsXzQE8/es1CJtV34LVar6mxa1u3TVyCe0lt3mC+PmSXpf+R5S29t+VBW7emcsC1lFr/xbXATqobrazYcSiP2pyzzTFQzlWvYs3AY8T8xMVWxliXpmNFXFvbwCdwamEW+WPaDdYjahyAMU7Vmc/efvHCJ2C8y/Ox6evAr1PfpS/u4ek1XMu8BH/D1wvli2JiqrZ/jZ2l5Oy4l5RvK39eTsDiRtt2KOfG+6nsPrV131jr663u4XrxetSycN/Zl/KoXq9rqHts15Nyxz6AP7En4Cf1Sbx9ok3HOa1vWavPPN+re4zyFMpnzOrfwhi3rzZJfViLLRU7496sJwv5PrYOdFaL87pxyDeybYkbQvofe9AXj+/jafUrvJiR6kLWhHrz1H6sn09Zs9gafWi+SXU65oP9ffzTrxnwj7uxLBLqtrx64r5npEAXBGrcQn5IKEYVyr+i7Zl5zBzj0pxwyB/0nlWhDtNYFu2P5kjVDjB3oDVN+mwG7b/mPFafZ5k9S4Xv2GuNEdt8i/ouoHnMjc9BDcZ2+Ok986C+puZGV+tkH1rzfB3l1uIf8iP3Ss97fqNtT73K85ojUh6nHPDZA/tso+pYrpXNHbIt9xi1Qtg3taeg3Xf9fiefv1oTs+pLefZU+crGQT0sBb5j3YfXZ99zUorL0Q/jYTbuqjZD+7N2BXuuzzJgH+eyscPWu+AY9sCrVVHeJS95NXU8r3FQHoNswC/xdcxqXRxjknpP5GH1ecHYWDzGQFnBnnzskV8dD8VwkaNkzLXPN/CeYWIuyfaP4zafxmdp//WyLYs9g18QU7OKMXrtQjk14lius5efAz7SnA5zZZgnMbtXH2vryuyYsP7Q31hTG7NfrZ9drYW1sRG7DjtkjWx+i7/5XJDmJUM4WHWQjcl4z2dLDmvkxXLQBnl4lWX1d/h8FdeZfdGHod5ivb/qGMyH2FBzh5wTcs7esyOe3JAfUQvW91wpeAL3VAxEmRquiVt9Psc+X2ePs/+Zz+9jU4vzwnV1/fFbvc7Lf2v9oH2+cejZPi/mydp2lZvV3KI+U3NtJ94HvaH+LuMGuA58RJ1IfaW5Y+Vz6N53bvjUiPoJvo/nD2JvVO+qv9SHN7jmnA/4gjpDcFzwWQvNj3o8+M2ztwdrHhiveM7j16/Um67y5Gq+xPpxKvu2VgTvI7C6xasR1mdWQnWZWE+vFlNjcNbue3XH+hv2wsN3luhbsCbN1qYN1Z/BT0B7XWvrJ4RitvY+6ut49S3gX1uzS54QHdObfw3xjvpW6PO/7/7LkZeXYRvmCLV/2kNti1qlGFygse2pjd0JCmH8mGcr+3Qi1tDvo+sLhuZv116fK97TeT/L6YNvejffz/J8+36W6bdDz9RXtDzwwCvHIIa7STC3/P6xR55aEU+W+Svh2nfOH4Hib+2PIXp+B23b99GR9sE2UMv4vOr8J0f2Pmy30vcU/n7lsStX2mzGmKa/jx/fOD42NZkgjB2fEE18Ytw8B0If+Dw5bxcizgXfn/zCB0Y8zu96nr8fHugTtPG261auxxx0HOwL49ax6T1AmK/+xrqwz1CbofmBsP78jvXEOYyD6wg6c+rUYL/aD9YDUGaz7AHmrb/PTFVHaI1IfJxDj33+igOL76G9tP0wnOit7UTWZDqmFb4+Nn8Mn+vM9kin6bUcG4/ZT/D08YvGI12b0NqHCOvG+1BmZqWVyyvH7p2qZM6Z/SkfWD7SY+RF9IH90D2fzfdZY8qY9vPcqYuF8Kjy069cckmn/8nAvWP41OO/6Rw7e6A6hOuE+XP/8Im1OXLfN0b87ukZq3NwT6vzbDv27/XL9lYvkud4LfaAe8h5D40PbVTHkiYyHuyR7g+IcFvHxfXEd+wp2/AYZII8p/NWnYfvVuZUd3p6R4l9AXbq+nhk+9W+Q/oA/eO8lUPqoInRR5Qtq3vQDmtk56HtJo4Oun8KHQ5NoeQQz6PNP736BR07hnHNaSGvM119fJQjVzo/q7/tcTt/T7fOrn1qJ8bLMaDNmXlf6PfUXH55D7bDb7Wr4K0r73pwJkdze+/JA3h7++H7Fu24xwtdMJd/nmephcqoygoIvPOGXbs6x9AeOhxEO4B54R6eHlGZ1POwWRMj2wtdNZc7ttP+cB8r56rrcA/0iU/iLNtGZZ7rgE/aJbYl4bjOAWP80kM7FjzG+5AXaY9PiY6eGP5hPyk8au2zHsN6KW+qXNAWTSLsys/cdl1nfM+b8tXp+f6Sr/Xak46eszYfYTzus8qbyoa9nvsXspeqC+x5K6MYO9JU333iieCaox/IHdpR1+A3ZYlrd9eD7+i13Z5Op5x5a8Px6t6QX8F3um7k05NzueY5z+6R6B+oXcaxrVPfg+dUHpTXSf/z6MdHKkNoD18IRBui89Z9OyP6E2sAXcZzE9G7E1k/6hKeV74hxurDi7gG16v+XbEP07npXvTZArbh2nR0qYN3QhgF7cEz1p/UvaXOwTGQxXDWD9V9pV8KuaTcKJ7h3lgcxZJgPWbnwDb6m/fFfdQWsS3lBrpPcZY3bvat8zxp5mn1Bvdc95W6Tm0o8COP4xPjga9lZfOeKWbw9IHqT6tX9px/Y69vEjrGtfj6TRvW+qnztYnxAcjzC79MfBtgUts32n8Hj0hP2+H7Pz92ZWcdYMtjfWWlP/+1PQuMZmMelhdJxM7KD5a3PWzvHUP/35q/nsKew7gYX7H4nRiBmOe6HzzHXfuJ+LyKkegHhWQefKKYnOPhvamvJ3OdaI9bOR36Tvyha67H2FYfOeFesR3Hx9+Ub/7mNfiEjqLN0vM8Bvmw14KI3XQc9NHxnf6y4m4PG1BnWBzCOUEueB3jK+obaOwD10FmQvYc91laWloZz2ljy3Atxm19BO7lCg8+/ajrY4d8brWxlvfsvpNXgDdVhyr/hO6jbagzvHaWf/W48q69H/v/sxMnRmrj+tr2+Q6e7U0h7s3onN/r8I718aiLQljN2iJLel774VzRP+7HuREj6X7T1sDHOCnj4eck4BNMMnwH4F7L88DAGlOMIS/2ij5+d+/eNdjq61Pd4GGwSc/6Mo4xMb8p8xrrwyexScgm9/lB2A/YRO2zz0/z+Jf7dvz4xo6/buUI5xjnJ39rex7jfGjTbJ/E4iGZou7G3P7ov64ee32E/BquH30ujtnDl6H1Bi97Opv7pXFWXg/e1H0cig2jvbWlGldQm6dYlHultimkg3jNIgcTEY/kWtn4ivaptjgkA7NYyyymwLyEnmNMIkRcb5t3GYov0hfry0mE5EP3TPWh5x/qNeQHxaTU05bfgDe8+/Pe9Om5xkEZlnuxnY5/RUcP5BG8mKzlby92yNg014N7Zceha2/zZXo+FPflOLAmfb6ex888H8pLenlPxNJtm6um+8U4JGNkIazvydhlnz13geFtTIjH3jLF9DH+mfYR2qO+nNEKHp5iy385e3uv/+Rdf+OGlyxw4v1XHOjoFc0xh7A0eIzf2Qb7CnkAllbcTqxOrO1hcf4zFOhD74Xz98zxs+o21VnQR2xLG0R+wW/VyervUaY0Nu3t+da53VNfZYhP2J7xLsZK2Ba2mecn4v+gPEf3zOqMe6eY1u6lxldDesHqTU8X2Vggr7O87uVsvftgfR+W3LVn37y8nGIVxk0U0/TFGkN7MeTnI7bHuD/tMufzlTlm1DoMxtC8+Kc3T8Vq3Hf1ub16DU830f+1/ojFBBwbc3k2x0Jc5PlD2o/mRkEeftP4Cq+nX6v9/clF40XeUutMdB0nZqwqxx5fMleL/oDB6OdpW9yT/Gr9ZcvbIJvLw28PK4SwAXjHy99Z3GLxox2TYl58Z3/MQWqeEp/fmecY7Li6tSNrcxaa7/bsNfAB99OuDeaqsqWxJsuj3DPmi2hPbRygz0/hXto4Hz41f6D8qXJtY9S839ZPvndkY4t2n7AeyB1bu23z2BwX85VD+VrPf2W/3AOsM/ED5ml1LufFNlpvoHzYx//kW8Vv+M68AD7RP3nA9m1lbijP7dkqD1cO+dDUO3MdEGzv2TEvfsI9tLF93b9QHgexbo1BUPfDbmu+0voDfXkhD0/bMVFG+vJJurYWy/McPs/M8yfoT3WO1micMTqPfs4QAavYGBD7tDk8+nZ9eQ1Ph3p5UdtWa98mxs/89mVb1uhqXVvPD8KxPn0/hMm1T4u5QrgrxVfweEhtgbU/1gfskyFcCyxBHlF+U34n/7AeZSIxv9B91B6onqV9sdhSYx175r6e1usphmf8yMu1ePFm9TFuntevHZ/n8HSeE3Mvi+PY5pjBY9Y+DcWD+JvrTHu2ee6/2TjUacl/efGmUAwFvK/+reKTmNjLUM475Efbvqxt8myM5hQnztp72FXXnPhefQrNPfN6/v7h1n0u77AGQGOpNi/JfGSohuKYwyPwJXWvY4hYwq4Bx0T8Rj9Vx6j5S/bRXa/bO30rhuNxqytC+gTroNiLen4ywEewV1ob9LZtT3dkMUbnenx8yqmzBm89HNCNSj+4+6KFTJ4xOR5br6mxa1u3rblRxvX+6oEHRkOYKGSX2b+NfXnxUuVBLz6ntpn8bq+1utHKih2H8qjNOdscw6J2QPTqwyv4ZXlEzK85aXsPYBEdK+JInbjP1CewuT+NWdFusB5Rr2WMU3Xmi5/8wsInYLzL87Hp68CvU9+lL+7h6TVcy7wEf987fw2Qly+Kiana/jV2lpKz415Svq38TQZr0JfX5A+Gcm68n8ruf179uk59vdU9XC9eD3w/MTUzyqPWn1BMEqrXWjwnMO8D+POk8ak1BqZEmz7D7ctr6lD6fC97jPIUymdgv/swbl9tkvqwFlsqdp7ZmVk9Wcj3sXWguP5vrvjeKOQb2bbEDSH9z3xEam0s+jDP1ATjsGj7XYnX2HluNs9Ldfz8uaxZbI0+NN+kOh3zwf7u++gjoz7/2MaySKjb8uqJ+56RYrxoyGexmMCLUYXyr2h7ch4zn5j6vJA/6D2rQh22Jj9gcqQdOzDPHWhNkz6bQfuvOQ+1HcTg2GuNEdt8yzFT68TaH7TVeEMovsJP75kH9TU1N8pPu38at7D4h/zIvdLznt9o209EP+qc1Q7R16FdR5zL82m4F1wrmztkW+4xaoWwb2pPQZ/837d39p06+LSpJbT2VPnKxkE9LAW+Y92H12ffc1KKy+f4Z00dlv5mTChk/7Hn9lkG8LrW+ekeYw+8WhXLJ+zXy2PaOCiPQTbgl3h8pHVxjElqX8jDhur4+7AGxkBZwZ5cds7jgzFc5CgZc+3zDbxnmJhLsv3juM2n8Vnamzd8arF/K88KRNSsYoxeu1BOjTiW6+zl5+6d/5P15EPmyihbXk7dxnU8Hx3rD/2NNbUx+9Vn4VZrYT0fWu/9fVkjm9/ibz4X5NXXWBysOqgvRmNrf0/Nx+z5I8jDqyyrv7N4vmq+zuyLPgz1Fuv9bY2uxhrUXwch5+w9O+LJDfkRtWB9z5WCJ3BPxUCUqSHfVZ/Psc/X2eNnFjmup4PxV4vzQnV1Q/Fbvc7Lf6u/bJ9vHHq2z4t5srZd5Ya2RWWGPECeYo5Y9QPvCz4if1Nf2Ty8xhFRQ0z9BKzh+YPYG/tMdEhP2ti2Pt8FvqDOII7TGIzNx2h+1OPBPSYfYvMtuP4Fn79iRe7Zt+ZLrB/XscWmVoQ6eCgu58U27D2Oz/W0rcXUGJy1+0MxQNgLD9+twcRz34I1abY2baj+DH4CscKxgJ8Qitna+6iv49W3EPt5OWvup43hefijT88xJv0L89hNKL/iPU9Ie9jVYxujcIHGtqEHqDs9jD9JyAV566AxqL66htD87drrc8XdV7R868MnjvAVLZtmr2g5ctPN7zp2YPmG/Xc8U1/PshT19/6x/z312NC5vmtsO7bl99QxpNwvZg1S/nTsXn+h+YTu77XPGaM3R9t3TL9D44vpM3WOsTzVt7ahMebw1hDf5vQXu45DYx/iydBah9ZhiPdK5aVEF/XNI3VtcsYYqwNi5D7lfqU6u29fU/VqyZxy7EjJGGNtSUy7kj32+FRls6Y9yrXnMfogJG8pNmJIb8ba0VibFmtP+7BHilzF8H0Or8WseSxf9fFyjJ1M1bkpfJm6lznrnttvKt7N5Y++MQ7114enS/Yt9t6xspdiV2LkNdZGxt6/T3aG5Cp1fDH6MFVP5a51yt6H5htrz0p5I1f/xNqimhgtF9vGyG6qTiqdX4q85/jNQ3xf6nulrEeOX1Jbr8bop5S4Sar+KPGth3RkrFzmYOQYPknFgrE+eow9irVBQzKSGmOKnWdMHKFEpmKxdE48pUT+Y7B1rXhCDu78SfiMMTYjRdfU1Eultiinn9TjsRi0JFabinFqxG1yMEeu3ijd29rxkJg4ROo+lsR6a8Y018unKI3Fp9rZlNhCrO5N5ZUaWCvHpyqNS6fa59TPXB+lRvypVgy1RLfm5qJKYuVDOLAEz+TGcFLxZkqMspY8pmDsVPxUgm/71i8Vd6bGv0t8mxw/scT2pGKQWv58Ck+XrlmOPNbIUebm7kvsbWyMP5Z3hvIntXy9VJxWSx5r+JE5OGgoThIbo4jtr8SHTIlDlMS6avmZKf5rjJ3weL8UT6TEk2rq41LfoobfHIPhc+oHYnNQufPPjZXFxCRr2PKUGFAt3z/FRyrJMdaqR1wv/yg195XD5zE6vURXpeZpa8ZshnInpXnEEj7IjTmW5KJLsWCuz56Le0pjOKny0edPlMYA16MOINZOpGDRHJ7J8f/Xy66WxnNy9FtszUyJva0Zn4rBDTXwWE3cFbs2OfVoOf5vig9VIk8lscv1wCy1nhvJ8UlL6mFr1cIN4ZrUmqz1jKuX8uIQTiyJV6TUIOTYk5oykVpTE7N+qb7Uej2XVIoxc2NUtZ6TysF4Of52Lj74ScZWcuxiSU44VYfUiI3Ueo6qVs469ZrS5xZS4lalz9Pk4P/1kIlYux+rm1P3Mac2pSROUAv3D/FKKR6pXTOcUhNbysu54+3DRrHxp1QfPXcdYzFUrVhpSU1Jrbrv1DHUeMY+Vkemxp5q2ueSGvyQvqtRvx9be58TVyzBrKUx0Bjer1WvnBufzMUHKRg5Ny67HutQs2avlm1LmV8NjLV2DbqvZ/nIB3+0ma9nOc+8nuX6W5YPPXPf0OJpi0aNGjVq1KhRo0aNGjVq1KhRo0aNGjVq1KhRo0aNGjVq1KhRo0aNGjVq1KhRo0aNGjVq1KhRo0aNGjVq1KhRo0aNGjVq1KhRo0aNGjUqo+4bWvbccM/X+IaWTUvnLd1yYPUNLXc8c1/PUvJS5BovC6z1wtdQ333nar44NHRex7FeLxasNceYF0wP7VPMC8qGXqymbe33mBfzhdqF5lTrJU+pLwAsXcPQS1h/HPyeyt+pa17ycuOY/R5aq5Rrcl8qWmMNQvJR8jLZ1Jcaxq5LzZeupvBbzBiGdEPOmGNkI0YvpvBALf1f6yW3pS9+TrXtKbYl5eW5MTaoli3pk7PUFyfH8FGKDomZS5/urP0C6px5p65xyUslY3RjzD7G6Pwaspi7R6nYvvQFrjVfihzSMam4MUVnp9rbFL2Vuz85e1ZLdnP4svZ65/hrNe1syUt+c7Fg7guaY/zzWBlLxaOpPBOy5zH9pPoBJZjtx4HXUvRqzvhLbNDQnqTGpmJxVw5mL9WRpVhuyIfwxpmiu2JtbqndL8ExsfwQ6wfUlMla8YBcfirFmLX0TI6vlcKrqT5YKs7O2duU2FINHypGZ6bozhyMUxLfS4mpxeYESuWmNHaZEp/MWau+OHktn6+WTivF+Km4oFRX1Ip9xY45Zl1T55gy11jZyol1xWLzoXvG7kNsrCzXh4uNA6TKfWrMrCaf5mKCFB1UEyfUil3UtgElcZdUnVVDZ6fmi2JxTw7uyslhpMhZKvaI0SW5uf3cdQthxxSdG+Mn19CRKXa8xI6WrnWfPNbQbaV1HzWwW47vVzOHXeK7x/iwKfJe4gesh1zExoRq4YOcOF5JDChVP5RiotS8eW2+Tr1fbXwRE//M8alq6phUW1+yj7m27ceNp2vEm2vU6uXEFktjBTnYp8Y8c/O5sfn9Ut2ZE5erFauqXV9VgiVKMNCQz53jl8TY7NR65aF1Ts395uYta/ucpTG7XPueWiMX6xfl1sfnxsVy6xhz8FefbsvB1amYIGZdc331WFkLUY4eza35yhlzrO+d029OjUYsbsv1g3PjwjnYoXZ8syTvmCNDqbJTC9elYLmceHlJbeaQTMSudUpMK7a2uzZ2KvF7a9SMlGLWlNxwbkwiR95Tashic1Q19jzFFqTMr6Z+WO+agvXyPfpijrl8HRuDi8kT5dro3NrqnOtT88mlccGaPBkTP8v1n1L8kRx7VPJMY2zt7HrV0MVilpoxzRKbWjuuUCIvqXKS6q/X2POSGrDcnHOuPU2Ze61nVnJr8FNkN1Xv1rBbKfKynjHDGH8gJaaxHs+Pp+Cdms9KlWDz1D2pMc4YnyxVP9esQcjFiDXmHavTcnzknFxI7XcrrP51X8/ykQ/+aDNfz3KeeT3L9bcsH3rmvqGlvWinUaNGjRo1atSoUaNGjRo1atSoUaNGjRo1atSoUaNGjRo1atSoUaNGjRo1atSoUaNGjRo1atSoUaNGjRo1atSoUaNGjRo1atSoUaNG9WnP7rN+6uINq+9o+eGX3nwN3rsCep6+2+Us/P/mw3hRy9GDy0dvP7r/DryvxV5/+uCb3s3rn9+5/r7LzfV85Yvt4lsfPnGEXWzqdPGbd690sXhpjDsCvGWGl5/Xufzfvt69PDSAPTfc8zV/AH/4grM6b61JHMCF+85yX3qzZ/ezzkaLs6f/fWi60I/+BVb7/wGZqKOF'\nHYB_START_CUTOFF_S = 7.2 * 3600\nHYB_DEADLINE_S = 8.72 * 3600\nHYB_IMG_SIZE = 224\nHYB_N_POSITIONS = 5\nHYB_N_SLOTS = 6\nHYB_SLOT_META_DIM = 8\nHYB_STUDY_META_DIM = 13\nHYB_RAD_DIM = 14\nHYB_RAD_AGG_DIM = 56\nHYB_SEEDS = (20260809, 20260810, 20260811, 20260814, 20260815, 20260816, 20260817, 20260818)\nHYB_LM_FAMILY_WEIGHTS = (0.5227272727272728, 0.27272727272727276, 0.20454545454545456, 0.0)\nHYB_FAMILIES = ('lr', 'et', 'hgb', 'exact_lr')\nHYB_FAMILY_WEIGHTS = (0.46, 0.24, 0.18, 0.12)\nHYB_LR_CS = (0.015, 0.05, 0.16, 0.5)\nHYB_EXACT_LR_CS = (0.01, 0.03, 0.1, 0.3)\nHYB_TARGET = 'Lateral Meniscus'\nHYB_OA_TARGET = 'Lateral OA'\nHYB_OA_SEEDS = (20260809, 20260810, 20260811, 20260812, 20260813, 20260814, 20260815, 20260816, 20260817, 20260818)\nHYB_OA_FAMILY_WEIGHTS = (0.5227272727272728, 0.27272727272727276, 0.20454545454545456, 0.0)\nHYB_PLANES = ('Sagittal', 'Coronal', 'Axial')\nHYB_CONTRASTS = ('Fluid', 'Structural')\nHYB_SLOT_NAMES = tuple((f'{plane}_{contrast}' for plane in HYB_PLANES for contrast in HYB_CONTRASTS))\n\ndef _hyb_required_cache_names():\n    names = []\n    for split in ('train', 'test'):\n        stem = f'{split}_{HYB_PREFIX}_'\n        names.extend((stem + suffix for suffix in ('slot_features.npy', 'slot_mask.npy', 'slot_meta.npy', 'radiomics.npy', 'study_meta.npy', 'sex.npy', 'ids.npy')))\n    names.append(f'{HYB_PREFIX}_ipca_192.joblib')\n    return names\n\ndef _hyb_find_cache_dir():\n    local = globals().get('HYB_LOCAL_CACHE_DIR')\n    candidates = []\n    if local:\n        candidates.append(Path(local))\n    root = Path('/kaggle/input')\n    if root.is_dir():\n        marker = f'train_{HYB_PREFIX}_slot_features.npy'\n        candidates.extend((hit.parent for hit in root.glob(f'*/{marker}')))\n        candidates.extend((hit.parent for hit in root.glob(f'*/*/{marker}')))\n    required = _hyb_required_cache_names()\n    for path in candidates:\n        if all(((path / name).is_file() for name in required)):\n            return path\n    raise FileNotFoundError('the complete public hybrid feature/PCA cache is absent')\n\ndef _hyb_find_dino_small():\n    preferred = (Path('/kaggle/input/models/metaresearch/dinov2/pytorch/small/1'), Path('/kaggle/input/dinov2/pytorch/small/1'), Path('/kaggle/input/dinov2-small/pytorch/small/1'))\n    for path in preferred:\n        if (path / 'config.json').is_file():\n            return path\n    for top in Path('/kaggle/input').glob('*'):\n        if not top.is_dir() or 'dino' not in top.name.lower():\n            continue\n        for config_path in top.glob('**/config.json'):\n            try:\n                cfg = json.loads(config_path.read_text())\n                if cfg.get('model_type') == 'dinov2' and int(cfg.get('hidden_size', -1)) == 384:\n                    return config_path.parent\n            except Exception:\n                continue\n    raise FileNotFoundError('offline DINOv2-small model is absent')\n\ndef _hyb_numeric(value, default=0.0):\n    try:\n        value = float(value)\n        return value if np.isfinite(value) else default\n    except Exception:\n        return default\n\ndef _hyb_sex_to_id(row):\n    value = str(row.get('PatientSex', '')).strip().lower()\n    if value.startswith('m'):\n        return 1\n    if value.startswith('f'):\n        return 2\n    return 0\n\ndef _hyb_choose_series(part, contrast, used_ids):\n    if len(part) == 0:\n        return None\n    fluid = part['Fluid_Sensitive'].fillna(0).astype(float)\n    fat = part['Fat_Suppression'].fillna(0).astype(float)\n    if contrast == 'Fluid':\n        score = 4.0 * fluid + 2.0 * fat\n    else:\n        score = 3.5 * (1.0 - fluid) + 1.5 * (1.0 - fat)\n    ordered = part.assign(_slot_score=score).sort_values('_slot_score', ascending=False)\n    for _, row in ordered.iterrows():\n        series_id = str(row['SeriesInstanceUID'])\n        if series_id not in used_ids:\n            return row\n    return ordered.iloc[0]\n\ndef _hyb_build_slots(series_df):\n    slots, study_meta = ({}, {})\n    for study_id, rows in series_df.groupby('StudyInstanceUID', sort=False):\n        study_id = str(study_id)\n        selected, meta = ({}, [])\n        plane_lower = rows['Anatomical_Plane'].astype(str).str.lower()\n        for plane in HYB_PLANES:\n            part = rows[plane_lower == plane.lower()]\n            count = len(part)\n            fluid_mean = part['Fluid_Sensitive'].fillna(0).astype(float).mean() if count else 0.0\n            fat_mean = part['Fat_Suppression'].fillna(0).astype(float).mean() if count else 0.0\n            meta.extend([np.log1p(count) / 3.0, fluid_mean, fat_mean])\n            used = set()\n            for contrast in HYB_CONTRASTS:\n                row = _hyb_choose_series(part, contrast, used)\n                if row is None:\n                    continue\n                sid = str(row['SeriesInstanceUID'])\n                used.add(sid)\n                selected[f'{plane}_{contrast}'] = {'series_id': sid, 'contrast': contrast, 'fluid': _hyb_numeric(row.get('Fluid_Sensitive', 0)), 'fat': _hyb_numeric(row.get('Fat_Suppression', 0))}\n        total = len(rows)\n        meta.extend([np.log1p(total) / 4.0, rows['Fluid_Sensitive'].fillna(0).astype(float).mean() if total else 0.0, rows['Fat_Suppression'].fillna(0).astype(float).mean() if total else 0.0, rows['SeriesInstanceUID'].nunique() / 12.0 if total else 0.0])\n        slots[study_id] = selected\n        study_meta[study_id] = np.asarray(meta, dtype=np.float32)\n    return (slots, study_meta)\n\ndef _hyb_locate_series_dir(study_uid, series_uid):\n    candidates = (ROOT / 'test_series' / str(study_uid) / str(series_uid), ROOT / 'test_series' / str(series_uid), ROOT / 'test' / str(study_uid) / str(series_uid), ROOT / 'test_images' / str(study_uid) / str(series_uid), ROOT / 'test_dicom' / str(study_uid) / str(series_uid), ROOT / 'test_dicoms' / str(study_uid) / str(series_uid), ROOT / 'images' / 'test' / str(study_uid) / str(series_uid))\n    for path in candidates:\n        if path.is_dir():\n            return path\n    raise FileNotFoundError(f'hybrid series missing: study={study_uid}, series={series_uid}')\n\ndef _hyb_read_header(path):\n    try:\n        return pydicom.dcmread(str(path), stop_before_pixels=True, force=True)\n    except Exception:\n        return None\n\ndef _hyb_header_position(ds):\n    if ds is None:\n        return None\n    try:\n        ipp = np.asarray([float(x) for x in ds.ImagePositionPatient], dtype=np.float64)\n        iop = np.asarray([float(x) for x in ds.ImageOrientationPatient], dtype=np.float64)\n        return float(np.dot(ipp, np.cross(iop[:3], iop[3:])))\n    except Exception:\n        pass\n    for name in ('SliceLocation', 'InstanceNumber'):\n        try:\n            return float(getattr(ds, name))\n        except Exception:\n            continue\n    return None\n\n@lru_cache(maxsize=8192)\ndef _hyb_ordered_files(folder_str):\n    files = sorted(Path(folder_str).glob('*.dcm'))\n    if not files:\n        files = sorted((path for path in Path(folder_str).iterdir() if path.is_file()))\n    keyed, ok = ([], 0)\n    for fallback, path in enumerate(files):\n        key = _hyb_header_position(_hyb_read_header(path))\n        if key is None:\n            key = fallback\n        else:\n            ok += 1\n        keyed.append((key, str(path)))\n    if ok >= max(3, len(files) // 3):\n        keyed.sort(key=lambda pair: pair[0])\n    return tuple((path for _, path in keyed))\n\ndef _hyb_spacing(ds):\n    spacing_x = spacing_y = thickness = 0.0\n    try:\n        ps = [float(x) for x in ds.PixelSpacing]\n        spacing_y, spacing_x = (ps[0], ps[1])\n    except Exception:\n        pass\n    for name in ('SliceThickness', 'SpacingBetweenSlices'):\n        try:\n            thickness = float(getattr(ds, name))\n            break\n        except Exception:\n            continue\n    return (spacing_x, spacing_y, thickness)\n\ndef _hyb_read_pixel(path):\n    ds = pydicom.dcmread(str(path), force=True)\n    array = ds.pixel_array.astype(np.float32)\n    array = array * _hyb_numeric(getattr(ds, 'RescaleSlope', 1.0), 1.0)\n    array += _hyb_numeric(getattr(ds, 'RescaleIntercept', 0.0), 0.0)\n    if str(getattr(ds, 'PhotometricInterpretation', '')).upper() == 'MONOCHROME1':\n        array = array.max() - array\n    return (array, ds)\n\ndef _hyb_robust_uint8(stack):\n    stack = np.asarray(stack, dtype=np.float32)\n    finite = stack[np.isfinite(stack)]\n    if finite.size == 0:\n        return np.zeros(stack.shape, dtype=np.uint8)\n    low, high = np.percentile(finite, [1.0, 99.4])\n    if high <= low:\n        low, high = (float(finite.min()), float(finite.max()))\n    if high <= low:\n        return np.zeros(stack.shape, dtype=np.uint8)\n    return (255.0 * np.clip((stack - low) / (high - low), 0.0, 1.0)).astype(np.uint8)\n\ndef _hyb_crop_foreground(image):\n    gray = image.max(axis=2)\n    mask = gray > max(8, np.percentile(gray, 55) * 0.18)\n    if mask.sum() < 64:\n        return image\n    ys, xs = np.where(mask)\n    y0, y1, x0, x1 = (int(ys.min()), int(ys.max()) + 1, int(xs.min()), int(xs.max()) + 1)\n    pad_y, pad_x = (int(0.08 * (y1 - y0 + 1)), int(0.08 * (x1 - x0 + 1)))\n    y0, y1 = (max(0, y0 - pad_y), min(image.shape[0], y1 + pad_y))\n    x0, x1 = (max(0, x0 - pad_x), min(image.shape[1], x1 + pad_x))\n    if y1 - y0 < 32 or x1 - x0 < 32:\n        return image\n    return image[y0:y1, x0:x1]\n\ndef _hyb_resize(image):\n    return cv2.resize(image, (HYB_IMG_SIZE, HYB_IMG_SIZE), interpolation=cv2.INTER_AREA)\n\ndef _hyb_view_radiomics(image):\n    gray = image.astype(np.float32).mean(axis=2) / 255.0\n    height, width = gray.shape\n    q = np.percentile(gray, [1, 5, 10, 25, 50, 75, 90, 95, 99])\n    center = gray[height // 4:3 * height // 4, width // 4:3 * width // 4]\n    gy, gx = np.gradient(gray)\n    grad = np.sqrt(gx * gx + gy * gy)\n    foreground = gray > 0.08\n    return np.asarray([gray.mean(), gray.std(), q[0], q[2], q[4], q[6], q[8], center.mean(), center.std(), grad.mean(), grad.std(), foreground.mean(), gray[foreground].mean() if foreground.any() else 0.0, gray[foreground].std() if foreground.any() else 0.0], dtype=np.float32)\n\ndef _hyb_make_view(paths):\n    arrays, first_ds = ([], None)\n    for path in paths:\n        try:\n            array, ds = _hyb_read_pixel(path)\n            first_ds = ds if first_ds is None else first_ds\n            arrays.append(array)\n        except Exception:\n            return (None, np.zeros(HYB_RAD_DIM, np.float32), (0.0, 0.0, 0.0))\n    image = np.transpose(_hyb_robust_uint8(np.stack(arrays)), (1, 2, 0))\n    image = _hyb_resize(_hyb_crop_foreground(image))\n    return (np.transpose(image, (2, 0, 1)), _hyb_view_radiomics(image), _hyb_spacing(first_ds) if first_ds is not None else (0.0, 0.0, 0.0))\n\ndef _hyb_sampled_triplets(study_uid, series_uid):\n    files = list(_hyb_ordered_files(str(_hyb_locate_series_dir(study_uid, series_uid))))\n    if not files:\n        return ([], 0)\n    centers = np.round(np.linspace(0.08 * (len(files) - 1), 0.92 * (len(files) - 1), HYB_N_POSITIONS)).astype(int)\n    centers = np.clip(centers, 0, len(files) - 1)\n    return ([[files[max(0, center - 1)], files[center], files[min(len(files) - 1, center + 1)]] for center in centers], len(files))\n\ndef _hyb_load_study(row, slots, study_meta):\n    study_uid = str(row['StudyInstanceUID'])\n    images = np.zeros((6, 5, 3, 224, 224), np.uint8)\n    view_mask = np.zeros((6, 5), bool)\n    slot_meta = np.zeros((6, 8), np.float32)\n    radiomics = np.zeros((6, 56), np.float32)\n    selected = slots.get(study_uid, {})\n    for slot_index, slot_name in enumerate(HYB_SLOT_NAMES):\n        info = selected.get(slot_name)\n        if info is None:\n            continue\n        triplets, n_files = _hyb_sampled_triplets(study_uid, info['series_id'])\n        rad_values, spacings = ([], [])\n        for pos_index, paths in enumerate(triplets[:5]):\n            image, rad, spacing = _hyb_make_view(paths)\n            if image is None:\n                continue\n            images[slot_index, pos_index] = image\n            view_mask[slot_index, pos_index] = True\n            rad_values.append(rad)\n            spacings.append(spacing)\n        if rad_values:\n            rad_array = np.stack(rad_values).astype(np.float32)\n            radiomics[slot_index] = np.concatenate([rad_array.mean(0), rad_array.std(0), rad_array.min(0), rad_array.max(0)])\n            spacing_mean = np.asarray(spacings, np.float32).mean(0)\n        else:\n            spacing_mean = np.zeros(3, np.float32)\n        slot_meta[slot_index] = np.asarray([info['fluid'], info['fat'], float(info['contrast'] == 'Structural'), np.log1p(n_files) / 6.0, float(view_mask[slot_index].mean()), spacing_mean[0] / 2.5 if spacing_mean[0] else 0.0, spacing_mean[1] / 2.5 if spacing_mean[1] else 0.0, spacing_mean[2] / 8.0 if spacing_mean[2] else 0.0], np.float32)\n    return (images, view_mask, slot_meta, radiomics, study_meta.get(study_uid, np.zeros(13, np.float32)), _hyb_sex_to_id(row), study_uid)\n\nclass _HybridDinoEncoder(nn.Module):\n\n    def __init__(self, backbone):\n        super().__init__()\n        self.backbone = backbone\n\n    def forward(self, pixel_values):\n        tokens = self.backbone(pixel_values=pixel_values).last_hidden_state\n        patches = tokens[:, 1:]\n        return torch.cat([F.normalize(tokens[:, 0], dim=1), F.normalize(patches.mean(dim=1), dim=1), F.normalize(patches.amax(dim=1), dim=1)], dim=1)\n\ndef _hyb_load_cached_split(cache_dir, split):\n    stem = f'{split}_{HYB_PREFIX}_'\n    return tuple((np.load(cache_dir / (stem + suffix), mmap_mode=None if suffix == 'ids.npy' else 'r', allow_pickle=suffix == 'ids.npy') for suffix in ('slot_features.npy', 'slot_mask.npy', 'slot_meta.npy', 'radiomics.npy', 'study_meta.npy', 'sex.npy', 'ids.npy')))\n\ndef _hyb_extract_test_bundle(test_df, series_df, cache_dir, dev):\n    expected_uids = test_df['StudyInstanceUID'].astype(str).to_numpy()\n    cached = _hyb_load_cached_split(cache_dir, 'test')\n    if np.array_equal(np.asarray(cached[-1]).astype(str), expected_uids):\n        log('hybrid: exact attached visible-test features reused')\n        return cached\n    if time.time() - T0 > HYB_START_CUTOFF_S:\n        raise TimeoutError('insufficient runtime reserve for hybrid feature extraction')\n    slots, study_meta = _hyb_build_slots(series_df)\n    backbone = AutoModel.from_pretrained(str(_hyb_find_dino_small()), local_files_only=True, trust_remote_code=False)\n    if int(backbone.config.hidden_size) != 384:\n        raise AssertionError('the hybrid specialist requires DINOv2-small hidden size 384')\n    model = _HybridDinoEncoder(backbone).eval().to(dev)\n    for parameter in model.parameters():\n        parameter.requires_grad_(False)\n    mean = torch.tensor([0.485, 0.456, 0.406], device=dev).view(1, 3, 1, 1)\n    std = torch.tensor([0.229, 0.224, 0.225], device=dev).view(1, 3, 1, 1)\n\n    @torch.inference_mode()\n    def encode(images):\n        outputs = []\n        for start in range(0, len(images), 64):\n            batch = images[start:start + 64].to(dev, non_blocking=True).float().div_(255.0)\n            batch = (batch - mean) / std\n            with torch.autocast('cuda', dtype=torch.float16, enabled=dev.type == 'cuda'):\n                outputs.append(model(batch).float().cpu())\n        return torch.cat(outputs, dim=0)\n    n = len(test_df)\n    features = np.zeros((n, 6, 3456), np.float16)\n    masks = np.zeros((n, 6), bool)\n    slot_meta_array = np.zeros((n, 6, 8), np.float16)\n    radiomics_array = np.zeros((n, 6, 56), np.float16)\n    study_meta_array = np.zeros((n, 13), np.float16)\n    sexes = np.zeros(n, np.int8)\n    ids = expected_uids.astype(object)\n\n    def safe_load(index):\n        try:\n            return _hyb_load_study(test_df.iloc[index], slots, study_meta)\n        except Exception as exc:\n            uid = str(test_df.iloc[index]['StudyInstanceUID'])\n            log(f'hybrid study decode failed safely: {uid}: {exc}')\n            return (np.zeros((6, 5, 3, 224, 224), np.uint8), np.zeros((6, 5), bool), np.zeros((6, 8), np.float32), np.zeros((6, 56), np.float32), np.zeros(13, np.float32), 0, uid)\n    workers = max(1, min(8, os.cpu_count() or 8))\n    with ThreadPoolExecutor(max_workers=workers) as executor:\n        for start in range(0, n, 6):\n            if time.time() - T0 > HYB_DEADLINE_S:\n                raise TimeoutError('hybrid deadline reached during feature extraction')\n            stop = min(start + 6, n)\n            items = list(executor.map(safe_load, range(start, stop)))\n            image_batch = torch.from_numpy(np.stack([item[0] for item in items]))\n            view_mask = torch.from_numpy(np.stack([item[1] for item in items]))\n            valid = view_mask.reshape(-1)\n            view_features = torch.zeros(len(items) * 30, 1152, dtype=torch.float32)\n            if valid.any():\n                flat_images = image_batch.reshape(-1, 3, 224, 224)\n                view_features[valid] = encode(flat_images[valid])\n            view_features = view_features.reshape(len(items), 6, 5, 1152)\n            aggregate = torch.zeros(len(items), 6, 3456, dtype=torch.float32)\n            slot_mask = view_mask.any(dim=2)\n            for batch_index in range(len(items)):\n                for slot_index in range(6):\n                    present = view_mask[batch_index, slot_index]\n                    if present.any():\n                        values = view_features[batch_index, slot_index, present]\n                        aggregate[batch_index, slot_index] = torch.cat([values.mean(0), values.amax(0), values.std(0, unbiased=False)])\n            features[start:stop] = aggregate.numpy().astype(np.float16)\n            masks[start:stop] = slot_mask.numpy()\n            slot_meta_array[start:stop] = np.stack([item[2] for item in items]).astype(np.float16)\n            radiomics_array[start:stop] = np.stack([item[3] for item in items]).astype(np.float16)\n            study_meta_array[start:stop] = np.stack([item[4] for item in items]).astype(np.float16)\n            sexes[start:stop] = np.asarray([item[5] for item in items], np.int8)\n            if start == 0 or stop % 100 == 0 or stop == n:\n                log(f'hybrid features {stop}/{n}')\n    del model, backbone\n    gc.collect()\n    if dev.type == 'cuda':\n        torch.cuda.empty_cache()\n    return (features, masks, slot_meta_array, radiomics_array, study_meta_array, sexes, ids)\n\ndef _hyb_align_bundle(bundle, expected_uids):\n    ids = np.asarray(bundle[-1]).astype(str)\n    expected_uids = np.asarray(expected_uids).astype(str)\n    if np.array_equal(ids, expected_uids):\n        return bundle\n    if len(set(ids)) != len(ids) or set(ids) != set(expected_uids):\n        raise AssertionError('hybrid cache StudyInstanceUID set mismatch')\n    positions = {uid: index for index, uid in enumerate(ids)}\n    order = np.asarray([positions[uid] for uid in expected_uids], dtype=int)\n    return tuple((np.asarray(array)[order] for array in bundle[:-1])) + (expected_uids,)\n\ndef _hyb_transform(bundle, pca):\n    features, masks, slot_meta, radiomics, study_meta, sex, _ = bundle\n    n = len(features)\n    result = np.zeros((n, 6, 192), np.float32)\n    for start in range(0, n, 96):\n        stop = min(start + 96, n)\n        block = np.asarray(features[start:stop], np.float32)\n        block_mask = np.asarray(masks[start:stop]).reshape(-1).astype(bool)\n        flat = block.reshape(-1, block.shape[-1])\n        transformed = np.zeros((len(flat), 192), np.float32)\n        if block_mask.any():\n            transformed[block_mask] = pca.transform(flat[block_mask]).astype(np.float32)\n        result[start:stop] = transformed.reshape(stop - start, 6, 192)\n    sex_onehot = np.eye(3, dtype=np.float32)[np.asarray(sex, dtype=int).clip(0, 2)]\n    matrix = np.concatenate([result.reshape(n, -1), np.asarray(masks, np.float32), np.asarray(slot_meta, np.float32).reshape(n, -1), np.asarray(radiomics, np.float32).reshape(n, -1), np.asarray(study_meta, np.float32), sex_onehot], axis=1)\n    matrix = np.nan_to_num(matrix, nan=0.0, posinf=0.0, neginf=0.0)\n    if matrix.shape != (n, 1558):\n        raise AssertionError(f'unexpected hybrid matrix shape {matrix.shape}')\n    return matrix.astype(np.float32)\n\ndef _hyb_rank(values, denominator_offset=0.0):\n    values = np.asarray(values, np.float64)\n    if len(values) <= 1 or np.ptp(values) < 1e-12:\n        return np.full(len(values), 0.5, np.float64)\n    return rankdata(values, method='average') / (len(values) + denominator_offset)\n\ndef _hyb_payload():\n    raw = zlib.decompress(base64.b64decode(HYB_TEACHER_PAYLOAD.encode('ascii')))\n    with np.load(io.BytesIO(raw), allow_pickle=False) as payload:\n        return {name: payload[name].astype(np.float32) for name in payload.files}\n\ndef _hyb_select_top(indices, scores, labels, class_value, cap=2200):\n    keep = indices[labels == class_value]\n    if len(keep) <= cap:\n        return keep\n    keep_scores = scores[labels == class_value]\n    return keep[np.argsort(-keep_scores)[:cap]]\n\ndef _hyb_training_arrays(pseudo_y, pseudo_conf, exact_mask, exact_y):\n    exact_idx = np.flatnonzero(exact_mask)\n    pseudo_pool = ~exact_mask\n    confidence = np.clip(pseudo_conf, 0.0, 1.0)\n    candidates = np.flatnonzero(pseudo_pool & (confidence >= 0.2))\n    candidate_labels = (pseudo_y[candidates] >= 0.5).astype(int)\n    pos = _hyb_select_top(candidates, confidence[candidates], candidate_labels, 1)\n    neg = _hyb_select_top(candidates, confidence[candidates], candidate_labels, 0)\n    pseudo_idx = np.concatenate([pos, neg]).astype(int)\n    pseudo_labels = (pseudo_y[pseudo_idx] >= 0.5).astype(int)\n    pseudo_weight = 0.18 + 1.15 * np.power(np.clip(confidence[pseudo_idx], 0, 1), 1.4)\n    fit_idx = np.concatenate([exact_idx, pseudo_idx]).astype(int)\n    fit_y = np.concatenate([exact_y[exact_idx].astype(int), pseudo_labels]).astype(int)\n    fit_weight = np.concatenate([np.full(len(exact_idx), 7.0, np.float32), pseudo_weight.astype(np.float32)])\n    return (fit_idx, fit_y, fit_weight, exact_idx, exact_y[exact_idx].astype(int))\n\ndef _hyb_fit_lr(x_fit, y_fit, x_test, weights, cs, seed):\n    predictions = []\n    for index, c_value in enumerate(cs):\n        model = LogisticRegression(C=c_value, solver='liblinear', class_weight='balanced', max_iter=3000, random_state=seed + 31 * index)\n        model.fit(x_fit, y_fit, sample_weight=weights)\n        predictions.append(model.predict_proba(x_test)[:, 1])\n    return np.mean(predictions, axis=0).astype(np.float32)\n\ndef _hyb_fit_family(family, x_train, fit_idx, fit_y, fit_weight, x_test, seed):\n    if time.time() - T0 > HYB_DEADLINE_S:\n        raise TimeoutError('hybrid deadline reached during model fitting')\n    if family == 'lr':\n        return _hyb_fit_lr(x_train[fit_idx], fit_y, x_test, fit_weight, HYB_LR_CS, seed)\n    if family == 'exact_lr':\n        return _hyb_fit_lr(x_train[fit_idx], fit_y, x_test, fit_weight, HYB_EXACT_LR_CS, seed)\n    if family == 'et':\n        model = ExtraTreesClassifier(n_estimators=420, max_features='sqrt', min_samples_leaf=4, min_samples_split=8, bootstrap=False, class_weight='balanced', random_state=seed, n_jobs=-1)\n    elif family == 'hgb':\n        model = HistGradientBoostingClassifier(learning_rate=0.035, max_iter=180, max_leaf_nodes=15, min_samples_leaf=18, l2_regularization=0.25, early_stopping=True, validation_fraction=0.15, random_state=seed)\n    else:\n        raise ValueError(f'unknown hybrid family {family}')\n    model.fit(x_train[fit_idx], fit_y, sample_weight=fit_weight)\n    return model.predict_proba(x_test)[:, 1].astype(np.float32)\n\ndef _hyb_teacher_arm(x_train, x_test, pseudo_y, pseudo_conf, exact_mask, exact_y, exact_lr):\n    fit_idx, fit_y, fit_weight, _, _ = _hyb_training_arrays(pseudo_y, pseudo_conf, exact_mask, exact_y)\n    predictions = {}\n    for family in ('lr', 'et', 'hgb'):\n        seed_predictions = []\n        for seed in HYB_SEEDS:\n            model_seed = seed + 101 * TARGETS.index(HYB_TARGET) + len(family)\n            seed_predictions.append(_hyb_fit_family(family, x_train, fit_idx, fit_y, fit_weight, x_test, model_seed))\n        predictions[family] = np.mean(np.stack(seed_predictions), axis=0)\n    predictions['exact_lr'] = exact_lr\n    weighted_rank = np.zeros(len(x_test), np.float64)\n    weighted_prob = np.zeros(len(x_test), np.float64)\n    for weight, family in zip(HYB_FAMILY_WEIGHTS, HYB_FAMILIES):\n        pred = np.clip(predictions[family], 1e-05, 1.0 - 1e-05)\n        weighted_rank += weight * _hyb_rank(pred, denominator_offset=1.0)\n        weighted_prob += weight * pred\n    return 0.9 * weighted_rank + 0.1 * weighted_prob\n\ndef run_hybrid_lm_and_lateral_oa_specialists():\n    if time.time() - T0 > HYB_START_CUTOFF_S:\n        log('hybrid skipped: insufficient runtime reserve')\n        return False\n    cache_dir = _hyb_find_cache_dir()\n    primary_path = Path('submission.csv')\n    primary = pd.read_csv(primary_path, dtype={'StudyInstanceUID': str})\n    train_df = pd.read_csv(ROOT / 'train.csv', dtype={'StudyInstanceUID': str})\n    test_df = pd.read_csv(ROOT / 'test.csv', dtype={'StudyInstanceUID': str})\n    series_df = pd.read_csv(ROOT / 'test_series.csv', dtype={'StudyInstanceUID': str, 'SeriesInstanceUID': str})\n    if primary.columns.tolist() != ['StudyInstanceUID'] + TARGETS:\n        raise AssertionError('primary submission schema mismatch')\n    train_uids = train_df['StudyInstanceUID'].astype(str).to_numpy()\n    uid_hash = hashlib.sha256('\\n'.join(train_uids).encode()).hexdigest()\n    if uid_hash != HYB_EXPECTED_TRAIN_ID_SHA256:\n        raise AssertionError('competition train StudyInstanceUID order drifted')\n    test_uids = test_df['StudyInstanceUID'].astype(str).to_numpy()\n    dev = DEVS[0]\n    train_bundle = _hyb_align_bundle(_hyb_load_cached_split(cache_dir, 'train'), train_uids)\n    test_bundle = _hyb_align_bundle(_hyb_extract_test_bundle(test_df, series_df, cache_dir, dev), test_uids)\n    pca = joblib.load(cache_dir / f'{HYB_PREFIX}_ipca_192.joblib')\n    x_train_raw = _hyb_transform(train_bundle, pca)\n    x_test_raw = _hyb_transform(test_bundle, pca)\n    joint = np.concatenate([x_train_raw, x_test_raw], axis=0)\n    mean = joint.mean(axis=0, dtype=np.float64).astype(np.float32)\n    scale = joint.std(axis=0, dtype=np.float64).astype(np.float32)\n    scale[scale < 1e-05] = 1.0\n    x_train = ((x_train_raw - mean) / scale).astype(np.float32)\n    x_test = ((x_test_raw - mean) / scale).astype(np.float32)\n    payload = _hyb_payload()\n    if any((len(payload[name]) != len(train_df) for name in payload)):\n        raise AssertionError('embedded hybrid teacher length mismatch')\n\n    def fit_target(target_name, pseudo_y, pseudo_conf, seeds):\n        exact_mask = train_df[target_name].notna().to_numpy()\n        exact_y = np.nan_to_num(train_df[target_name].to_numpy(np.float32), nan=0.0)\n        exact_idx = np.flatnonzero(exact_mask)\n        exact_labels = exact_y[exact_idx].astype(int)\n        exact_predictions = []\n        for seed in seeds:\n            model_seed = seed + 101 * TARGETS.index(target_name) + len('exact_lr')\n            exact_predictions.append(_hyb_fit_family('exact_lr', x_train, exact_idx, exact_labels, np.full(len(exact_idx), 3.0, np.float32), x_test, model_seed))\n        exact_lr = np.mean(np.stack(exact_predictions), axis=0)\n        fit_idx, fit_y, fit_weight, _, _ = _hyb_training_arrays(pseudo_y, pseudo_conf, exact_mask, exact_y)\n        predictions = {}\n        for family in ('lr', 'et', 'hgb'):\n            seed_predictions = []\n            for seed in seeds:\n                model_seed = seed + 101 * TARGETS.index(target_name) + len(family)\n                seed_predictions.append(_hyb_fit_family(family, x_train, fit_idx, fit_y, fit_weight, x_test, model_seed))\n            predictions[family] = np.mean(np.stack(seed_predictions), axis=0)\n        predictions['exact_lr'] = exact_lr\n        weighted_rank = np.zeros(len(x_test), np.float64)\n        weighted_prob = np.zeros(len(x_test), np.float64)\n        family_weights = HYB_LM_FAMILY_WEIGHTS if target_name == HYB_TARGET else HYB_OA_FAMILY_WEIGHTS\n        for weight, family in zip(family_weights, HYB_FAMILIES):\n            pred = np.clip(predictions[family], 1e-05, 1.0 - 1e-05)\n            weighted_rank += weight * _hyb_rank(pred, denominator_offset=1.0)\n            weighted_prob += weight * pred\n        return 0.9 * weighted_rank + 0.1 * weighted_prob\n    lm_consensus = fit_target(HYB_TARGET, payload['lm_consensus_y'], payload['lm_consensus_conf'], HYB_SEEDS)\n    lm_pilkwang = fit_target(HYB_TARGET, payload['lm_pilkwang_y'], payload['lm_pilkwang_conf'], HYB_SEEDS)\n    lm_teacher_rank = 1.0 * _hyb_rank(lm_consensus) + 0.0 * _hyb_rank(lm_pilkwang)\n    oa_pilkwang = fit_target(HYB_OA_TARGET, payload['oa_pilkwang_y'], payload['oa_pilkwang_conf'], HYB_OA_SEEDS)\n    oa_teacher_rank = _hyb_rank(oa_pilkwang)\n    train_mean = x_train_raw.mean(axis=0, dtype=np.float64).astype(np.float32)\n    train_scale = x_train_raw.std(axis=0, dtype=np.float64).astype(np.float32)\n    train_scale[train_scale < 1e-05] = 1.0\n    x_train_pca = ((x_train_raw - train_mean) / train_scale).astype(np.float32)\n    x_test_pca = ((x_test_raw - train_mean) / train_scale).astype(np.float32)\n    pca_specs = {HYB_TARGET: (128, 0.1), 'Synovitis': (32, 1.0)}\n    pca_predictions = {target: [] for target in pca_specs}\n    for pca_seed in (20260809, 20260819, 20260829, 20260839):\n        decomposition = PCA(n_components=128, whiten=True, svd_solver='randomized', n_oversamples=20, random_state=pca_seed)\n        train_embedding = decomposition.fit_transform(x_train_pca)\n        test_embedding = decomposition.transform(x_test_pca)\n        for target_name, (dimensions, c_value) in pca_specs.items():\n            exact_mask = train_df[target_name].notna().to_numpy()\n            exact_labels = train_df.loc[exact_mask, target_name].to_numpy(int)\n            if int(exact_mask.sum()) != 58 or set(np.unique(exact_labels)) != {0, 1}:\n                raise AssertionError(f'unexpected exact-label support for {target_name}')\n            model = make_pipeline(StandardScaler(), LogisticRegression(C=c_value, solver='liblinear', class_weight='balanced', max_iter=5000, random_state=pca_seed))\n            model.fit(train_embedding[exact_mask, :dimensions], exact_labels)\n            pca_predictions[target_name].append(model.predict_proba(test_embedding[:, :dimensions])[:, 1])\n    pca_predictions = {target: np.mean(np.stack(predictions), axis=0) for target, predictions in pca_predictions.items()}\n    result = primary.copy()\n    primary_uids = result['StudyInstanceUID'].astype(str)\n    if set(primary_uids) != set(test_uids):\n        raise AssertionError('hybrid and primary StudyInstanceUID sets differ')\n    lm_by_uid = pd.Series(lm_teacher_rank, index=test_uids).reindex(primary_uids.values)\n    oa_by_uid = pd.Series(oa_teacher_rank, index=test_uids).reindex(primary_uids.values)\n    lm_pca_by_uid = pd.Series(_hyb_rank(pca_predictions[HYB_TARGET]), index=test_uids).reindex(primary_uids.values)\n    syn_pca_by_uid = pd.Series(_hyb_rank(pca_predictions['Synovitis']), index=test_uids).reindex(primary_uids.values)\n    lm_base_rank = result[HYB_TARGET].rank(pct=True).to_numpy(np.float64)\n    oa_base_rank = result[HYB_OA_TARGET].rank(pct=True).to_numpy(np.float64)\n    syn_base_rank = result['Synovitis'].rank(pct=True).to_numpy(np.float64)\n    result[HYB_TARGET] = 0.125 * lm_base_rank + 0.5 * lm_by_uid.to_numpy(np.float64) + 0.375 * lm_pca_by_uid.to_numpy(np.float64)\n    result[HYB_OA_TARGET] = 0.125 * oa_base_rank + 0.875 * oa_by_uid.to_numpy(np.float64)\n    result['Synovitis'] = 0.75 * syn_base_rank + 0.25 * syn_pca_by_uid.to_numpy(np.float64)\n    changed = {HYB_TARGET, HYB_OA_TARGET, 'Synovitis'}\n    untouched = [target for target in TARGETS if target not in changed]\n    if not result[untouched].equals(primary[untouched]):\n        raise AssertionError('hybrid changed an unevaluated target')\n    if result.shape != primary.shape or not np.isfinite(result[TARGETS].to_numpy()).all():\n        raise AssertionError('invalid hybrid blend')\n    temp_path = Path('submission_v34_hybrid.tmp.csv')\n    result.to_csv(temp_path, index=False)\n    reread = pd.read_csv(temp_path)\n    if reread.shape != primary.shape or not np.isfinite(reread[TARGETS].to_numpy()).all():\n        raise AssertionError('serialized hybrid blend is invalid')\n    temp_path.replace(primary_path)\n    log('hybrid complete: LM 0.125 base / 0.500 consensus / 0.375 PCA; Lateral OA 0.125 base / 0.875 Pilkwang; Synovitis 0.75 base / 0.25 PCA; nine other targets preserved')\n    return True"},{"cell_type":"code","execution_count":null,"id":"6d89808e","metadata":{},"outputs":[],"source":"def write_submission(pred, studies, test_df, path):\n    sub = pd.DataFrame(pd.DataFrame(pred).rank(pct=True).values, columns=TARGETS)\n    sub.insert(0, 'StudyInstanceUID', studies)\n    sub = test_df[['StudyInstanceUID']].merge(sub, on='StudyInstanceUID', how='left')\n    sub[TARGETS] = sub[TARGETS].fillna(0.5)\n    sub.to_csv(path, index=False)\n    return sub\n\ndef write_benchmark_submission():\n    t = pd.read_csv(ROOT / 'test.csv')\n    for c in TARGETS:\n        t[c] = 0.5\n    t.to_csv('submission.csv', index=False)\n\ndef _v37_validate_submission(path, test_df, tag):\n    path = Path(path)\n    frame = pd.read_csv(path)\n    expected = ['StudyInstanceUID'] + TARGETS\n    if list(frame.columns) != expected:\n        raise ValueError(f'{tag}: columns differ from the competition contract')\n    if len(frame) != len(test_df) or not frame['StudyInstanceUID'].is_unique:\n        raise ValueError(f'{tag}: row count or StudyInstanceUID uniqueness failed')\n    if set(frame['StudyInstanceUID'].astype(str)) != set(test_df['StudyInstanceUID'].astype(str)):\n        raise ValueError(f'{tag}: StudyInstanceUID set differs from test.csv')\n    values = frame[TARGETS].to_numpy(np.float64)\n    if not np.isfinite(values).all():\n        raise ValueError(f'{tag}: non-finite prediction')\n    return test_df[['StudyInstanceUID']].merge(frame, on='StudyInstanceUID', how='left')\n\ndef _v37_find_yash_submission():\n    candidates = []\n    local = globals().get('YASH_LOCAL_SOURCE_DIR')\n    if local:\n        candidates.append(Path(local) / 'submission.csv')\n    root = Path('/kaggle/input')\n    candidates.append(root / 'rsna-knee-infer-v1' / 'submission.csv')\n    if root.is_dir():\n        candidates.extend((meta.parent / 'submission.csv' for meta in root.glob('**/infer_meta.json')))\n    seen = set()\n    for path in candidates:\n        key = str(path)\n        if key in seen or not path.is_file():\n            continue\n        seen.add(key)\n        meta_path = path.with_name('infer_meta.json')\n        if meta_path.is_file():\n            meta = json.loads(meta_path.read_text())\n            if int(meta.get('errors', -1)) != 0:\n                raise ValueError(f\"Yash source reports {meta.get('errors')} inference errors\")\n        return path\n    raise FileNotFoundError('the attached yashbishnoi98/rsna-knee-infer-v1 output is absent')\n\ndef run_yash_public_ensemble():\n    import shutil\n    test_df = pd.read_csv(ROOT / 'test.csv')\n    native_path = Path('submission.csv')\n    public_path = Path('submission_public_0899.csv')\n    native = _v37_validate_submission(native_path, test_df, 'native V36')\n    public = _v37_validate_submission(public_path, test_df, 'public DINO family')\n    yash_path = _v37_find_yash_submission()\n    yash = _v37_validate_submission(yash_path, test_df, 'Yash public image family')\n    meta_path = yash_path.with_name('infer_meta.json')\n    if meta_path.is_file():\n        meta = json.loads(meta_path.read_text())\n        if int(meta.get('studies', -1)) != len(test_df):\n            raise ValueError('Yash source study count differs from test.csv')\n    shutil.copyfile(native_path, 'submission_native_v36.csv')\n    shutil.copyfile(yash_path, 'submission_yash_reference.csv')\n    yr = yash[TARGETS].rank(pct=True).to_numpy(np.float64)\n    dr = public[TARGETS].rank(pct=True).to_numpy(np.float64)\n    blend = 0.55 * yr + 0.45 * dr\n    result = test_df[['StudyInstanceUID']].copy()\n    result[TARGETS] = blend\n    if result.shape != yash.shape or not np.isfinite(result[TARGETS].to_numpy()).all():\n        raise AssertionError('invalid Yash/DINO rank blend')\n    candidate_path = Path('submission_yash_dino_rankblend.csv')\n    result.to_csv(candidate_path, index=False)\n    reread = _v37_validate_submission(candidate_path, test_df, 'Yash/DINO rank blend')\n    changed = sum((tuple(reread[target].rank(method='first')) != tuple(yash[target].rank(method='first')) for target in TARGETS))\n    if changed == 0:\n        raise AssertionError('Yash/DINO blend is rank-identical to its Yash parent')\n    temp_path = Path('submission_v37_yash_dino.tmp.csv')\n    reread.to_csv(temp_path, index=False)\n    temp_path.replace(native_path)\n    log(f'Yash public family banked; V37 primary = 0.55 Yash / 0.45 public DINO rank blend ({changed} target orderings differ from Yash); exact Yash and native V36 outputs retained')\n    return True\n\ndef main():\n    write_benchmark_submission()\n    pkg = find_weights()\n    if pkg is not None:\n        dev = DEVS[0]\n        infer_from_package(pkg, dev)\n        try:\n            test_df = pd.read_csv(ROOT / 'test.csv')\n            native_path = Path('submission.csv')\n            public_path = Path('submission_public_0899.csv')\n            native = _v37_validate_submission(native_path, test_df, 'native 24-member')\n            public = _v37_validate_submission(public_path, test_df, 'public DINO frontier')\n            native.to_csv('submission_native_v38.csv', index=False)\n            public.to_csv(native_path, index=False)\n            promoted = _v37_validate_submission(native_path, test_df, 'V40 primary')\n            if not promoted.equals(public):\n                raise AssertionError('V40 serialization differs from validated public frontier')\n            log('V40 primary = exact no-jitter public-frontier target pooling; native 24-member output retained')\n        except Exception as public_frontier_error:\n            log(f'public-frontier promotion skipped safely: {public_frontier_error}')\n            traceback.print_exc()\n        log('done')\n        return\n    read_labels(pd.read_csv(ROOT / 'train.csv', usecols=['StudyInstanceUID', 'Report']))\n    test_df = pd.read_csv(ROOT / 'test.csv')\n    test_series = pd.read_csv(ROOT / 'test_series.csv')\n    train_df = pd.read_csv(ROOT / 'train.csv')\n    train_series = pd.read_csv(ROOT / 'train_series.csv')\n    log(f'train {train_df.shape} test {test_df.shape}')\n    both = pd.concat([train_series, test_series])\n    plane_map = dict(zip(both['SeriesInstanceUID'], both['Anatomical_Plane']))\n    log('header pass: test')\n    hte = annotate(walk('test_series'))\n    log(f'  {len(hte)} test series')\n    log('header pass: train')\n    htr = annotate(walk('train_series'))\n    log(f'  {len(htr)} train series')\n    slots_te, slots_tr = (pick_slots(hte, plane_map), pick_slots(htr, plane_map))\n    cov = pd.Series([len(v) for v in slots_tr.values()]).describe()\n    log(f\"train slots per study: mean {cov['mean']:.2f} min {cov['min']:.0f} max {cov['max']:.0f}\")\n    st_tr, Ctr, Mtr = build_cache(slots_tr, plane_map, lat_of(htr, 'train '), 'train')\n    st_te, Cte, Mte = build_cache(slots_te, plane_map, lat_of(hte, 'test '), 'test')\n    t_lab = time.time()\n    lab = read_labels(train_df)\n    log(f'derived labels for {len(lab)} studies in {time.time() - t_lab:.1f}s')\n    gold = train_df.set_index('StudyInstanceUID')[TARGETS]\n    gold = gold[gold.notna().all(axis=1)]\n    Y = np.zeros((len(st_tr), len(TARGETS)), np.float32)\n    W = np.zeros_like(Y)\n    for i, st in enumerate(st_tr):\n        if st in gold.index:\n            Y[i], W[i] = (gold.loc[st].values, 3.0)\n        elif st in lab.index:\n            r = lab.loc[st]\n            Y[i] = r[TARGETS].values\n            W[i] = 0.25 + 0.75 * r[[t + '__conf' for t in TARGETS]].values\n    keep = np.where(W.sum(1) > 0)[0]\n    log(f'supervised {len(keep)} of {len(st_tr)} studies (annotated {len(gold)})')\n    import hashlib\n    rep = train_df.set_index('StudyInstanceUID')['Report'].fillna('')\n    grp = np.array([int(hashlib.md5(rep.get(s, s).encode()).hexdigest()[:8], 16) % 5 for s in st_tr])\n    va = np.array([i for i in keep if grp[i] == 0])\n    tr = np.array([i for i in keep if grp[i] != 0])\n    if len(va) == 0 or len(tr) < BATCH_STUDIES:\n        cut = max(1, len(keep) // 5)\n        va, tr = (keep[:cut], keep[cut:])\n    log(f'train {len(tr)} / holdout {len(va)} studies')\n    gpos = {s: i for i, s in enumerate(st_tr)}\n    va_set = set(va.tolist())\n    gi = np.array([gpos[s] for s in gold.index if s in gpos and gpos[s] in va_set])\n    gold_y = gold.loc[[st_tr[i] for i in gi]].values.astype(int) if len(gi) else None\n    yv = (Y[va] > 0.5).astype(int)\n    log(f'annotation check: {len(gi)} of {len(gold)} annotated studies are in the holdout')\n    dev = DEVS[0]\n    results, test_preds = ({}, {})\n    for cfg in RUNS:\n        pitch = CROP_MM / cfg['img']\n        log(f\"=== {cfg['name']}: {cfg['img']} px, {pitch:.3f} mm/pixel, {pitch * 14:.2f} mm per patch token ===\")\n        torch.manual_seed(SEED)\n        model = build_model(UNFREEZE_LAST).to(dev)\n        opt = torch.optim.AdamW([{'params': [p for p in model.backbone.parameters() if p.requires_grad], 'lr': LR_BACKBONE}, {'params': model.head.parameters(), 'lr': LR_HEAD}], weight_decay=WEIGHT_DECAY)\n        steps = max(EPOCHS * (len(tr) // BATCH_STUDIES), 1)\n        sched = torch.optim.lr_scheduler.OneCycleLR(opt, max_lr=[LR_BACKBONE, LR_HEAD], total_steps=steps, pct_start=0.15)\n        scaler = torch.amp.GradScaler('cuda', enabled=dev.type == 'cuda')\n        best, best_state, best_annot = (-1.0, None, float('nan'))\n        for ep in range(EPOCHS):\n            model.train()\n            perm = np.random.permutation(tr)\n            tot, nstep = (0.0, 0)\n            for b in range(0, len(perm) - BATCH_STUDIES + 1, BATCH_STUDIES):\n                sel = perm[b:b + BATCH_STUDIES]\n                rows = torch.from_numpy(Ctr[sel]).to(dev)\n                g = int(torch.randint(N_GROUP, (1,)).item())\n                imgs = augment(take_group(rows, g))\n                m = torch.from_numpy(Mtr[sel]).to(dev)\n                y = torch.from_numpy(Y[sel]).to(dev)\n                w = torch.from_numpy(W[sel]).to(dev)\n                with torch.autocast('cuda', enabled=dev.type == 'cuda'):\n                    loss = (F.binary_cross_entropy_with_logits(model(imgs, m, cfg['img']), y, reduction='none') * w).mean()\n                opt.zero_grad(set_to_none=True)\n                scaler.scale(loss).backward()\n                scaler.step(opt)\n                scaler.update()\n                sched.step()\n                tot += loss.item()\n                nstep += 1\n            pv = predict(model, Ctr, Mtr, va, dev, cfg['img'])\n            d = macro_auc(yv, pv)\n            g_auc = float('nan')\n            if gold_y is not None and len(gi):\n                g_auc = macro_auc(gold_y, predict(model, Ctr, Mtr, gi, dev, cfg['img']))\n            log(f'  epoch {ep + 1}/{EPOCHS}  loss {tot / max(nstep, 1):.4f}  holdout {d:.4f}  annot(n={len(gi)}) {g_auc:.4f}')\n            if d > best:\n                best, best_annot = (d, g_auc)\n                best_state = {k: v.detach().cpu().clone() for k, v in model.state_dict().items()}\n            if time.time() - T0 > TIME_BUDGET:\n                log('  time budget reached')\n                break\n        if best_state is not None:\n            model.load_state_dict(best_state)\n        results[cfg['name']] = (best, best_annot)\n        test_preds[cfg['name']] = predict(model, Cte, Mte, np.arange(len(st_te)), dev, cfg['img'])\n        log(f\"  {cfg['name']}: best holdout {best:.4f} (annot {best_annot:.4f})\")\n        del model, opt, sched, scaler, best_state\n        gc.collect()\n        if dev.type == 'cuda':\n            torch.cuda.empty_cache()\n    log('---- summary ----')\n    for n, (d, g_auc) in results.items():\n        log(f'  {n:12s} holdout {d:.4f}   annot {g_auc:.4f}')\n    pick = max(results, key=lambda k: results[k][0])\n    log(f'best on the holdout: {pick} ({results[pick][0]:.4f})')\n    for name, pred in test_preds.items():\n        sub = write_submission(pred, st_te, test_df, f'submission_{name}.csv')\n        log(f'  submission_{name}.csv {sub.shape}; nulls {int(sub[TARGETS].isna().sum().sum())}')\n    ens = np.mean([pd.DataFrame(p).rank(pct=True).values for p in test_preds.values()], axis=0)\n    write_submission(ens, st_te, test_df, 'submission_rankmean.csv')\n    log(f'  submission_rankmean.csv (rank mean of {len(test_preds)})')\n    sub = write_submission(test_preds[pick], st_te, test_df, 'submission.csv')\n    log(f'submission.csv = {pick}; {sub.shape}; nulls {int(sub[TARGETS].isna().sum().sum())}')\n    print(sub.head().to_string())"},{"cell_type":"code","execution_count":null,"id":"38d9a19b","metadata":{},"outputs":[],"source":"try:\n    main()\nexcept LabelSourceError:\n    traceback.print_exc()\n    raise\nexcept Exception:\n    traceback.print_exc()\n    t = pd.read_csv(find_root() / 'test.csv')\n    for c in TARGETS:\n        t[c] = 0.5\n    t.to_csv('submission.csv', index=False)\n    print('wrote fallback submission.csv')\nlog('done')"},{"cell_type":"code","execution_count":null,"id":"2657ab22","metadata":{},"outputs":[],"source":"_A5_SAVED = dict(globals())\nimport gc, os, time, warnings\nfrom concurrent.futures import ProcessPoolExecutor, as_completed\nfrom pathlib import Path\nimport cv2\nimport numpy as np\nimport pandas as pd\nimport pydicom\nimport timm\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nwarnings.filterwarnings('ignore')\ncv2.setNumThreads(1)\nCROP_MM = 130.0\nSIZE = 336\nSLICE_BAND = (0.12, 0.88)\nN_SLICE = 16\nINTENSITY = 'slice'\nSLOTS = [('Sagittal', 1), ('Sagittal', 0), ('Coronal', 1), ('Coronal', 0), ('Axial', 1), ('Axial', 0)]\nN_SLOT = len(SLOTS)\nLABELS = ['ACL', 'MCL', 'Medial Meniscus', 'Lateral Meniscus', 'Medial OA', 'Lateral OA', 'PF OA', 'Effusion', 'Synovitis', \"Baker's\", 'Contusion', 'Fracture']\n\ndef _find_dir(*names):\n    root = Path('/kaggle/input')\n    cand = []\n    for n in names:\n        cand += [root / n, root / 'competitions' / n, root / 'datasets' / n]\n        for parent in (root / 'datasets', root / 'competitions', root):\n            if parent.is_dir():\n                try:\n                    cand += [d / n for d in parent.iterdir() if d.is_dir()]\n                except OSError:\n                    pass\n    for p in cand:\n        if p.is_dir():\n            return p\n    return None\nCOMP = _find_dir('rsna-knee-abnormality-detection')\nCKPT = _find_dir('knee-mri-fold-weights')\nassert COMP is not None, 'competition data not attached'\nassert CKPT is not None, 'fold weights not attached'\nassert (COMP / 'sample_submission.csv').exists(), f'no competition data at {COMP}'\nassert list(CKPT.glob('*_f*.pt')), f'no checkpoints at {CKPT}'\nDEV = 'cuda' if torch.cuda.is_available() else 'cpu'\nprint(f'competition : {COMP}')\nprint(f'checkpoints : {CKPT}')\nprint(f'device      : {DEV}')\nfor i in range(torch.cuda.device_count() if DEV == 'cuda' else 0):\n    cc = torch.cuda.get_device_capability(i)\n    print(f'  gpu{i}       : {torch.cuda.get_device_name(i)} sm_{cc[0]}{cc[1]}, {torch.cuda.get_device_properties(i).total_memory / 2 ** 30:.0f} GiB, native bf16={cc >= (8, 0)}')"},{"cell_type":"code","execution_count":null,"id":"c83321bb","metadata":{},"outputs":[],"source":"SERIES_ROOT = COMP / 'test_series'\nif not SERIES_ROOT.exists():\n    SERIES_ROOT = COMP / 'train_series'\nprint('series root:', SERIES_ROOT)\n\ndef ordered_files(sdir, cap=64):\n    keyed = []\n    for f in sdir.glob('*.dcm'):\n        try:\n            ds = pydicom.dcmread(str(f), stop_before_pixels=True)\n            keyed.append((int(ds.InstanceNumber), str(f)))\n        except Exception:\n            continue\n        if len(keyed) >= cap * 4:\n            break\n    return [f for _, f in sorted(keyed)]\n\ndef series_side(path):\n    try:\n        return float(pydicom.dcmread(path, stop_before_pixels=True).ImagePositionPatient[0])\n    except Exception:\n        return 0.0\n\ndef read_crop(path):\n    try:\n        ds = pydicom.dcmread(path)\n        arr = ds.pixel_array.astype(np.float32)\n    except Exception:\n        return None\n    try:\n        ps = float(ds.PixelSpacing[0])\n    except Exception:\n        ps = CROP_MM / max(arr.shape)\n    half = int(round(CROP_MM / ps / 2))\n    cy, cx = (arr.shape[0] // 2, arr.shape[1] // 2)\n    y0, y1 = (max(0, cy - half), min(arr.shape[0], cy + half))\n    x0, x1 = (max(0, cx - half), min(arr.shape[1], cx + half))\n    crop = arr[y0:y1, x0:x1]\n    return None if crop.size == 0 else crop\n\ndef window(crop, lo, hi, flip):\n    c = np.clip((crop - lo) / max(hi - lo, 1e-06), 0, 1)\n    img = cv2.resize(c, (SIZE, SIZE), interpolation=cv2.INTER_AREA)\n    return img[:, ::-1].copy() if flip else img\n\ndef render(path, flip):\n    crop = read_crop(path)\n    if crop is None:\n        return None\n    lo, hi = np.percentile(crop[::4, ::4], [1, 99])\n    return window(crop, lo, hi, flip)\n\ndef build_study(args):\n    idx, study, recs = args\n    out = np.zeros((N_SLOT, N_SLICE, SIZE, SIZE), np.uint8)\n    mask = np.zeros(N_SLOT, np.uint8)\n    rows = pd.DataFrame(recs)\n    if len(rows):\n        for s_i, (plane, fs) in enumerate(SLOTS):\n            sub = rows[(rows.Anatomical_Plane == plane) & (rows.Fat_Suppression == fs)]\n            if sub.empty:\n                continue\n            files = ordered_files(SERIES_ROOT / study / sub.iloc[0].SeriesInstanceUID)\n            if not files:\n                continue\n            flip = plane != 'Sagittal' and series_side(files[0]) < 0\n            lo, hi = SLICE_BAND\n            i0 = int(round(lo * (len(files) - 1)))\n            i1 = int(round(hi * (len(files) - 1)))\n            avail = list(range(i0, i1 + 1))\n            if len(avail) >= N_SLICE:\n                picks = [avail[int(round(t))] for t in np.linspace(0, len(avail) - 1, N_SLICE)]\n                off = 0\n            else:\n                picks, off = (avail, (N_SLICE - len(avail)) // 2)\n            if INTENSITY == 'series':\n                crops = [read_crop(files[p]) for p in picks]\n                got = [x for x in crops if x is not None]\n                if got:\n                    samp = np.concatenate([x[::4, ::4].ravel() for x in got])\n                    lo_, hi_ = np.percentile(samp, [1, 99])\n                    for c, x in enumerate(crops):\n                        if x is None:\n                            x = read_crop(files[min(len(files) - 1, picks[c] + 1)])\n                        if x is not None:\n                            out[s_i, off + c] = (window(x, lo_, hi_, flip) * 255).astype(np.uint8)\n            else:\n                for c, p in enumerate(picks):\n                    img = render(files[p], flip)\n                    if img is None:\n                        img = render(files[min(len(files) - 1, p + 1)], flip)\n                    if img is not None:\n                        out[s_i, off + c] = (img * 255).astype(np.uint8)\n            mask[s_i] = len(picks)\n    return (idx, out, mask)\nsub_df = pd.read_csv(COMP / 'sample_submission.csv')\nser_csv = pd.read_csv(COMP / 'test_series.csv')\nif not (COMP / 'test_series').exists():\n    ser_csv = pd.read_csv(COMP / 'train_series.csv')\nser_csv = ser_csv.loc[:, ~ser_csv.columns.duplicated()]\nstudies = sub_df.StudyInstanceUID.tolist()\nby = {s: g.to_dict('records') for s, g in ser_csv[ser_csv.StudyInstanceUID.isin(set(studies))].groupby('StudyInstanceUID')}\nprint(f'{len(studies):,} test studies, {len(by):,} with series metadata')"},{"cell_type":"code","execution_count":null,"id":"7490e913","metadata":{},"outputs":[],"source":"N_SLOT_TYPES, MASK_IDX = (6, 0)\n\ndef segment_softmax(scores, sidx, B):\n    T, K = scores.shape\n    idx = sidx.unsqueeze(1).expand(-1, K)\n    m = torch.full((B, K), float('-inf'), device=scores.device, dtype=scores.dtype)\n    m = m.scatter_reduce(0, idx, scores, reduce='amax', include_self=True)\n    e = (scores - m[sidx]).exp()\n    s = torch.zeros(B, K, device=scores.device, dtype=scores.dtype).index_add_(0, sidx, e)\n    return e / s[sidx].clamp(min=1e-06)\n\nclass MeanMaxPool(nn.Module):\n\n    def forward(self, f, sidx, B, slot=None, return_attn=False):\n        D = f.shape[1]\n        cnt = torch.zeros(B, device=f.device, dtype=f.dtype).index_add_(0, sidx, torch.ones(f.shape[0], device=f.device, dtype=f.dtype))\n        mean = torch.zeros(B, D, device=f.device, dtype=f.dtype).index_add_(0, sidx, f)\n        mean = mean / cnt.clamp(min=1).unsqueeze(1)\n        mx = torch.full((B, D), -10000.0, device=f.device, dtype=f.dtype)\n        mx = mx.scatter_reduce(0, sidx.unsqueeze(1).expand(-1, D), f, reduce='amax', include_self=True)\n        return (torch.cat([mean, mx], 1), None)\n\nclass LabelAttentionPool(nn.Module):\n\n    def __init__(self, d, n_labels=12, n_heads=4, slot_bias=True):\n        super().__init__()\n        self.d, self.k, self.h = (d, n_labels, n_heads)\n        self.q = nn.Parameter(torch.randn(n_labels, d) * 0.02)\n        self.key, self.val = (nn.Linear(d, d), nn.Linear(d, d))\n        self.slot_bias = nn.Parameter(torch.zeros(n_labels, N_SLOT_TYPES + 1)) if slot_bias else None\n\n    def forward(self, f, sidx, B, slot=None, return_attn=False):\n        scores = self.key(f) @ self.q.t() / self.d ** 0.5\n        if self.slot_bias is not None and slot is not None:\n            scores = scores + self.slot_bias.t()[slot]\n        a = segment_softmax(scores, sidx, B)\n        out = torch.zeros(B, self.k, self.d, device=f.device, dtype=f.dtype)\n        out = out.index_add_(0, sidx, a.unsqueeze(-1) * self.val(f).unsqueeze(1))\n        return (out, a)\n\nclass TokenXAttnPool(nn.Module):\n\n    def __init__(self, d, n_labels=12, n_heads=6, dropout=0.2):\n        super().__init__()\n        self.d, self.k = (d, n_labels)\n        self.q = nn.Parameter(torch.randn(n_labels, d) * 0.02)\n        self.slot_emb = nn.Embedding(N_SLOT_TYPES + 1, d, padding_idx=0)\n        self.kv_norm = nn.LayerNorm(d)\n        self.attn = nn.MultiheadAttention(d, n_heads, dropout=dropout, batch_first=True)\n\n    def forward(self, tok, sidx, B, slot=None, return_attn=False):\n        T, N, D = tok.shape\n        cnt = torch.bincount(sidx, minlength=B)\n        S = int(cnt.max().item())\n        starts = torch.cumsum(cnt, 0) - cnt\n        pos = torch.arange(T, device=tok.device) - starts[sidx]\n        kv = tok + self.slot_emb(slot).unsqueeze(1)\n        pad = tok.new_zeros(B, S, N, D)\n        pad[sidx, pos] = kv\n        keep = torch.zeros(B, S, dtype=torch.bool, device=tok.device)\n        keep[sidx, pos] = True\n        kpm = ~keep.repeat_interleave(N, dim=1)\n        pad = self.kv_norm(pad.reshape(B, S * N, D))\n        q = self.q.unsqueeze(0).expand(B, -1, -1)\n        att, w = self.attn(q, pad, pad, key_padding_mask=kpm, need_weights=return_attn, average_attn_weights=True)\n        cls = tok[:, 0]\n        mean = torch.zeros(B, D, device=tok.device, dtype=tok.dtype).index_add_(0, sidx, cls) / cnt.clamp(min=1).unsqueeze(1)\n        mx = torch.full((B, D), -10000.0, device=tok.device, dtype=tok.dtype)\n        mx = mx.scatter_reduce(0, sidx.unsqueeze(1).expand(-1, D), cls, reduce='amax', include_self=True)\n        base = torch.cat([mean, mx], 1).unsqueeze(1).expand(-1, self.k, -1)\n        return (torch.cat([att, base], -1), w)\n\nclass ViTSlotToken(nn.Module):\n\n    def __init__(self, vit, n_cat, dim=None):\n        super().__init__()\n        self.vit = vit\n        d = dim or vit.embed_dim\n        self.tok = nn.Embedding(n_cat + 1, d, padding_idx=MASK_IDX)\n        self.num_features = vit.num_features\n        self._orig_prefix = getattr(vit, 'num_prefix_tokens', 1)\n        vit.num_prefix_tokens = self._orig_prefix + 1\n        for blk in vit.blocks:\n            a = getattr(blk, 'attn', None)\n            if a is not None and hasattr(a, 'num_prefix_tokens'):\n                a.num_prefix_tokens = a.num_prefix_tokens + 1\n\n    @staticmethod\n    def _maybe(mod, x):\n        return x if mod is None else mod(x)\n\n    def forward_features(self, x, cat):\n        v = self.vit\n        x = v.patch_embed(x)\n        pos = v._pos_embed(x)\n        rope = None\n        if isinstance(pos, tuple):\n            x, rope = pos\n        else:\n            x = pos\n        x = self._maybe(getattr(v, 'patch_drop', None), x)\n        x = self._maybe(getattr(v, 'norm_pre', None), x)\n        npt = self._orig_prefix\n        tok = self.tok(cat).unsqueeze(1)\n        x = torch.cat([x[:, :npt], tok, x[:, npt:]], dim=1)\n        if rope is not None:\n            if getattr(v, 'rope_mixed', False):\n                for i, blk in enumerate(v.blocks):\n                    x = blk(x, rope=rope[i])\n            else:\n                for blk in v.blocks:\n                    x = blk(x, rope=rope)\n        else:\n            x = v.blocks(x)\n        return v.norm(x)\n\n    def forward_head(self, x, pre_logits=True):\n        return self.vit.forward_head(x, pre_logits=pre_logits)\nIMAGENET_MEAN = (0.485, 0.456, 0.406)\nIMAGENET_STD = (0.229, 0.224, 0.225)\n\nclass _GatedDepthBlock(nn.Module):\n\n    def __init__(self, n_slice, dropout=0.0, ls_init=0.1):\n        super().__init__()\n        self.norm = nn.GroupNorm(1, n_slice)\n        self.v = nn.Conv2d(n_slice, n_slice, 1)\n        self.g = nn.Conv2d(n_slice, n_slice, 1)\n        self.out = nn.Conv2d(n_slice, n_slice, 1)\n        self.gamma = nn.Parameter(torch.full((n_slice, 1, 1), ls_init))\n        self.drop = nn.Dropout2d(dropout) if dropout else nn.Identity()\n\n    def forward(self, x):\n        z = self.norm(x)\n        return x + self.gamma * self.drop(self.out(self.v(z) * F.silu(self.g(z))))\n\nclass DepthCompress(nn.Module):\n\n    def __init__(self, n_slice=16, out_ch=3, depth=1, dropout=0.0, ls_init=0.1, imagenet=True, proj_noise=0.25):\n        super().__init__()\n        self.imagenet = imagenet\n        self.blocks = nn.ModuleList([_GatedDepthBlock(n_slice, dropout, ls_init) for _ in range(depth)])\n        self.proj = nn.Conv2d(n_slice, out_ch, 1, bias=True)\n        if imagenet:\n            self.register_buffer('mu', torch.tensor(IMAGENET_MEAN).view(1, -1, 1, 1))\n            self.register_buffer('sd', torch.tensor(IMAGENET_STD).view(1, -1, 1, 1))\n\n    def forward(self, x):\n        keep = (x.amax(dim=1, keepdim=True) > 0).to(x.dtype)\n        z = x\n        for b in self.blocks:\n            z = b(z)\n        z = self.proj(z)\n        if self.imagenet:\n            z = (z - self.mu.to(z.dtype)) / self.sd.to(z.dtype)\n        return z * keep\nN_PLANE, N_CONTRAST = (3, 2)\n_PLANE_OF = lambda s: torch.clamp(s - 1, 0, 5) // 2\n_CONTRAST_OF = lambda s: torch.clamp(s - 1, 0, 5) % 2\n\nclass SlotDepthMixer(nn.Module):\n\n    def __init__(self, n_slice=16, ksize=5, alpha_max=0.25):\n        super().__init__()\n        self.n_slice, self.ksize, self.r = (n_slice, ksize, ksize // 2)\n        self.alpha_max = alpha_max\n        b = torch.tensor([1.0, 4.0, 6.0, 4.0, 1.0])\n        self.register_buffer('base', b.log()[self.r:])\n        n_u = self.r + 1\n        self.shared = nn.Parameter(torch.zeros(n_u))\n        self.plane_k = nn.Parameter(torch.zeros(N_PLANE, n_u))\n        self.contrast_k = nn.Parameter(torch.zeros(N_CONTRAST, n_u))\n        self.g0 = nn.Parameter(torch.zeros(()))\n        self.gate_p = nn.Parameter(torch.zeros(N_PLANE))\n        self.gate_c = nn.Parameter(torch.zeros(N_CONTRAST))\n        idx = torch.arange(n_slice)\n        self.register_buffer('off', idx[None, :] - idx[:, None])\n\n    def kernel(self, slot):\n        p, c = (_PLANE_OF(slot), _CONTRAST_OF(slot))\n        half = self.base + self.shared + self.plane_k[p] + self.contrast_k[c]\n        full = torch.cat([half.flip(-1)[..., :self.r], half], dim=-1)\n        return F.softmax(full, dim=-1)\n\n    def alpha(self, slot):\n        p, c = (_PLANE_OF(slot), _CONTRAST_OF(slot))\n        return self.alpha_max * torch.tanh(self.g0 + self.gate_p[p] + self.gate_c[c])\n\n    def forward(self, x, slot, vmask):\n        T, S, H, W = x.shape\n        if vmask is None:\n            raise ValueError('stem=mixer requires the padding mask')\n        k = self.kernel(slot)\n        v = vmask.to(k.dtype)\n        d = self.off + self.r\n        inb = (d >= 0) & (d < self.ksize)\n        kk = k[:, d.clamp(0, self.ksize - 1)] * inb\n        M = kk * v[:, None, :]\n        den = M.sum(-1, keepdim=True)\n        eye = torch.eye(S, device=x.device, dtype=M.dtype).expand(T, S, S)\n        ok = (den > 1e-06) & v[:, :, None].bool()\n        M = torch.where(ok, M / den.clamp(min=1e-06), eye)\n        a = self.alpha(slot)[:, None, None]\n        Aop = ((1.0 - a) * eye + a * M).to(x.dtype)\n        if x.is_contiguous(memory_format=torch.channels_last) and (not x.is_contiguous()):\n            y = torch.bmm(x.permute(0, 2, 3, 1).reshape(T, H * W, S), Aop.transpose(1, 2))\n            return y.reshape(T, H, W, S).permute(0, 3, 1, 2)\n        return torch.bmm(Aop, x.reshape(T, S, H * W)).reshape(T, S, H, W)\n\ndef _seg_mean_max(v, sidx, B):\n    D = v.shape[1]\n    cnt = torch.zeros(B, device=v.device, dtype=v.dtype).index_add_(0, sidx, torch.ones(v.shape[0], device=v.device, dtype=v.dtype))\n    mean = torch.zeros(B, D, device=v.device, dtype=v.dtype).index_add_(0, sidx, v)\n    mean = mean / cnt.clamp(min=1).unsqueeze(1)\n    mx = torch.full((B, D), -10000.0, device=v.device, dtype=v.dtype)\n    mx = mx.scatter_reduce(0, sidx.unsqueeze(1).expand(-1, D), v, reduce='amax', include_self=True)\n    return torch.cat([mean, mx], 1)\n\ndef _pad_kv(x, sidx, B, norm):\n    T, P, D = x.shape\n    cnt = torch.bincount(sidx, minlength=B)\n    S = int(cnt.max().item())\n    starts = torch.cumsum(cnt, 0) - cnt\n    pos = torch.arange(T, device=x.device) - starts[sidx]\n    pad = x.new_zeros(B, S, P, D)\n    pad[sidx, pos] = x\n    keep = torch.zeros(B, S, dtype=torch.bool, device=x.device)\n    keep[sidx, pos] = True\n    return (norm(pad.reshape(B, S * P, D)), ~keep.repeat_interleave(P, dim=1))\n\nclass _GatedDelta(nn.Module):\n\n    def __init__(self, d, n_labels, n_heads, dropout):\n        super().__init__()\n        self.q = nn.Parameter(torch.randn(n_labels, d) * 0.02)\n        self.kv_norm = nn.LayerNorm(d)\n        self.attn = nn.MultiheadAttention(d, n_heads, dropout=dropout, batch_first=True)\n        self.d_norm = nn.LayerNorm(d)\n        self.dw = nn.Parameter(torch.randn(n_labels, d) * (1.0 / d ** 0.5))\n        self.db = nn.Parameter(torch.zeros(n_labels))\n        self.gate = nn.Parameter(torch.zeros(n_labels))\n\n    def delta(self, pat, sidx, B, return_attn):\n        kv, kpm = _pad_kv(pat, sidx, B, self.kv_norm)\n        q = self.q.unsqueeze(0).expand(B, -1, -1)\n        att, w = self.attn(q, kv, kv, key_padding_mask=kpm, need_weights=return_attn, average_attn_weights=True)\n        return ((self.d_norm(att) * self.dw).sum(-1) + self.db, w)\n\nclass TokenResidualPool(_GatedDelta):\n\n    def __init__(self, d, n_labels=12, n_heads=6, pe=64, dropout=0.2):\n        super().__init__(d, n_labels, n_heads, dropout)\n        self.base = nn.Sequential(nn.LayerNorm(2 * d + pe), nn.Dropout(dropout), nn.Linear(2 * d + pe, n_labels))\n\n    def forward(self, tok, slot, sidx, B, pres, return_attn=False):\n        base = self.base(torch.cat([_seg_mean_max(tok[:, 1:].mean(1), sidx, B), pres], 1))\n        d_, w = self.delta(tok[:, 1:], sidx, B, return_attn)\n        return (base + self.gate * d_, w)\n\nclass CodexResidualPool(_GatedDelta):\n\n    def __init__(self, d, n_labels=12, n_heads=6, pe=64, dropout=0.2):\n        super().__init__(d, n_labels, n_heads, dropout)\n        self.base = nn.Sequential(nn.LayerNorm(2 * d + pe), nn.Dropout(dropout), nn.Linear(2 * d + pe, n_labels))\n\n    def forward(self, tok, slot, sidx, B, pres, return_attn=False):\n        base = self.base(torch.cat([_seg_mean_max(tok[:, 0], sidx, B), pres], 1))\n        d_, w = self.delta(tok[:, 1:], sidx, B, return_attn)\n        return (base + self.gate * d_, w)\n\nclass ClsAddPool(nn.Module):\n\n    def __init__(self, d, n_labels=12, pe=64, dropout=0.2):\n        super().__init__()\n        self.net = nn.Sequential(nn.LayerNorm(4 * d + pe), nn.Dropout(dropout), nn.Linear(4 * d + pe, n_labels))\n\n    def forward(self, tok, slot, sidx, B, pres, return_attn=False):\n        return (self.net(torch.cat([_seg_mean_max(tok[:, 1:].mean(1), sidx, B), _seg_mean_max(tok[:, 0], sidx, B), pres], 1)), None)\n\nclass Readout(nn.Module):\n\n    def __init__(self, pool, d, n_labels=12, pe=64):\n        super().__init__()\n        self.pool_kind, self.k = (pool, n_labels)\n        self.pres_emb = nn.Embedding(N_SLOT_TYPES + 1, pe, padding_idx=0)\n        if pool in ('xres', 'clsadd', 'xcodex'):\n            self.pool = {'xres': TokenResidualPool, 'clsadd': ClsAddPool, 'xcodex': CodexResidualPool}[pool](d, n_labels, pe=pe)\n        elif pool in ('attn', 'xattn'):\n            if pool == 'xattn':\n                self.pool = TokenXAttnPool(d, n_labels)\n                wd = 3 * d + pe\n            else:\n                self.pool = LabelAttentionPool(d, n_labels)\n                wd = d + pe\n            self.norm = nn.LayerNorm(wd)\n            self.w = nn.Parameter(torch.randn(n_labels, wd) * (1.0 / wd ** 0.5))\n            self.b = nn.Parameter(torch.zeros(n_labels))\n        else:\n            self.pool = MeanMaxPool()\n            self.net = nn.Sequential(nn.LayerNorm(2 * d + pe), nn.Dropout(0.2), nn.Linear(2 * d + pe, n_labels))\n        self.drop = nn.Dropout(0.2)\n\n    def forward(self, f, slot, sidx, B, return_attn=False):\n        pe = self.pres_emb(slot)\n        pres = torch.zeros(B, pe.shape[1], device=f.device, dtype=f.dtype).index_add_(0, sidx, pe)\n        if self.pool_kind in ('xres', 'clsadd', 'xcodex'):\n            return self.pool(f, slot, sidx, B, pres)[0]\n        pooled, attn = self.pool(f, sidx, B, slot=slot, return_attn=return_attn)\n        if self.pool_kind in ('attn', 'xattn'):\n            x = torch.cat([pooled, pres.unsqueeze(1).expand(-1, self.k, -1)], -1)\n            x = self.drop(self.norm(x))\n            return (x * self.w).sum(-1) + self.b\n        return self.net(torch.cat([pooled, pres], 1))\n\nclass Net(nn.Module):\n\n    def __init__(self, enc, cond, n_meta=0, pool='mean_max', stem='native', n_slice=16):\n        super().__init__()\n        self.enc, self.cond = (enc, cond)\n        self.compress = DepthCompress(n_slice, 3) if stem == 'compress' else None\n        self.mixer = SlotDepthMixer(n_slice) if stem == 'mixer' else None\n        self.tokens = pool in ('xattn', 'xres', 'clsadd', 'xcodex')\n        D = enc.num_features\n        self.meta_mlp = nn.Sequential(nn.LayerNorm(n_meta), nn.Linear(n_meta, 128), nn.GELU(), nn.Linear(128, D)) if n_meta > 0 else None\n        self.readout = Readout(pool, D)\n        if cond == 'post':\n            self.slot_emb = nn.Embedding(N_SLOT_TYPES + 1, D, padding_idx=MASK_IDX)\n\n    def forward(self, im, slot, smeta, sidx, B, vm=None):\n        if self.mixer is not None:\n            im = self.mixer(im, slot, vm)\n        if self.compress is not None:\n            im = self.compress(im)\n        f = self.enc.forward_features(im, slot) if self.cond == 'token' else self.enc.forward_features(im)\n        if self.tokens:\n            inner = getattr(self.enc, 'vit', self.enc)\n            orig = getattr(self.enc, '_orig_prefix', getattr(inner, 'num_prefix_tokens', 1))\n            f = torch.cat([f[:, :1], f[:, orig:]], 1)\n        else:\n            f = self.enc.forward_head(f, pre_logits=True)\n            if f.dim() > 2:\n                f = f.flatten(1)\n        ex = (lambda v: v.unsqueeze(1)) if self.tokens else lambda v: v\n        if self.cond == 'post':\n            f = f + ex(self.slot_emb(slot))\n        if self.meta_mlp is not None and smeta.shape[1] > 0:\n            mt = self.meta_mlp(smeta)\n            f = torch.cat([f, mt.unsqueeze(1)], 1) if self.tokens else f + mt\n        return self.readout(f, slot, sidx, B)\nmodels = []\nfor ckpt_path in sorted(CKPT.glob('*_f*.pt')):\n    z = torch.load(ckpt_path, map_location='cpu', weights_only=False)\n    cfg = z['cfg']\n    _stem = cfg.get('stem', 'native')\n    _in = 3 if _stem == 'compress' else cfg.get('n_slice', 16)\n    enc = timm.create_model(cfg['backbone'], pretrained=False, num_classes=0, in_chans=_in, **{'img_size': cfg['img']} if 'vit_' in cfg['backbone'] else {})\n    if cfg['cond'] == 'token':\n        enc = ViTSlotToken(enc, N_SLOT_TYPES)\n    m = Net(enc, cfg['cond'], cfg.get('n_meta', 0), cfg['pool'], stem=_stem, n_slice=cfg.get('n_slice', 16))\n    missing, unexpected = m.load_state_dict(z['state_dict'], strict=False)\n    assert not [k for k in missing if not k.startswith('enc.')], f'missing {missing[:5]}'\n    assert not unexpected, f'unexpected {unexpected[:5]}'\n    models.append(m.eval())\n    print(f\"loaded {ckpt_path.name}  fold {z['fold']}  {cfg['backbone']} pool={cfg['pool']} meta={cfg['meta']}\")\nCFG = cfg\nassert CFG.get('n_meta', 0) == 0, f\"checkpoint expects {CFG['n_meta']} metadata features -- build slot_meta for the TEST studies and pass it to predict() before submitting\"\nprint(f\"\\n{len(models)} fold models ready | input norm: {CFG.get('norm', 'none')}\")"},{"cell_type":"code","execution_count":null,"id":"ab362eba","metadata":{},"outputs":[],"source":"AMP_PREF = 'bf16'\n\ndef amp_for(dev):\n    if not str(dev).startswith('cuda'):\n        return (torch.float32, False)\n    cc = torch.cuda.get_device_capability(dev)\n    if AMP_PREF == 'bf16':\n        return (torch.bfloat16, True)\n    if AMP_PREF == 'fp16':\n        return (torch.float16, True)\n    if AMP_PREF == 'fp32':\n        return (torch.float32, False)\n    return (torch.bfloat16 if cc >= (8, 0) else torch.float16, True)\nAMP_DT, AMP_ON = amp_for(DEV)\nWORKERS = max(1, min(4, os.cpu_count() or 4))\nCHUNK = 48\nMICRO = 8\nmodels = [m.to(DEV).eval() for m in models]\nprint(f\"device {DEV} | amp {str(AMP_DT).split('.')[-1]} (on={AMP_ON}) | workers {WORKERS} | chunk {CHUNK} | micro {MICRO}\")\n\ndef _norm_(im):\n    k = CFG.get('norm', 'none')\n    if k == 'zscore':\n        m = (im > 0).float()\n        n = m.sum(dim=(1, 2, 3), keepdim=True).clamp(min=1.0)\n        mu = (im * m).sum(dim=(1, 2, 3), keepdim=True) / n\n        var = (((im - mu) * m) ** 2).sum(dim=(1, 2, 3), keepdim=True) / n\n        return (im - mu) / (var.sqrt() + 1e-06) * m\n    if k == 'imagenet':\n        m = (im > 0).float()\n        return (im - 0.485) / 0.229 * m\n    return im\n\n@torch.no_grad()\ndef _micro(images, masks):\n    dev = DEV\n    ims, slots, sidx, vms = ([], [], [], [])\n    for b in range(len(masks)):\n        present = np.nonzero(masks[b] > 0)[0]\n        if len(present) == 0:\n            continue\n        blk = images[b][present]\n        ims.append(torch.from_numpy(blk))\n        vms.append(torch.from_numpy(blk.reshape(blk.shape[0], blk.shape[1], -1).max(2) > 0))\n        slots.append(torch.from_numpy(present + 1).long())\n        sidx.append(torch.full((len(present),), b, dtype=torch.long))\n    out = np.full((len(models), len(masks), len(LABELS)), np.nan, np.float32)\n    if not ims:\n        return out\n    im = _norm_(torch.cat(ims).to(dev, non_blocking=True).float().div_(255.0))\n    sl = torch.cat(slots).to(dev)\n    si = torch.cat(sidx).to(dev)\n    vm = torch.cat(vms).to(dev)\n    sm = torch.zeros(len(sl), CFG.get('n_meta', 0), device=dev)\n    per = torch.zeros(len(models), len(masks), len(LABELS), device=dev, dtype=torch.float32)\n    with torch.autocast('cuda' if str(dev).startswith('cuda') else 'cpu', dtype=AMP_DT, enabled=AMP_ON):\n        for _mi, m in enumerate(models):\n            per[_mi] = torch.sigmoid(m(im, sl, sm, si, len(masks), vm=vm).float())\n    got = per.cpu().numpy()\n    keep = np.array([(masks[b] > 0).any() for b in range(len(masks))])\n    out[:, keep] = got[:, keep]\n    return out\n\ndef predict(images, masks):\n    out = np.full((len(models), len(masks), len(LABELS)), np.nan, np.float32)\n    for a in range(0, len(masks), MICRO):\n        b = min(a + MICRO, len(masks))\n        out[:, a:b] = _micro(images[a:b], masks[a:b])\n    return out\n# Per-fold predictions are kept whole: the metric is macro ROC-AUC, so folds are\n# combined on RANKS across the full test set, matching what the DINOv2 frontier\n# and the RadImageNet stage already do. Averaging probabilities first lets a\n# fold with a shifted output range dominate the mean. (Observation due to\n# romantamrazov, RSNA Knee | DINOsaur V2.)\npreds = np.full((len(models), len(studies), len(LABELS)), np.nan, np.float32)\nt0, done = (time.time(), 0)\nwith ProcessPoolExecutor(max_workers=WORKERS) as ex:\n    for c0 in range(0, len(studies), CHUNK):\n        block = studies[c0:c0 + CHUNK]\n        imgs = np.zeros((len(block), N_SLOT, N_SLICE, SIZE, SIZE), np.uint8)\n        msks = np.zeros((len(block), N_SLOT), np.uint8)\n        futs = [ex.submit(build_study, (i, s, by.get(s, []))) for i, s in enumerate(block)]\n        for f in as_completed(futs):\n            try:\n                i, a, k = f.result()\n                imgs[i], msks[i] = (a, k)\n            except Exception as e:\n                print(f'  study failed: {type(e).__name__}: {e}')\n        preds[:, c0:c0 + len(block)] = predict(imgs, msks)\n        done += len(block)\n        el = time.time() - t0\n        print(f'  {done:,}/{len(studies):,}  {el / 60:.1f}m  eta {el / done * (len(studies) - done) / 60:.1f}m', flush=True)\n        del imgs, msks\n        gc.collect()\nprint(f'\\ninference done in {(time.time() - t0) / 60:.1f} min')\nA5_W = 0.45\nA5_LABELS = list(LABELS)\n_a5_ok = np.isfinite(preds).all(axis=(0, 2))\n_a5_rankavg = np.zeros((len(studies), len(LABELS)), np.float64)\nfor _f in range(preds.shape[0]):\n    _blk = preds[_f][_a5_ok]\n    _o = _blk.argsort(0).argsort(0).astype(np.float64)\n    _a5_rankavg[_a5_ok] += _o / max(len(_blk) - 1, 1)\n_a5_rankavg /= preds.shape[0]\n_a5_rankavg[~_a5_ok] = np.nan\nprint(f'a5: rank-averaged {preds.shape[0]} folds over {int(_a5_ok.sum()):,} studies')\nA5_PREDS = dict(zip(sub_df['StudyInstanceUID'].astype(str), _a5_rankavg.astype(np.float32)))\nfor _a5k, _a5v in _A5_SAVED.items():\n    globals()[_a5k] = _a5v\ndel _A5_SAVED, _a5k, _a5v"},{"cell_type":"code","execution_count":null,"id":"e3cb6256","metadata":{},"outputs":[],"source":"_a5_sub = pd.read_csv('/kaggle/working/submission.csv',\n                      dtype={'StudyInstanceUID': str})\nassert _a5_sub.columns.tolist()[1:] == A5_LABELS, 'submission schema drift'\nif A5_W > 0:\n    _a5_ours = np.stack([A5_PREDS[_u]\n                         for _u in _a5_sub['StudyInstanceUID'].astype(str)])\n    _a5_base_rank = _a5_sub[A5_LABELS].rank(method='average', pct=True)\n    _a5_ours_rank = pd.DataFrame(_a5_ours, columns=A5_LABELS,\n                                 index=_a5_sub.index).rank(method='average', pct=True)\n    _a5_sub[A5_LABELS] = (1.0 - A5_W) * _a5_base_rank + A5_W * _a5_ours_rank\n    assert np.isfinite(_a5_sub[A5_LABELS].to_numpy()).all()\n    _a5_sub.to_csv('/kaggle/working/submission.csv', index=False)"},{"id":"8a50740e-d5db-4c50-b964-f08b1a6e14e8","cell_type":"code","source":"# E9: independent RadImageNet ResNet-50 arm for the verified E2 parent.\n#\n# Adapted 2026-08-11 from the V52 cell in the public Kaggle competition notebook\n# prvsiyan/rsna-knee-read-the-report-then-the-knee (latest source SHA-256\n# b54aa529f38dc6f594478e7975d86459ddbff898453a2c2633f4c3be4b909e61).\n# Kaggle's public-code rule deems public Competition Code open-source; Meta Kaggle\n# documents public notebooks under Apache-2.0. Changes here remove the unavailable B3\n# arm, pin the public E2 OOF bundle, add per-target/no-regression gates, and preserve E2\n# byte-for-byte on every failure. This module is appended as the final notebook cell;\n# its imports and DICOM helpers are supplied by the parent notebook.\nSLOTS = [\n    (\"SAG_FS\", \"Sagittal\", None, True),\n    (\"COR_FS\", \"Coronal\", None, True),\n    (\"AX_FS\", \"Axial\", None, True),\n]\nN_SLOT = len(SLOTS)\nCACHE_SLICES = 8\nTIME_BUDGET = 8.72 * 3600\nIMG = CACHE_IMG = 224\n# Match the released RadImageNet model's full-frame pretraining. Setting a crop\n# larger than every acquisition disables the optional physical crop in read_slot.\nCROP_MM = 10_000.0\nSLICE_BAND = (0.12, 0.88)\n# The audited public OOF was produced after the notebook's legacy member group, which\n# left this process-global pixel contract active. Pin it instead of inheriting whichever\n# E2 group happened to run last. This uses public preprocessing code only; no legacy\n# checkpoint or unknown-license asset is attached.\nRULES = dict(RULES_LEGACY)\nTOKEN_DIM = 2048             # official ResNet-50 global-average feature\nHEAD_DIM = 512\nPINNED_HEADS_SHA256 = \"0f465649799ecfbccaac1767844639e7ced44e1bc9babde6e4bac7c5d9b89eaa\"\nPINNED_REMOTE_AUDIT_SHA256 = \"267f948078710d3ca8a6f0de4ce0a5e75e850e1f08d36451549a137d878a6fe8\"\nPINNED_PUBLIC_DIAGNOSTIC_SHA256 = \"0f2f82fb40f0570d6766f73b0d7f51489df6d0faa8fd6e4c0f45bcd6c4c7b283\"\nPINNED_E9B_CONTRACT_SHA256 = \"6777c0a0ba7dd044752fac948752dc39e9ca35b3c280fe74ee6d27a5865d87e7\"\n# E10 repairs E9's censored search: its alpha grid stopped at 0.25 and four of five outer\n# folds selected that ceiling, so the deployed 0.20 was a boundary artifact rather than an\n# optimum. The ladder, the two-source per-target gains and every deployable weight map live\n# in the hash-pinned contract; this cell recomputes the remote half in-kernel before use.\n# The contract also carries the held-out form of E10's own weight choice: selecting the rung\n# on four grouped folds and scoring the fifth picks 0.60 (public) and 0.70 (v15) in all five\n# outer folds, and never picks 0.20. So every deployable rung at or below 0.35 is below what\n# honest selection would choose, which is the answer to \"you tuned on the 58 gold rows\".\nPINNED_E10_CONTRACT_SHA256 = \"219c91f40905181c222e2966b3fed01a96570ddfb64862357d5fd6cad500cd45\"\nE10_CONFIG = \"uniform_060\"\nE10_PRESERVED_TARGETS = [\"Baker's\", \"Fracture\"]\n\n# E11 trains a third arm whose diversity is in the pixels rather than in the weights. The\n# existing arm reads three fat-suppressed slots at full frame; every one of E2's twenty\n# members reads one DINOv2 recipe. Nothing in the portfolio has yet looked at a\n# non-fat-suppressed series, where meniscal and ligament morphology is conventionally read,\n# and nothing has given RadImageNet a physically normalised field of view. E11 changes both:\n# three non-suppressed slots plus one suppressed anchor, cropped to 130 mm, which the parent\n# notebook establishes is below the acquired field of view of 99.6% of series while still\n# containing the joint. It is a training mode only; it never touches the submission.\nARM_MODE = \"e11\"\nE11_SLOTS = [\n    (\"SAG_NOFS\", \"Sagittal\", None, False),\n    (\"COR_NOFS\", \"Coronal\", None, False),\n    (\"AX_NOFS\", \"Axial\", None, False),\n    (\"SAG_FS\", \"Sagittal\", None, True),\n]\nE11_CROP_MM = 130.0\nE11_CACHE_SLICES = 8\nE11_IMG = 224\n# Availability of non-suppressed series per plane is unmeasured, so a fill floor rather than\n# the parent's 90% rule: below this the run has found something structurally wrong, above it\n# an empty slot is simply masked out of the token set like any other absent series.\nE11_MIN_FILL = 0.45\n\n\ndef _v52_as_bool(value):\n    if pd.isna(value):\n        return None\n    text = str(value).strip().upper()\n    if text in {\"1\", \"TRUE\", \"T\", \"YES\", \"Y\"}:\n        return True\n    if text in {\"0\", \"FALSE\", \"F\", \"NO\", \"N\"}:\n        return False\n    try:\n        number = float(text)\n        return True if number == 1 else False if number == 0 else None\n    except Exception:\n        return None\n\n\ndef audit_official_sequence_metadata(inferred, official):\n    \"\"\"Audit metadata agreement without changing the checkpoint pixel contract.\"\"\"\n    needed = {\"SeriesInstanceUID\", \"Fluid_Sensitive\", \"Fat_Suppression\"}\n    if inferred.empty or official.empty or not needed.issubset(official.columns):\n        return\n    inferred_flags = inferred[[\"SeriesInstanceUID\", \"fluid\", \"fatsat\"]].copy()\n    official_flags = official[\n        [\"SeriesInstanceUID\", \"Fluid_Sensitive\", \"Fat_Suppression\"]\n    ].copy()\n    official_flags[\"official_fluid\"] = official_flags[\"Fluid_Sensitive\"].map(\n        _v52_as_bool\n    )\n    official_flags[\"official_fatsat\"] = official_flags[\"Fat_Suppression\"].map(\n        _v52_as_bool\n    )\n    merged = inferred_flags.merge(\n        official_flags[[\"SeriesInstanceUID\", \"official_fluid\", \"official_fatsat\"]],\n        on=\"SeriesInstanceUID\",\n        how=\"inner\",\n    )\n    for inferred_col, official_col, name in [\n        (\"fluid\", \"official_fluid\", \"Fluid_Sensitive\"),\n        (\"fatsat\", \"official_fatsat\", \"Fat_Suppression\"),\n    ]:\n        valid = merged[official_col].notna() & merged[inferred_col].notna()\n        if valid.any():\n            agreement = (\n                merged.loc[valid, inferred_col].astype(bool).to_numpy()\n                == merged.loc[valid, official_col].astype(bool).to_numpy()\n            ).mean()\n            log(\n                f\"V52 metadata audit {name}: {agreement:.1%} agreement \"\n                f\"on {int(valid.sum())} series\"\n            )\n\n\ndef find_input_file(name):\n    for root, dirs, files in os.walk(\"/kaggle/input\"):\n        dirs[:] = [d for d in dirs if d not in (\"train_series\", \"test_series\")]\n        if name in files:\n            return Path(root) / name\n    raise FileNotFoundError(name)\n\n\ndef find_input_dir(name):\n    for root, dirs, files in os.walk(\"/kaggle/input\"):\n        if Path(root).name == name:\n            return Path(root)\n    raise FileNotFoundError(name)\n\n\ndef make_targets(train):\n    \"\"\"Three independent public report teachers; image-read gold always wins.\"\"\"\n    uid = \"StudyInstanceUID\"\n    sources = [\n        pd.read_csv(find_input_file(\"report_labels_v2.csv\")),\n        pd.read_csv(find_input_file(\"llm_labels_v2.csv\")),\n        pd.read_csv(find_input_file(\"labels_llm_gpt56sol.csv\")),\n    ]\n    cube = []\n    for frame in sources:\n        if frame[uid].duplicated().any():\n            raise ValueError(\"duplicate study in report-label source\")\n        aligned = train[[uid]].merge(frame[[uid] + TARGETS], on=uid, how=\"left\")\n        cube.append(aligned[TARGETS].to_numpy(float))\n    cube = np.stack(cube)\n    available = np.isfinite(cube).sum(0)\n    if np.any(available < 2):\n        raise ValueError(\"fewer than two report teachers for a study/target\")\n    y = np.nanmean(cube, axis=0).astype(np.float32)\n    disagreement = np.nanmean(np.abs(cube - y[None]), axis=0)\n    agreement = np.clip(1.0 - 2.0 * disagreement, 0, 1)\n    certainty = np.clip(2.0 * np.abs(y - .5), 0, 1)\n    w = (.15 + .85 * (.65 * agreement + .35 * certainty)).astype(np.float32)\n    gold = train[TARGETS].notna().all(axis=1).to_numpy()\n    y[gold] = train.loc[gold, TARGETS].to_numpy(np.float32)\n    w[gold] = 3.0\n    return y, w, gold\n\n\ndef report_groups(train):\n    report = (train.Report.fillna(\"\").astype(str).str.lower()\n              .str.replace(r\"\\s+\", \" \", regex=True).str.strip())\n    return np.array([hashlib.sha256(x.encode()).hexdigest()[:24] for x in report])\n\n\ndef _v52_sha256(path):\n    digest = hashlib.sha256()\n    with open(path, \"rb\") as handle:\n        for chunk in iter(lambda: handle.read(8 << 20), b\"\"):\n            digest.update(chunk)\n    return digest.hexdigest()\n\n\ndef load_radimagenet(device):\n    \"\"\"Strictly load the official RadImageNet ResNet-50 PyTorch checkpoint.\"\"\"\n    from torchvision.models import resnet50\n\n    checkpoint = find_input_file(\"ResNet50.pt\")\n    expected_checkpoint = \"08629f7e7bd3e29b8ee9522ca3f65ce4d010a7ddf74f0ea3c7e3f3d0bbab0734\"\n    observed_checkpoint = _v52_sha256(checkpoint)\n    if observed_checkpoint != expected_checkpoint:\n        raise RuntimeError(f\"RadImageNet checkpoint drift: {observed_checkpoint}\")\n\n    class RadImageNetEncoder(nn.Module):\n        def __init__(self):\n            super().__init__()\n            self.backbone = nn.Sequential(\n                *list(resnet50(weights=None).children())[:-2]\n            )\n\n        def forward(self, image):\n            return self.backbone(image).mean(dim=(2, 3))\n\n    model = RadImageNetEncoder()\n    state = torch.load(checkpoint, map_location=\"cpu\", weights_only=True)\n    if not state or not all(str(key).startswith(\"backbone.\") for key in state):\n        raise RuntimeError(\"unexpected RadImageNet state-dict namespace\")\n    model.load_state_dict(state, strict=True)\n    parameter_count = sum(parameter.numel() for parameter in model.parameters())\n    if parameter_count != 23_508_032:\n        raise RuntimeError(f\"unexpected RadImageNet parameter count {parameter_count}\")\n    model.eval().to(device)\n    for parameter in model.parameters():\n        parameter.requires_grad_(False)\n    gpu_count = torch.cuda.device_count() if device.type == \"cuda\" else 0\n    if gpu_count > 1:\n        model = nn.DataParallel(model, device_ids=list(range(gpu_count)))\n    log(\n        f\"RadImageNet strict load: {parameter_count:,} params; \"\n        f\"inference GPUs={max(1, gpu_count)}\"\n    )\n    return model\n\n\n@torch.inference_mode()\ndef encode_radimagenet(cache, slot_mask, device):\n    \"\"\"Encode acquired slices with the official [-1, 1] RadImageNet contract.\"\"\"\n    n, slots, slices, h, w = cache.shape\n    features = np.zeros((n, slots * slices, TOKEN_DIM), np.float16)\n    token_mask = np.repeat(slot_mask[:, :, None], slices, axis=2).reshape(n, -1)\n    valid = np.flatnonzero(token_mask.reshape(-1) > 0)\n    flat = cache.reshape(-1, h, w)\n    model = load_radimagenet(device)\n    if device.type == \"cuda\":\n        batch = 192 if torch.cuda.device_count() > 1 else 96\n    else:\n        batch = 8\n    for b0 in range(0, len(valid), batch):\n        ix = valid[b0:b0 + batch]\n        x = torch.from_numpy(flat[ix]).to(device).float().div_(127.5).sub_(1.0)\n        x = x.unsqueeze(1).expand(-1, 3, -1, -1).contiguous()\n        with torch.autocast(\"cuda\", enabled=device.type == \"cuda\"):\n            feat = model(x)\n        if feat.shape[1:] != (TOKEN_DIM,):\n            raise RuntimeError(f\"unexpected RadImageNet feature shape {tuple(feat.shape)}\")\n        features.reshape(-1, TOKEN_DIM)[ix] = (\n            feat.float().cpu().numpy().astype(np.float16)\n        )\n        if b0 % (batch * 100) == 0:\n            log(f\"RadImageNet encoded {b0}/{len(valid)} acquired slices\")\n    del model\n    if device.type == \"cuda\":\n        torch.cuda.empty_cache()\n    return features, token_mask.astype(np.float32)\n\n\nclass FoundationQueryHead(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.project = nn.Sequential(nn.LayerNorm(TOKEN_DIM),\n                                     nn.Linear(TOKEN_DIM, HEAD_DIM), nn.GELU())\n        self.plane = nn.Parameter(torch.randn(N_SLOT, HEAD_DIM) * .01)\n        self.position = nn.Parameter(torch.randn(CACHE_SLICES, HEAD_DIM) * .01)\n        self.query = nn.Parameter(torch.randn(len(TARGETS), HEAD_DIM) * .02)\n        self.attn = nn.MultiheadAttention(HEAD_DIM, 8, dropout=.10, batch_first=True)\n        self.fuse = nn.Sequential(\n            nn.LayerNorm(HEAD_DIM * 4), nn.Linear(HEAD_DIM * 4, HEAD_DIM),\n            nn.GELU(), nn.Dropout(.15),\n        )\n        self.weight = nn.Parameter(torch.randn(len(TARGETS), HEAD_DIM) * .02)\n        self.bias = nn.Parameter(torch.zeros(len(TARGETS)))\n\n    def forward(self, feature, mask):\n        token = self.project(feature.float())\n        token = token.view(len(token), N_SLOT, CACHE_SLICES, HEAD_DIM)\n        token = token + self.plane[None, :, None] + self.position[None, None]\n        token = token.flatten(1, 2)\n        key_padding = mask <= 0\n        # No study should be empty, but keep MHA numerically defined if one is.\n        all_empty = key_padding.all(1)\n        if all_empty.any():\n            key_padding = key_padding.clone()\n            key_padding[all_empty, 0] = False\n        query = self.query.unsqueeze(0).expand(len(token), -1, -1)\n        attended = query + self.attn(query, token, token,\n                                     key_padding_mask=key_padding,\n                                     need_weights=False)[0]\n        denom = mask.sum(1, keepdim=True).clamp_min(1).unsqueeze(-1)\n        mean = (token * mask.unsqueeze(-1)).sum(1, keepdims=True) / denom\n        mean = mean.expand(-1, len(TARGETS), -1)\n        fused = self.fuse(torch.cat(\n            [attended, mean, torch.abs(attended - mean), attended * mean], -1))\n        return (fused * self.weight.unsqueeze(0)).sum(-1) + self.bias\n\n\ndef macro_auc(y, pred):\n    from sklearn.metrics import roc_auc_score\n    hard = (np.asarray(y) >= .5).astype(np.uint8)\n    values = [roc_auc_score(hard[:, j], pred[:, j])\n              for j in range(hard.shape[1]) if np.unique(hard[:, j]).size == 2]\n    return float(np.mean(values))\n\n\ndef _v52_target_auc(y, pred):\n    from sklearn.metrics import roc_auc_score\n    hard = (np.asarray(y) >= .5).astype(np.uint8)\n    return {\n        target: float(roc_auc_score(hard[:, index], pred[:, index]))\n        for index, target in enumerate(TARGETS)\n    }\n\n\n@torch.inference_mode()\ndef predict_head(model, features, masks, indices, device, batch=64):\n    model.eval()\n    pred = []\n    for b0 in range(0, len(indices), batch):\n        ix = indices[b0:b0 + batch]\n        x = torch.from_numpy(features[ix]).to(device)\n        m = torch.from_numpy(masks[ix]).to(device)\n        with torch.autocast(\"cuda\", enabled=device.type == \"cuda\"):\n            pred.append(torch.sigmoid(model(x, m)).float().cpu())\n    return torch.cat(pred).numpy()\n\n\ndef train_fold(features, masks, y, weights, train_idx, val_idx, fold, device):\n    from torch.utils.data import DataLoader, Dataset\n    class Rows(Dataset):\n        def __init__(self, indices): self.indices = np.asarray(indices)\n        def __len__(self): return len(self.indices)\n        def __getitem__(self, k):\n            i = self.indices[k]\n            return features[i], masks[i], y[i], weights[i]\n    model = FoundationQueryHead().to(device)\n    optimizer = torch.optim.AdamW(model.parameters(), lr=2e-4, weight_decay=3e-3)\n    generator = torch.Generator().manual_seed(SEED + 100 + fold)\n    loader = DataLoader(Rows(train_idx), batch_size=48, shuffle=True,\n                        generator=generator, num_workers=2, pin_memory=True,\n                        persistent_workers=True)\n    best, best_auc, stale = None, -1.0, 0\n    for epoch in range(24):\n        model.train()\n        for x, m, target, weight in loader:\n            x, m = x.to(device), m.to(device)\n            target, weight = target.to(device), weight.to(device)\n            with torch.autocast(\"cuda\", enabled=device.type == \"cuda\"):\n                logits = model(x, m)\n                raw = F.binary_cross_entropy_with_logits(logits, target,\n                                                          reduction=\"none\")\n                loss = (raw * weight).sum() / weight.sum().clamp_min(1)\n            optimizer.zero_grad(set_to_none=True)\n            loss.backward()\n            nn.utils.clip_grad_norm_(model.parameters(), 1.0)\n            optimizer.step()\n        pred = predict_head(model, features, masks, val_idx, device)\n        score = macro_auc(y[val_idx], pred)\n        log(f\"fold {fold} epoch {epoch}: grouped weak-val AUC {score:.5f}\")\n        if score > best_auc + 2e-4:\n            best_auc, stale = score, 0\n            best = {k: v.detach().cpu() for k, v in model.state_dict().items()}\n        else:\n            stale += 1\n            if stale >= 5: break\n    return best, best_auc\n\n\ndef _v52_rank_columns(values):\n    frame = pd.DataFrame(np.asarray(values, dtype=np.float64))\n    return frame.rank(method=\"average\", pct=True).to_numpy(np.float64)\n\n\ndef _v52_validate_submission(frame, expected_ids):\n    expected_columns = [\"StudyInstanceUID\", *TARGETS]\n    if frame.columns.tolist() != expected_columns:\n        raise RuntimeError(\"V52 submission schema drift\")\n    ids = frame[\"StudyInstanceUID\"].astype(str).tolist()\n    if ids != list(map(str, expected_ids)) or len(ids) != len(set(ids)):\n        raise RuntimeError(\"V52 submission study identity/order drift\")\n    values = frame[TARGETS].to_numpy(np.float64)\n    if not np.isfinite(values).all() or values.min() < 0 or values.max() > 1:\n        raise RuntimeError(\"V52 submission values are invalid\")\n\n\ndef main_v52():\n    import shutil\n    from sklearn.model_selection import GroupKFold\n\n    output = Path(\"/kaggle/working/rsna_rad_e9\")\n    output.mkdir(parents=True, exist_ok=True)\n    primary = Path(\"/kaggle/working/submission.csv\")\n    preserved = Path(\"/kaggle/working/submission_e2_preserved.csv\")\n    audit_path = Path(\"/kaggle/working/rad_e9_audit.json\")\n    audit = {\n        \"status\": \"E2_PRESERVED\",\n        \"evidence_boundary\": (\n            \"All OOF values are local diagnostics on 58 official image labels; \"\n            \"they are not Kaggle competition scores. E2 remains the primary unless \"\n            \"strict artifact, OOF, inference, and submission gates all pass.\"\n        ),\n        \"encoder\": \"RadImageNet ResNet-50 official PyTorch release\",\n        \"encoder_license\": \"CC-BY-NC-SA-4.0 (Kaggle-hosted weight metadata)\",\n        \"encoder_sha256\": \"08629f7e7bd3e29b8ee9522ca3f65ce4d010a7ddf74f0ea3c7e3f3d0bbab0734\",\n        \"encoder_source_commit\": \"0ce16f7375db4236e646829d1eca61cdb4282133\",\n        \"base_oof_sha256\": \"62d47ba4c0c8347b5b24e7fd2aa517aae0fd6d4656fd5d829fd3a50f0159909c\",\n        \"parent\": \"E2 captured 20-member DINOv2 rank ensemble\",\n        \"blend_contract\": \"rank columns independently, then 80% E2 plus 20% RadImageNet\",\n        \"pixel_rules\": dict(RULES),\n    }\n    if not primary.is_file():\n        raise FileNotFoundError(\"E2 parent submission is absent\")\n    shutil.copy2(primary, preserved)\n\n    try:\n        device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n        if device.type != \"cuda\":\n            raise RuntimeError(\"V52 RadImageNet experiment requires CUDA\")\n        elapsed = max(0.0, time.time() - float(globals().get(\"T0\", time.time())))\n        available = 8.72 * 3600 - elapsed\n        audit[\"elapsed_before_v52_seconds\"] = elapsed\n        audit[\"available_at_start_seconds\"] = available\n        if available < 2.0 * 3600:\n            raise TimeoutError(f\"only {available / 60:.1f} minutes remain\")\n\n        train = pd.read_csv(ROOT / \"train.csv\", dtype={\"StudyInstanceUID\": str})\n        train_series = pd.read_csv(\n            ROOT / \"train_series.csv\",\n            dtype={\"StudyInstanceUID\": str, \"SeriesInstanceUID\": str},\n        )\n        if len(train) != 4407:\n            raise RuntimeError(f\"unexpected train study count {len(train)}\")\n        plane = dict(zip(train_series.SeriesInstanceUID, train_series.Anatomical_Plane))\n        headers = annotate(walk(\"train_series\"))\n        audit_official_sequence_metadata(headers, train_series)\n        studies, pixels, slot_mask = build_cache(\n            pick_slots(headers, plane), plane, lat_of(headers, \"train-v52 \"), \"train-v52\"\n        )\n        by_uid = {str(uid): i for i, uid in enumerate(studies)}\n        missing = [uid for uid in train.StudyInstanceUID if uid not in by_uid]\n        if missing:\n            raise RuntimeError(f\"{len(missing)} train studies absent from cache\")\n        order = np.array([by_uid[uid] for uid in train.StudyInstanceUID], dtype=np.int64)\n        pixels, slot_mask = pixels[order], slot_mask[order]\n        train_token_count = int(np.repeat(slot_mask[:, :, None], CACHE_SLICES, 2).sum())\n        if train_token_count < int(0.90 * len(train) * N_SLOT * CACHE_SLICES):\n            raise RuntimeError(f\"insufficient acquired train slices: {train_token_count}\")\n        features, token_mask = encode_radimagenet(pixels, slot_mask, device)\n        del pixels, slot_mask, headers\n        gc.collect()\n\n        y, weights, gold = make_targets(train)\n        if int(gold.sum()) != 58:\n            raise RuntimeError(f\"expected 58 fully gold studies, observed {int(gold.sum())}\")\n        groups = report_groups(train)\n        if len(np.unique(groups)) < 4000:\n            raise RuntimeError(\"unexpected report-group collapse\")\n\n        splits = list(GroupKFold(5).split(features, groups=groups))\n        fold_id = np.full(len(train), -1, dtype=np.int8)\n        folds = []\n        oof = np.zeros_like(y, dtype=np.float32)\n        for fold, (tr, va) in enumerate(splits):\n            if set(groups[tr]).intersection(groups[va]):\n                raise RuntimeError(f\"report leakage in fold {fold}\")\n            fold_id[va] = fold\n            state, score = train_fold(\n                features, token_mask, y, weights, tr, va, fold, device\n            )\n            if state is None:\n                raise RuntimeError(f\"fold {fold} produced no checkpoint\")\n            head = FoundationQueryHead().to(device)\n            head.load_state_dict(state, strict=True)\n            oof[va] = predict_head(head, features, token_mask, va, device)\n            folds.append({\"fold\": fold, \"weak_auc\": float(score), \"state_dict\": state})\n            del head\n            torch.cuda.empty_cache()\n        if (fold_id < 0).any() or not np.isfinite(oof).all():\n            raise RuntimeError(\"incomplete V52 OOF\")\n\n        weak_auc = macro_auc(y, oof)\n        gold_auc = macro_auc(y[gold], oof[gold])\n        log(f\"V52 RadImageNet OOF weak macro AUC {weak_auc:.5f}\")\n        log(f\"V52 RadImageNet OOF gold macro AUC {gold_auc:.5f} on 58 studies\")\n        torch.save(\n            {\n                \"version\": \"v52-radimagenet-resnet50-official-1\",\n                \"targets\": TARGETS,\n                \"encoder_sha256\": audit[\"encoder_sha256\"],\n                \"encoder_source_commit\": audit[\"encoder_source_commit\"],\n                \"img\": IMG,\n                \"slices_per_plane\": CACHE_SLICES,\n                \"feature\": \"global_average_pool\",\n                \"folds\": folds,\n                \"weak_oof_auc\": weak_auc,\n                \"gold_oof_auc\": gold_auc,\n            },\n            output / \"v52_radimagenet_heads.pt\",\n        )\n        oof_frame = pd.DataFrame(oof, columns=TARGETS)\n        oof_frame.insert(0, \"StudyInstanceUID\", train.StudyInstanceUID)\n        oof_frame[\"fold\"] = fold_id\n        oof_frame[\"is_gold\"] = gold.astype(np.uint8)\n        oof_frame.to_csv(output / \"v52_oof.csv\", index=False)\n\n        base_npz = find_input_file(\"oof.npz\")\n        observed_base_hash = _v52_sha256(base_npz)\n        if observed_base_hash != audit[\"base_oof_sha256\"]:\n            raise RuntimeError(f\"E2 OOF artifact drift: {observed_base_hash}\")\n        with np.load(base_npz, allow_pickle=False) as base_bundle:\n            expected_members = {\"ids\", \"pred\", \"y_derived\", \"gold_mask\", \"targets\"}\n            if set(base_bundle.files) != expected_members:\n                raise RuntimeError(f\"unexpected E2 OOF members: {base_bundle.files}\")\n            base_ids = base_bundle[\"ids\"].astype(str)\n            base_targets = base_bundle[\"targets\"].astype(str).tolist()\n            base_gold = base_bundle[\"gold_mask\"].astype(bool)\n            base_prediction = base_bundle[\"pred\"].astype(np.float64)\n        if base_targets != TARGETS:\n            raise RuntimeError(\"E2 OOF target order drift\")\n        if not np.array_equal(base_ids, train.StudyInstanceUID.astype(str).to_numpy()):\n            raise RuntimeError(\"E2 OOF study order drift\")\n        if not np.array_equal(base_gold, gold):\n            raise RuntimeError(\"E2 OOF gold mask differs from official train.csv\")\n        train_rows = np.flatnonzero(gold)\n        if len(train_rows) != 58:\n            raise RuntimeError(f\"expected 58 E2 gold rows, observed {len(train_rows)}\")\n        gold_y = train.loc[gold, TARGETS].to_numpy(np.float64)\n        exact_public = base_prediction[gold]\n        rad = oof[gold].astype(np.float64)\n        if not all(np.isfinite(x).all() for x in (gold_y, exact_public, rad)):\n            raise RuntimeError(\"non-finite aligned E2/RadImageNet OOF value\")\n\n        base_rank = _v52_rank_columns(exact_public)\n        rad_rank = _v52_rank_columns(rad)\n        base_score = macro_auc(gold_y, base_rank)\n        rad_score = macro_auc(gold_y, rad_rank)\n        alpha_grid = np.array([0.0, 0.025, 0.05, 0.10, 0.15, 0.20, 0.25])\n        gold_folds = fold_id[train_rows]\n        if sorted(np.unique(gold_folds).tolist()) != [0, 1, 2, 3, 4]:\n            raise RuntimeError(\"gold rows do not cover all five grouped folds\")\n        nested = np.zeros_like(base_rank)\n        choices = []\n        outer_train_scores = []\n        for outer in range(5):\n            tr = gold_folds != outer\n            va = ~tr\n            scored = []\n            for alpha in alpha_grid:\n                blend = (1.0 - alpha) * base_rank[tr] + alpha * rad_rank[tr]\n                score = macro_auc(gold_y[tr], blend) - 0.01 * float(alpha)\n                scored.append(float(score))\n            best = max(range(len(alpha_grid)), key=lambda i: (scored[i], -alpha_grid[i]))\n            alpha = float(alpha_grid[best])\n            choices.append(alpha)\n            outer_train_scores.append(scored)\n            nested[va] = (1.0 - alpha) * base_rank[va] + alpha * rad_rank[va]\n        nested_score = macro_auc(gold_y, nested)\n        # Deployment weight is fixed by the independently scored public 0.906 mechanism.\n        # The 58 gold rows may veto it and measure fold stability, but do not tune it.\n        final_alpha = 0.20\n        final_oof = (1.0 - final_alpha) * base_rank + final_alpha * rad_rank\n        final_score = macro_auc(gold_y, final_oof)\n        grid_scores = {\n            f\"{alpha:.3f}\": macro_auc(\n                gold_y, (1.0 - alpha) * base_rank + alpha * rad_rank\n            )\n            for alpha in alpha_grid\n        }\n        base_target_scores = _v52_target_auc(gold_y, base_rank)\n        rad_target_scores = _v52_target_auc(gold_y, rad_rank)\n        final_target_scores = _v52_target_auc(gold_y, final_oof)\n        target_deltas = {\n            target: final_target_scores[target] - base_target_scores[target]\n            for target in TARGETS\n        }\n        target_regressions = {\n            target: delta for target, delta in target_deltas.items() if delta < -1e-12\n        }\n        positive_folds = int(sum(alpha > 0 for alpha in choices))\n        supported = bool(\n            final_alpha > 0\n            and positive_folds >= 3\n            and nested_score >= base_score + 0.001\n            and final_score >= base_score + 0.001\n            and not target_regressions\n        )\n        audit[\"oof\"] = {\n            \"rows\": 58,\n            \"weak_macro_auc\": weak_auc,\n            \"rad_gold_macro_auc\": rad_score,\n            \"e2_macro_auc\": base_score,\n            \"outer_fold_choices\": choices,\n            \"outer_fold_penalized_train_scores\": outer_train_scores,\n            \"nested_blend_macro_auc\": nested_score,\n            \"final_alpha\": final_alpha,\n            \"final_descriptive_macro_auc\": final_score,\n            \"full_grid_macro_auc\": grid_scores,\n            \"positive_outer_folds\": positive_folds,\n            \"per_target\": {\n                target: {\n                    \"e2_auc\": base_target_scores[target],\n                    \"radimagenet_auc\": rad_target_scores[target],\n                    \"blend_auc\": final_target_scores[target],\n                    \"blend_delta\": target_deltas[target],\n                }\n                for target in TARGETS\n            },\n            \"target_regressions\": target_regressions,\n            \"gold_fold_counts\": {\n                str(fold): int((gold_folds == fold).sum()) for fold in range(5)\n            },\n            \"selection_supported\": supported,\n        }\n        audit[\"train_available_slice_tokens\"] = train_token_count\n        audit[\"head_count\"] = len(folds)\n        if not supported:\n            audit[\"status\"] = \"OOF_REJECTED_E2_PRESERVED\"\n            log(\n                f\"V52 rejected by nested OOF: base={base_score:.5f}, \"\n                f\"nested={nested_score:.5f}, final={final_score:.5f}, choices={choices}\"\n            )\n            return\n\n        del features, token_mask\n        gc.collect()\n        test = pd.read_csv(ROOT / \"test.csv\", dtype={\"StudyInstanceUID\": str})\n        test_series = pd.read_csv(\n            ROOT / \"test_series.csv\",\n            dtype={\"StudyInstanceUID\": str, \"SeriesInstanceUID\": str},\n        )\n        test_plane = dict(zip(test_series.SeriesInstanceUID, test_series.Anatomical_Plane))\n        test_headers = annotate(walk(\"test_series\"))\n        audit_official_sequence_metadata(test_headers, test_series)\n        test_studies, test_pixels, test_slot_mask = build_cache(\n            pick_slots(test_headers, test_plane),\n            test_plane,\n            lat_of(test_headers, \"test-v52 \"),\n            \"test-v52\",\n        )\n        test_by_uid = {str(uid): i for i, uid in enumerate(test_studies)}\n        test_missing = [uid for uid in test.StudyInstanceUID if uid not in test_by_uid]\n        if test_missing:\n            raise RuntimeError(f\"{len(test_missing)} test studies absent from cache\")\n        test_order = np.array([test_by_uid[uid] for uid in test.StudyInstanceUID])\n        test_pixels = test_pixels[test_order]\n        test_slot_mask = test_slot_mask[test_order]\n        test_token_count = int(\n            np.repeat(test_slot_mask[:, :, None], CACHE_SLICES, 2).sum()\n        )\n        if test_token_count < int(0.85 * len(test) * N_SLOT * CACHE_SLICES):\n            raise RuntimeError(f\"insufficient acquired test slices: {test_token_count}\")\n        test_features, test_token_mask = encode_radimagenet(\n            test_pixels, test_slot_mask, device\n        )\n        del test_pixels, test_slot_mask, test_headers\n        gc.collect()\n\n        fold_predictions = []\n        all_test = np.arange(len(test), dtype=np.int64)\n        for record in folds:\n            head = FoundationQueryHead().to(device)\n            head.load_state_dict(record[\"state_dict\"], strict=True)\n            fold_predictions.append(\n                predict_head(head, test_features, test_token_mask, all_test, device)\n            )\n            del head\n            torch.cuda.empty_cache()\n        if len(fold_predictions) != 5:\n            raise RuntimeError(\"test inference did not use all five heads\")\n        rad_test = np.mean(np.stack(fold_predictions), axis=0)\n        if not np.isfinite(rad_test).all():\n            raise RuntimeError(\"non-finite RadImageNet test prediction\")\n\n        baseline = pd.read_csv(preserved, dtype={\"StudyInstanceUID\": str})\n        _v52_validate_submission(baseline, test.StudyInstanceUID)\n        rad_frame = pd.DataFrame(rad_test, columns=TARGETS)\n        rad_frame.insert(0, \"StudyInstanceUID\", test.StudyInstanceUID)\n        _v52_validate_submission(rad_frame, test.StudyInstanceUID)\n        rad_frame.to_csv(output / \"submission_rad_only.csv\", index=False)\n        baseline_rank = _v52_rank_columns(baseline[TARGETS].to_numpy())\n        rad_test_rank = _v52_rank_columns(rad_test)\n        selected_path = None\n        for alpha in alpha_grid[1:]:\n            candidate = baseline.copy()\n            candidate[TARGETS] = (\n                (1.0 - alpha) * baseline_rank + alpha * rad_test_rank\n            )\n            _v52_validate_submission(candidate, test.StudyInstanceUID)\n            path = output / f\"submission_e2_rad_{int(round(1000 * alpha)):03d}.csv\"\n            candidate.to_csv(path, index=False)\n            if abs(float(alpha) - final_alpha) < 1e-12:\n                selected_path = path\n        if selected_path is None or not selected_path.is_file():\n            raise RuntimeError(\"selected V52 blend artifact is absent\")\n        selected = pd.read_csv(selected_path, dtype={\"StudyInstanceUID\": str})\n        _v52_validate_submission(selected, test.StudyInstanceUID)\n        audit[\"test_studies\"] = len(test)\n        audit[\"test_available_slice_tokens\"] = test_token_count\n        audit[\"test_head_count\"] = len(fold_predictions)\n        audit[\"selected_path\"] = str(selected_path)\n        audit[\"selected_sha256\"] = _v52_sha256(selected_path)\n        audit[\"fallback_sha256\"] = _v52_sha256(preserved)\n        shutil.copy2(selected_path, primary)\n        if _v52_sha256(primary) != audit[\"selected_sha256\"]:\n            raise RuntimeError(\"primary V52 copy hash mismatch\")\n        audit[\"status\"] = \"CANDIDATE_SELECTED\"\n        log(\n            f\"E9 selected alpha={final_alpha:.3f}; \"\n            f\"nested={nested_score:.5f} vs E2 OOF={base_score:.5f}\"\n        )\n    except Exception as error:\n        audit[\"status\"] = \"ERROR_E2_PRESERVED\"\n        audit[\"error\"] = f\"{type(error).__name__}: {error}\"\n        audit[\"traceback\"] = traceback.format_exc()\n        log(f\"E9 preserves E2: {audit['error']}\")\n    finally:\n        if audit.get(\"status\") != \"CANDIDATE_SELECTED\" and preserved.is_file():\n            shutil.copy2(preserved, primary)\n        audit[\"primary_sha256\"] = _v52_sha256(primary) if primary.is_file() else None\n        audit_path.write_text(json.dumps(audit, indent=2, sort_keys=True) + \"\\n\")\n\n\ndef _v52_load_pinned_e9b():\n    \"\"\"Load the public v15 heads and reconstruct the dual-OOF target gate.\"\"\"\n    heads_path = find_input_file(\"v52_radimagenet_heads.pt\")\n    remote_path = find_input_file(\"rad_e9_audit.json\")\n    public_path = find_input_file(\"public_oof_diagnostic.json\")\n    contract_path = find_input_file(\"e9b_contract.json\")\n    expected_hashes = {\n        heads_path: PINNED_HEADS_SHA256,\n        remote_path: PINNED_REMOTE_AUDIT_SHA256,\n        public_path: PINNED_PUBLIC_DIAGNOSTIC_SHA256,\n        contract_path: PINNED_E9B_CONTRACT_SHA256,\n    }\n    for path, expected in expected_hashes.items():\n        observed = _v52_sha256(path)\n        if observed != expected:\n            raise RuntimeError(f\"pinned E9b artifact drift for {path.name}: {observed}\")\n\n    remote = json.loads(remote_path.read_text())\n    public = json.loads(public_path.read_text())\n    contract = json.loads(contract_path.read_text())\n    if remote.get(\"status\") != \"OOF_REJECTED_E2_PRESERVED\":\n        raise RuntimeError(f\"unexpected v15 audit status {remote.get('status')}\")\n    if remote.get(\"primary_sha256\") != (\n        \"f9fb57b7bac8489a5d5285b3984b06df57f142572be6417eac6341c43e96707a\"\n    ):\n        raise RuntimeError(\"v15 did not preserve the exact E2 visible artifact\")\n    remote_oof = remote.get(\"oof\", {})\n    public_target = public.get(\"per_target\", {})\n    remote_target = remote_oof.get(\"per_target\", {})\n    if set(public_target) != set(TARGETS) or set(remote_target) != set(TARGETS):\n        raise RuntimeError(\"E9b diagnostic target set drift\")\n    if int(remote.get(\"head_count\", -1)) != 5:\n        raise RuntimeError(\"v15 remote audit does not contain five heads\")\n    if int(remote_oof.get(\"positive_outer_folds\", -1)) != 5:\n        raise RuntimeError(\"v15 remote outer-fold support drift\")\n    if int(public.get(\"positive_outer_folds\", -1)) != 5:\n        raise RuntimeError(\"public outer-fold support drift\")\n\n    selected_targets = [\n        target for target in TARGETS\n        if float(public_target[target][\"blend_delta\"]) > 1e-12\n        and float(remote_target[target][\"blend_delta\"]) > 1e-12\n    ]\n    preserved_targets = [target for target in TARGETS if target not in selected_targets]\n    if selected_targets != contract.get(\"selected_targets\"):\n        raise RuntimeError(f\"E9b selected-target contract drift: {selected_targets}\")\n    if preserved_targets != contract.get(\"preserved_targets\"):\n        raise RuntimeError(f\"E9b preserved-target contract drift: {preserved_targets}\")\n    if len(selected_targets) != 10 or preserved_targets != [\"Baker's\", \"Fracture\"]:\n        raise RuntimeError(\"E9b requires the ten-target dual-OOF intersection\")\n    alpha = float(contract.get(\"alpha\", -1))\n    if abs(alpha - 0.20) > 1e-12:\n        raise RuntimeError(f\"E9b alpha drift: {alpha}\")\n\n    def selective_macro(records, base_key, blend_key):\n        return float(np.mean([\n            float(records[target][blend_key] if target in selected_targets\n                  else records[target][base_key])\n            for target in TARGETS\n        ]))\n\n    public_base = float(public[\"base_gold_macro_auc\"])\n    remote_base = float(remote_oof[\"e2_macro_auc\"])\n    public_selective = selective_macro(public_target, \"base_auc\", \"blend_auc\")\n    remote_selective = selective_macro(remote_target, \"e2_auc\", \"blend_auc\")\n    if public_selective < public_base + 0.001:\n        raise RuntimeError(\"E9b public selective gate no longer improves E2\")\n    if remote_selective < remote_base + 0.001:\n        raise RuntimeError(\"E9b remote selective gate no longer improves E2\")\n\n    payload = torch.load(heads_path, map_location=\"cpu\", weights_only=True)\n    expected_payload = {\n        \"version\": \"v52-radimagenet-resnet50-official-1\",\n        \"targets\": TARGETS,\n        \"encoder_sha256\": (\n            \"08629f7e7bd3e29b8ee9522ca3f65ce4d010a7ddf74f0ea3c7e3f3d0bbab0734\"\n        ),\n        \"encoder_source_commit\": \"0ce16f7375db4236e646829d1eca61cdb4282133\",\n        \"img\": 224,\n        \"slices_per_plane\": 8,\n        \"feature\": \"global_average_pool\",\n    }\n    for key, expected in expected_payload.items():\n        if payload.get(key) != expected:\n            raise RuntimeError(f\"pinned E9b head contract drift for {key}\")\n    folds = payload.get(\"folds\")\n    if not isinstance(folds, list) or len(folds) != 5:\n        raise RuntimeError(\"pinned E9b payload requires five folds\")\n    if sorted(int(record.get(\"fold\", -1)) for record in folds) != list(range(5)):\n        raise RuntimeError(\"pinned E9b fold identity drift\")\n    if any(not isinstance(record.get(\"state_dict\"), dict) for record in folds):\n        raise RuntimeError(\"pinned E9b state dictionary is absent\")\n    if abs(float(payload.get(\"weak_oof_auc\", -1)) - 0.8278261335825697) > 1e-12:\n        raise RuntimeError(\"pinned E9b weak OOF drift\")\n    if abs(float(payload.get(\"gold_oof_auc\", -1)) - 0.8543239133509962) > 1e-12:\n        raise RuntimeError(\"pinned E9b gold OOF drift\")\n    return {\n        \"payload\": payload,\n        \"folds\": folds,\n        \"alpha\": alpha,\n        \"selected_targets\": selected_targets,\n        \"preserved_targets\": preserved_targets,\n        \"public_base\": public_base,\n        \"public_selective\": public_selective,\n        \"remote_base\": remote_base,\n        \"remote_selective\": remote_selective,\n        \"remote_outer_fold_choices\": remote_oof[\"outer_fold_choices\"],\n    }\n\n\ndef _v52_e10_remote_ladder(contract):\n    \"\"\"Recompute this account's half of the ladder from artifacts the kernel can read.\n\n    The contract carries per-target gains for two independent RadImageNet OOF runs. Only the\n    public run is unverifiable here, so its numbers stay data. The remote run is rebuilt from\n    the attached OOF table, the pinned E2 OOF bundle and the official labels, and must match\n    the contract exactly or E10 refuses to deploy.\n    \"\"\"\n    train = pd.read_csv(ROOT / \"train.csv\", dtype={\"StudyInstanceUID\": str})\n    gold = train[TARGETS].notna().all(axis=1).to_numpy()\n    oof_path = find_input_file(\"v52_oof.csv\")\n    observed = _v52_sha256(oof_path)\n    if observed != contract[\"remote_oof_sha256\"]:\n        raise RuntimeError(f\"E10 remote OOF drift: {observed}\")\n    rad_frame = pd.read_csv(oof_path, dtype={\"StudyInstanceUID\": str})\n    if rad_frame.columns.tolist() != [\"StudyInstanceUID\", *TARGETS, \"fold\", \"is_gold\"]:\n        raise RuntimeError(\"E10 remote OOF schema drift\")\n    aligned = train[[\"StudyInstanceUID\"]].merge(\n        rad_frame, on=\"StudyInstanceUID\", how=\"left\", validate=\"one_to_one\"\n    )\n    if aligned[TARGETS].isna().any().any():\n        raise RuntimeError(\"E10 remote OOF does not cover every official train study\")\n\n    base_npz = find_input_file(\"oof.npz\")\n    with np.load(base_npz, allow_pickle=False) as bundle:\n        if bundle[\"targets\"].astype(str).tolist() != TARGETS:\n            raise RuntimeError(\"E10 E2 OOF target order drift\")\n        if not np.array_equal(\n            bundle[\"ids\"].astype(str), train.StudyInstanceUID.astype(str).to_numpy()\n        ):\n            raise RuntimeError(\"E10 E2 OOF study order drift\")\n        if not np.array_equal(bundle[\"gold_mask\"].astype(bool), gold):\n            raise RuntimeError(\"E10 E2 gold mask differs from official train.csv\")\n        base_prediction = bundle[\"pred\"].astype(np.float64)\n\n    # Rank within the scored rows, matching both the E9b parent and test-time deployment\n    # where the ranked population and the scored population are the same studies.\n    base = _v52_rank_columns(base_prediction[gold])\n    rad = _v52_rank_columns(aligned[TARGETS].to_numpy(np.float64)[gold])\n    gold_y = train.loc[gold, TARGETS].to_numpy(np.float64)\n    if len(gold_y) != 58 or not np.isfinite(base).all() or not np.isfinite(rad).all():\n        raise RuntimeError(\"E10 gold alignment is incomplete or non-finite\")\n    reference = _v52_target_auc(gold_y, base)\n    if abs(\n        float(np.mean([reference[t] for t in TARGETS]))\n        - float(contract[\"base_gold_macro_auc\"])\n    ) > 1e-9:\n        raise RuntimeError(\"E10 base gold diagnostic drift\")\n\n    rebuilt = {}\n    for key in contract[\"ladder\"]:\n        alpha = float(key)\n        scores = _v52_target_auc(gold_y, (1.0 - alpha) * base + alpha * rad)\n        rebuilt[key] = {t: scores[t] - reference[t] for t in TARGETS}\n    pinned_remote = contract[\"per_target_ladder_delta\"][\"remote_v15\"]\n    if set(rebuilt) != set(pinned_remote):\n        raise RuntimeError(\"E10 ladder key drift\")\n    for key, deltas in rebuilt.items():\n        for target, delta in deltas.items():\n            if abs(delta - float(pinned_remote[key][target])) > 1e-9:\n                raise RuntimeError(\n                    f\"E10 recomputed remote gain disagrees at {key}/{target}: {delta}\"\n                )\n    if any(abs(reference[t] - float(contract[\"per_target_base_auc\"][t])) > 1e-9 for t in TARGETS):\n        raise RuntimeError(\"E10 per-target base AUC drift\")\n    return rebuilt, reference\n\n\ndef _v52_load_e10():\n    \"\"\"Validate the E10 contract, then return the weight map the kernel will deploy.\"\"\"\n    heads_path = find_input_file(\"v52_radimagenet_heads.pt\")\n    remote_path = find_input_file(\"rad_e9_audit.json\")\n    contract_path = find_input_file(\"e10_contract.json\")\n    for path, expected in (\n        (heads_path, PINNED_HEADS_SHA256),\n        (remote_path, PINNED_REMOTE_AUDIT_SHA256),\n        (contract_path, PINNED_E10_CONTRACT_SHA256),\n    ):\n        observed = _v52_sha256(path)\n        if observed != expected:\n            raise RuntimeError(f\"pinned E10 artifact drift for {path.name}: {observed}\")\n\n    contract = json.loads(contract_path.read_text())\n    if contract.get(\"version\") != \"e10-alpha-ladder-2\":\n        raise RuntimeError(f\"unexpected E10 contract version {contract.get('version')}\")\n    if contract.get(\"targets\") != TARGETS:\n        raise RuntimeError(\"E10 contract target order drift\")\n    remote = json.loads(remote_path.read_text())\n    if remote.get(\"status\") != \"OOF_REJECTED_E2_PRESERVED\":\n        raise RuntimeError(f\"unexpected v15 audit status {remote.get('status')}\")\n    if remote.get(\"primary_sha256\") != (\n        \"f9fb57b7bac8489a5d5285b3984b06df57f142572be6417eac6341c43e96707a\"\n    ):\n        raise RuntimeError(\"v15 did not preserve the exact E2 visible artifact\")\n    if int(remote.get(\"head_count\", -1)) != 5:\n        raise RuntimeError(\"v15 remote audit does not contain five heads\")\n\n    rebuilt, base_auc = _v52_e10_remote_ladder(contract)\n    public_ladder = contract[\"per_target_ladder_delta\"][\"public\"]\n    configuration = contract[\"configurations\"].get(E10_CONFIG)\n    if configuration is None:\n        raise RuntimeError(f\"E10 contract has no configuration {E10_CONFIG!r}\")\n    alpha_map = {t: float(configuration[\"alpha_map\"][t]) for t in TARGETS}\n    if any(alpha < 0.0 or alpha > 1.0 for alpha in alpha_map.values()):\n        raise RuntimeError(\"E10 weight outside the unit interval\")\n    preserved = sorted(t for t, alpha in alpha_map.items() if alpha == 0.0)\n    if preserved != sorted(E10_PRESERVED_TARGETS):\n        raise RuntimeError(f\"E10 preserved-target drift: {preserved}\")\n\n    # Two-tier gate. The scored objective is macro AUC, so the binding requirement is that\n    # the deployed map raise the macro in BOTH independent runs -- the public numbers as\n    # pinned data, this account's numbers as recomputed above. A per-target \"never harm any\n    # single label\" rule is strictly stronger than that objective and would veto rungs that\n    # trade a small loss on one label for a large gain on another, so it is enforced only for\n    # configurations that actually claim it. Whichever claim the contract makes is verified;\n    # a configuration cannot quietly assert dual-positivity it no longer has.\n    claims_dual_positive = bool(configuration[\"all_dual_positive\"])\n    observed_dual_positive = True\n    for target, alpha in alpha_map.items():\n        if alpha == 0.0:\n            continue\n        key = f\"{alpha:.2f}\"\n        if key not in rebuilt:\n            raise RuntimeError(f\"E10 weight {key} is outside the audited ladder\")\n        gains = (float(public_ladder[key][target]), float(rebuilt[key][target]))\n        if not all(gain > 0 for gain in gains):\n            observed_dual_positive = False\n            if claims_dual_positive:\n                raise RuntimeError(\n                    f\"E10 dual-source gate rejects {target} at {key}: {gains}\"\n                )\n    if observed_dual_positive != claims_dual_positive:\n        raise RuntimeError(\n            f\"E10 contract claims all_dual_positive={claims_dual_positive} for \"\n            f\"{E10_CONFIG!r} but recomputation observes {observed_dual_positive}\"\n        )\n\n    macro = {}\n    for name, ladder in ((\"public\", public_ladder), (\"remote_v15\", rebuilt)):\n        total = 0.0\n        for target, alpha in alpha_map.items():\n            gain = 0.0 if alpha == 0.0 else float(ladder[f\"{alpha:.2f}\"][target])\n            total += float(base_auc[target]) + gain\n        macro[name] = total / len(TARGETS)\n        if macro[name] <= float(contract[\"base_gold_macro_auc\"]):\n            raise RuntimeError(\n                f\"E10 macro gate rejects {E10_CONFIG!r}: {name} macro {macro[name]} \"\n                f\"does not beat base {contract['base_gold_macro_auc']}\"\n            )\n        if abs(macro[name] - float(configuration[\"descriptive_macro\"][name])) > 1e-9:\n            raise RuntimeError(\n                f\"E10 recomputed {name} macro {macro[name]} disagrees with the contract \"\n                f\"value {configuration['descriptive_macro'][name]}\"\n            )\n\n    payload = torch.load(heads_path, map_location=\"cpu\", weights_only=True)\n    expected_payload = {\n        \"version\": \"v52-radimagenet-resnet50-official-1\",\n        \"targets\": TARGETS,\n        \"encoder_sha256\": (\n            \"08629f7e7bd3e29b8ee9522ca3f65ce4d010a7ddf74f0ea3c7e3f3d0bbab0734\"\n        ),\n        \"encoder_source_commit\": \"0ce16f7375db4236e646829d1eca61cdb4282133\",\n        \"img\": 224,\n        \"slices_per_plane\": 8,\n        \"feature\": \"global_average_pool\",\n    }\n    for key, expected in expected_payload.items():\n        if payload.get(key) != expected:\n            raise RuntimeError(f\"pinned E10 head contract drift for {key}\")\n    folds = payload.get(\"folds\")\n    if not isinstance(folds, list) or len(folds) != 5:\n        raise RuntimeError(\"pinned E10 payload requires five folds\")\n    if sorted(int(record.get(\"fold\", -1)) for record in folds) != list(range(5)):\n        raise RuntimeError(\"pinned E10 fold identity drift\")\n    if any(not isinstance(record.get(\"state_dict\"), dict) for record in folds):\n        raise RuntimeError(\"pinned E10 state dictionary is absent\")\n    if abs(float(payload.get(\"gold_oof_auc\", -1)) - 0.8543239133509962) > 1e-12:\n        raise RuntimeError(\"pinned E10 gold OOF drift\")\n    return {\n        \"payload\": payload,\n        \"folds\": folds,\n        \"contract\": contract,\n        \"configuration\": E10_CONFIG,\n        \"alpha_map\": alpha_map,\n        \"preserved_targets\": sorted(E10_PRESERVED_TARGETS),\n        \"diagnostic_macro\": configuration[\"diagnostic_macro\"],\n        \"recomputed_macro\": macro,\n        \"all_dual_positive\": claims_dual_positive,\n        \"rationale\": configuration[\"rationale\"],\n        \"recomputed_remote_ladder\": rebuilt,\n    }\n\n\ndef main_v52_pinned_e9b():\n    \"\"\"Inference-only E9b from hash-pinned v15 heads; preserve E2 on any failure.\"\"\"\n    import shutil\n\n    output = Path(\"/kaggle/working/rsna_rad_e9b\")\n    output.mkdir(parents=True, exist_ok=True)\n    primary = Path(\"/kaggle/working/submission.csv\")\n    preserved = Path(\"/kaggle/working/submission_e2_preserved.csv\")\n    audit_path = Path(\"/kaggle/working/rad_e9b_audit.json\")\n    audit = {\n        \"status\": \"E2_PRESERVED\",\n        \"mode\": \"pinned_v15_heads_inference_only\",\n        \"evidence_boundary\": (\n            \"OOF values are diagnostics, not Kaggle scores. The fixed public 20-percent \"\n            \"vote is applied only to targets improving in two independent OOF runs.\"\n        ),\n        \"encoder\": \"RadImageNet ResNet-50 official PyTorch release\",\n        \"encoder_license\": \"CC-BY-NC-SA-4.0\",\n        \"encoder_sha256\": (\n            \"08629f7e7bd3e29b8ee9522ca3f65ce4d010a7ddf74f0ea3c7e3f3d0bbab0734\"\n        ),\n        \"heads_sha256\": PINNED_HEADS_SHA256,\n        \"parent\": \"E2 captured 20-member DINOv2 rank ensemble\",\n        \"pixel_rules\": dict(RULES),\n    }\n    if not primary.is_file():\n        raise FileNotFoundError(\"E2 parent submission is absent\")\n    shutil.copy2(primary, preserved)\n\n    try:\n        device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n        if device.type != \"cuda\":\n            raise RuntimeError(\"E9b RadImageNet inference requires CUDA\")\n        elapsed = max(0.0, time.time() - float(globals().get(\"T0\", time.time())))\n        available = 8.72 * 3600 - elapsed\n        audit[\"elapsed_before_e9b_seconds\"] = elapsed\n        audit[\"available_at_start_seconds\"] = available\n        if available < 45 * 60:\n            raise TimeoutError(f\"only {available / 60:.1f} minutes remain\")\n\n        pinned = _v52_load_pinned_e9b()\n        audit[\"oof_gate\"] = {\n            \"alpha\": pinned[\"alpha\"],\n            \"selected_targets\": pinned[\"selected_targets\"],\n            \"preserved_targets\": pinned[\"preserved_targets\"],\n            \"public_e2_macro_auc\": pinned[\"public_base\"],\n            \"public_selective_macro_auc\": pinned[\"public_selective\"],\n            \"remote_e2_macro_auc\": pinned[\"remote_base\"],\n            \"remote_selective_macro_auc\": pinned[\"remote_selective\"],\n            \"remote_outer_fold_choices\": pinned[\"remote_outer_fold_choices\"],\n            \"selection_supported\": True,\n        }\n\n        test = pd.read_csv(ROOT / \"test.csv\", dtype={\"StudyInstanceUID\": str})\n        test_series = pd.read_csv(\n            ROOT / \"test_series.csv\",\n            dtype={\"StudyInstanceUID\": str, \"SeriesInstanceUID\": str},\n        )\n        plane = dict(zip(test_series.SeriesInstanceUID, test_series.Anatomical_Plane))\n        headers = annotate(walk(\"test_series\"))\n        audit_official_sequence_metadata(headers, test_series)\n        studies, pixels, slot_mask = build_cache(\n            pick_slots(headers, plane), plane, lat_of(headers, \"test-e9b \"), \"test-e9b\"\n        )\n        by_uid = {str(uid): index for index, uid in enumerate(studies)}\n        missing = [uid for uid in test.StudyInstanceUID if uid not in by_uid]\n        if missing:\n            raise RuntimeError(f\"{len(missing)} test studies absent from E9b cache\")\n        order = np.asarray([by_uid[uid] for uid in test.StudyInstanceUID], dtype=np.int64)\n        pixels, slot_mask = pixels[order], slot_mask[order]\n        token_count = int(np.repeat(slot_mask[:, :, None], CACHE_SLICES, 2).sum())\n        if token_count < int(0.85 * len(test) * N_SLOT * CACHE_SLICES):\n            raise RuntimeError(f\"insufficient acquired E9b test slices: {token_count}\")\n        features, token_mask = encode_radimagenet(pixels, slot_mask, device)\n        del pixels, slot_mask, headers\n        gc.collect()\n\n        all_test = np.arange(len(test), dtype=np.int64)\n        fold_predictions = []\n        for record in pinned[\"folds\"]:\n            head = FoundationQueryHead().to(device)\n            head.load_state_dict(record[\"state_dict\"], strict=True)\n            fold_predictions.append(\n                predict_head(head, features, token_mask, all_test, device)\n            )\n            del head\n            torch.cuda.empty_cache()\n        if len(fold_predictions) != 5:\n            raise RuntimeError(\"E9b test inference did not use all five heads\")\n        rad_test = np.mean(np.stack(fold_predictions), axis=0)\n        if rad_test.shape != (len(test), len(TARGETS)) or not np.isfinite(rad_test).all():\n            raise RuntimeError(f\"invalid E9b prediction shape/value: {rad_test.shape}\")\n\n        baseline = pd.read_csv(preserved, dtype={\"StudyInstanceUID\": str})\n        _v52_validate_submission(baseline, test.StudyInstanceUID)\n        rad_frame = pd.DataFrame(rad_test, columns=TARGETS)\n        rad_frame.insert(0, \"StudyInstanceUID\", test.StudyInstanceUID)\n        _v52_validate_submission(rad_frame, test.StudyInstanceUID)\n        rad_frame.to_csv(output / \"submission_rad_only.csv\", index=False)\n        baseline_rank = _v52_rank_columns(baseline[TARGETS].to_numpy())\n        rad_rank = _v52_rank_columns(rad_test)\n        selected = baseline.copy()\n        alpha = pinned[\"alpha\"]\n        for target in pinned[\"selected_targets\"]:\n            index = TARGETS.index(target)\n            selected[target] = (\n                (1.0 - alpha) * baseline_rank[:, index] + alpha * rad_rank[:, index]\n            )\n        for target in pinned[\"preserved_targets\"]:\n            if not np.array_equal(\n                selected[target].to_numpy(), baseline[target].to_numpy()\n            ):\n                raise RuntimeError(f\"E9b failed to preserve {target}\")\n        _v52_validate_submission(selected, test.StudyInstanceUID)\n        selected_path = output / \"submission_e2_rad_robust_200.csv\"\n        selected.to_csv(selected_path, index=False)\n        selected = pd.read_csv(selected_path, dtype={\"StudyInstanceUID\": str})\n        _v52_validate_submission(selected, test.StudyInstanceUID)\n\n        audit.update({\n            \"test_studies\": len(test),\n            \"test_available_slice_tokens\": token_count,\n            \"test_head_count\": len(fold_predictions),\n            \"selected_path\": str(selected_path),\n            \"selected_sha256\": _v52_sha256(selected_path),\n            \"fallback_sha256\": _v52_sha256(preserved),\n        })\n        shutil.copy2(selected_path, primary)\n        if _v52_sha256(primary) != audit[\"selected_sha256\"]:\n            raise RuntimeError(\"primary E9b copy hash mismatch\")\n        audit[\"status\"] = \"CANDIDATE_SELECTED\"\n        log(\n            f\"E9b selected alpha={alpha:.3f} on \"\n            f\"{len(pinned['selected_targets'])} dual-OOF-stable targets; \"\n            \"Baker's and Fracture preserve E2\"\n        )\n    except Exception as error:\n        audit[\"status\"] = \"ERROR_E2_PRESERVED\"\n        audit[\"error\"] = f\"{type(error).__name__}: {error}\"\n        audit[\"traceback\"] = traceback.format_exc()\n        log(f\"E9b preserves E2: {audit['error']}\")\n    finally:\n        if audit.get(\"status\") != \"CANDIDATE_SELECTED\" and preserved.is_file():\n            shutil.copy2(preserved, primary)\n        audit[\"primary_sha256\"] = _v52_sha256(primary) if primary.is_file() else None\n        audit_path.write_text(json.dumps(audit, indent=2, sort_keys=True) + \"\\n\")\n\n\ndef _v52_rad_test_predictions(pinned, test, device, tag):\n    \"\"\"Five-head RadImageNet test prediction on the official test tree.\"\"\"\n    test_series = pd.read_csv(\n        ROOT / \"test_series.csv\",\n        dtype={\"StudyInstanceUID\": str, \"SeriesInstanceUID\": str},\n    )\n    plane = dict(zip(test_series.SeriesInstanceUID, test_series.Anatomical_Plane))\n    headers = annotate(walk(\"test_series\"))\n    audit_official_sequence_metadata(headers, test_series)\n    studies, pixels, slot_mask = build_cache(\n        pick_slots(headers, plane), plane, lat_of(headers, f\"{tag} \"), tag\n    )\n    by_uid = {str(uid): index for index, uid in enumerate(studies)}\n    missing = [uid for uid in test.StudyInstanceUID if uid not in by_uid]\n    if missing:\n        raise RuntimeError(f\"{len(missing)} test studies absent from {tag} cache\")\n    order = np.asarray([by_uid[uid] for uid in test.StudyInstanceUID], dtype=np.int64)\n    pixels, slot_mask = pixels[order], slot_mask[order]\n    token_count = int(np.repeat(slot_mask[:, :, None], CACHE_SLICES, 2).sum())\n    if token_count < int(0.85 * len(test) * N_SLOT * CACHE_SLICES):\n        raise RuntimeError(f\"insufficient acquired {tag} test slices: {token_count}\")\n    features, token_mask = encode_radimagenet(pixels, slot_mask, device)\n    del pixels, slot_mask, headers\n    gc.collect()\n\n    rows = np.arange(len(test), dtype=np.int64)\n    predictions = []\n    for record in pinned[\"folds\"]:\n        head = FoundationQueryHead().to(device)\n        head.load_state_dict(record[\"state_dict\"], strict=True)\n        predictions.append(predict_head(head, features, token_mask, rows, device))\n        del head\n        torch.cuda.empty_cache()\n    if len(predictions) != 5:\n        raise RuntimeError(f\"{tag} test inference did not use all five heads\")\n    rad_test = np.mean(np.stack(predictions), axis=0)\n    if rad_test.shape != (len(test), len(TARGETS)) or not np.isfinite(rad_test).all():\n        raise RuntimeError(f\"invalid {tag} prediction shape/value: {rad_test.shape}\")\n    return rad_test, token_count, len(predictions)\n\n\ndef main_v52_e10():\n    \"\"\"Deploy the audited E10 weight map; preserve the E2 parent on any failure.\"\"\"\n    import shutil\n\n    output = Path(\"/kaggle/working/rsna_rad_e10\")\n    output.mkdir(parents=True, exist_ok=True)\n    primary = Path(\"/kaggle/working/submission.csv\")\n    preserved = Path(\"/kaggle/working/submission_e2_preserved.csv\")\n    audit_path = Path(\"/kaggle/working/rad_e10_audit.json\")\n    audit = {\n        \"status\": \"E2_PRESERVED\",\n        \"mode\": \"pinned_v15_heads_inference_only\",\n        \"experiment\": \"E10\",\n        \"configuration\": E10_CONFIG,\n        \"evidence_boundary\": (\n            \"OOF values are 58-study diagnostics on official train labels, not Kaggle \"\n            \"scores. E10 widens the weight ladder that E9 truncated at 0.25 and votes only \"\n            \"where two independent OOF runs agree at the deployed weight.\"\n        ),\n        \"encoder\": \"RadImageNet ResNet-50 official PyTorch release\",\n        \"encoder_license\": \"CC-BY-NC-SA-4.0\",\n        \"heads_sha256\": PINNED_HEADS_SHA256,\n        \"contract_sha256\": PINNED_E10_CONTRACT_SHA256,\n        \"parent\": \"E2 captured 20-member DINOv2 rank ensemble\",\n        \"pixel_rules\": dict(RULES),\n    }\n    if not primary.is_file():\n        raise FileNotFoundError(\"E2 parent submission is absent\")\n    shutil.copy2(primary, preserved)\n\n    try:\n        device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n        if device.type != \"cuda\":\n            raise RuntimeError(\"E10 RadImageNet inference requires CUDA\")\n        elapsed = max(0.0, time.time() - float(globals().get(\"T0\", time.time())))\n        available = 8.72 * 3600 - elapsed\n        audit[\"elapsed_before_e10_seconds\"] = elapsed\n        audit[\"available_at_start_seconds\"] = available\n        if available < 45 * 60:\n            raise TimeoutError(f\"only {available / 60:.1f} minutes remain\")\n\n        pinned = _v52_load_e10()\n        audit[\"weight_gate\"] = {\n            \"configuration\": pinned[\"configuration\"],\n            \"alpha_map\": pinned[\"alpha_map\"],\n            \"preserved_targets\": pinned[\"preserved_targets\"],\n            \"diagnostic_macro\": pinned[\"diagnostic_macro\"],\n            \"recomputed_macro\": pinned[\"recomputed_macro\"],\n            \"all_dual_positive\": pinned[\"all_dual_positive\"],\n            \"base_gold_macro_auc\": pinned[\"contract\"][\"base_gold_macro_auc\"],\n            \"rationale\": pinned[\"rationale\"],\n            \"remote_ladder_recomputed_in_kernel\": True,\n        }\n\n        test = pd.read_csv(ROOT / \"test.csv\", dtype={\"StudyInstanceUID\": str})\n        rad_test, token_count, head_count = _v52_rad_test_predictions(\n            pinned, test, device, \"test-e10\"\n        )\n\n        baseline = pd.read_csv(preserved, dtype={\"StudyInstanceUID\": str})\n        _v52_validate_submission(baseline, test.StudyInstanceUID)\n        rad_frame = pd.DataFrame(rad_test, columns=TARGETS)\n        rad_frame.insert(0, \"StudyInstanceUID\", test.StudyInstanceUID)\n        _v52_validate_submission(rad_frame, test.StudyInstanceUID)\n        rad_frame.to_csv(output / \"submission_rad_only.csv\", index=False)\n        baseline_rank = _v52_rank_columns(baseline[TARGETS].to_numpy())\n        rad_rank = _v52_rank_columns(rad_test)\n\n        # Materialise every audited rung so the ladder is inspectable from one run; only the\n        # configured map is promoted to the visible submission.\n        written = {}\n        for name, configuration in sorted(pinned[\"contract\"][\"configurations\"].items()):\n            frame = baseline.copy()\n            for target, alpha in configuration[\"alpha_map\"].items():\n                alpha = float(alpha)\n                if alpha > 0:\n                    index = TARGETS.index(target)\n                    frame[target] = (\n                        (1.0 - alpha) * baseline_rank[:, index] + alpha * rad_rank[:, index]\n                    )\n            for target, alpha in configuration[\"alpha_map\"].items():\n                if float(alpha) == 0.0 and not np.array_equal(\n                    frame[target].to_numpy(), baseline[target].to_numpy()\n                ):\n                    raise RuntimeError(f\"E10 failed to preserve {target} in {name}\")\n            _v52_validate_submission(frame, test.StudyInstanceUID)\n            path = output / f\"submission_e10_{name}.csv\"\n            frame.to_csv(path, index=False)\n            written[name] = _v52_sha256(path)\n        audit[\"ladder_sha256\"] = written\n\n        selected_path = output / f\"submission_e10_{pinned['configuration']}.csv\"\n        selected = pd.read_csv(selected_path, dtype={\"StudyInstanceUID\": str})\n        _v52_validate_submission(selected, test.StudyInstanceUID)\n        for target in pinned[\"preserved_targets\"]:\n            if not np.array_equal(\n                selected[target].to_numpy(), baseline[target].to_numpy()\n            ):\n                raise RuntimeError(f\"E10 promoted file does not preserve {target}\")\n        audit.update({\n            \"test_studies\": len(test),\n            \"test_available_slice_tokens\": token_count,\n            \"test_head_count\": head_count,\n            \"selected_path\": str(selected_path),\n            \"selected_sha256\": _v52_sha256(selected_path),\n            \"fallback_sha256\": _v52_sha256(preserved),\n        })\n        shutil.copy2(selected_path, primary)\n        if _v52_sha256(primary) != audit[\"selected_sha256\"]:\n            raise RuntimeError(\"primary E10 copy hash mismatch\")\n        audit[\"status\"] = \"CANDIDATE_SELECTED\"\n        voted = sorted(t for t, alpha in pinned[\"alpha_map\"].items() if alpha > 0)\n        log(\n            f\"E10 promoted {pinned['configuration']} over {len(voted)} dual-OOF targets; \"\n            f\"{', '.join(pinned['preserved_targets'])} preserve E2\"\n        )\n    except Exception as error:\n        audit[\"status\"] = \"ERROR_E2_PRESERVED\"\n        audit[\"error\"] = f\"{type(error).__name__}: {error}\"\n        audit[\"traceback\"] = traceback.format_exc()\n        log(f\"E10 preserves E2: {audit['error']}\")\n    finally:\n        if audit.get(\"status\") != \"CANDIDATE_SELECTED\" and preserved.is_file():\n            shutil.copy2(preserved, primary)\n        audit[\"primary_sha256\"] = _v52_sha256(primary) if primary.is_file() else None\n        audit_path.write_text(json.dumps(audit, indent=2, sort_keys=True) + \"\\n\")\n\n\ndef _v52_e11_availability(headers, plane_map):\n    \"\"\"Count studies offering each (plane, fat-suppression) pair before any slot is picked.\n\n    The parent arm reads only fat-suppressed series and never had to ask how many studies\n    carry a non-suppressed one. E11 depends on that answer, so it is measured and logged\n    rather than assumed: a slot nobody can fill is a masked column, and a run that produced\n    one silently would look like a weak arm instead of an absent input.\n    \"\"\"\n    frame = headers[[\"StudyInstanceUID\", \"SeriesInstanceUID\", \"fatsat\"]].copy()\n    frame[\"plane\"] = frame.SeriesInstanceUID.map(plane_map)\n    total = frame.StudyInstanceUID.nunique()\n    table = {}\n    for plane in (\"Sagittal\", \"Coronal\", \"Axial\"):\n        for fatsat in (True, False):\n            selected = frame[(frame.plane == plane) & (frame.fatsat == bool(fatsat))]\n            studies = selected.StudyInstanceUID.nunique()\n            key = f\"{plane}_{'FS' if fatsat else 'NOFS'}\"\n            table[key] = {\n                \"studies\": int(studies),\n                \"fraction\": float(studies / total) if total else 0.0,\n                \"series\": int(len(selected)),\n            }\n            log(f\"E11 availability {key}: {studies}/{total} studies, {len(selected)} series\")\n    return table\n\n\ndef main_v52_e11():\n    \"\"\"Train a third arm on a deliberately different pixel recipe. Never ships a candidate.\"\"\"\n    import shutil\n    from sklearn.model_selection import GroupKFold\n\n    output = Path(\"/kaggle/working/rsna_rad_e11\")\n    output.mkdir(parents=True, exist_ok=True)\n    primary = Path(\"/kaggle/working/submission.csv\")\n    preserved = Path(\"/kaggle/working/submission_e2_preserved.csv\")\n    audit_path = Path(\"/kaggle/working/rad_e11_audit.json\")\n    audit = {\n        \"status\": \"E2_PRESERVED\",\n        \"mode\": \"e11-diverse-recipe-training-only\",\n        \"evidence_boundary\": (\n            \"Every value here is a local out-of-fold diagnostic on 58 official image \"\n            \"labels. It is not a Kaggle score, and this mode never replaces the parent \"\n            \"submission under any outcome.\"\n        ),\n        \"recipe\": {\n            \"slots\": [list(slot) for slot in E11_SLOTS],\n            \"crop_mm\": E11_CROP_MM,\n            \"cache_slices\": E11_CACHE_SLICES,\n            \"img\": E11_IMG,\n            \"differs_from_parent_arm\": (\n                \"parent reads 3 fat-suppressed slots at full frame; this reads 3 \"\n                \"non-suppressed slots plus 1 suppressed anchor at a 130 mm physical crop\"\n            ),\n        },\n        \"encoder\": \"RadImageNet ResNet-50 official PyTorch release\",\n        \"encoder_license\": \"CC-BY-NC-SA-4.0 (Kaggle-hosted weight metadata)\",\n        \"encoder_sha256\": \"08629f7e7bd3e29b8ee9522ca3f65ce4d010a7ddf74f0ea3c7e3f3d0bbab0734\",\n    }\n    if not primary.is_file():\n        raise FileNotFoundError(\"E2 parent submission is absent\")\n    shutil.copy2(primary, preserved)\n\n    try:\n        device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n        if device.type != \"cuda\":\n            raise RuntimeError(\"E11 training requires CUDA\")\n        elapsed = max(0.0, time.time() - float(globals().get(\"T0\", time.time())))\n        available = TIME_BUDGET - elapsed\n        audit[\"elapsed_before_e11_seconds\"] = elapsed\n        audit[\"available_at_start_seconds\"] = available\n        if available < 2.5 * 3600:\n            raise TimeoutError(f\"only {available / 60:.1f} minutes remain\")\n\n        # Point the shared pixel path at the E11 recipe. These are the same process globals\n        # the parent notebook's readers consult, so the override has to happen before any\n        # slot is picked or any pixel is decoded, and nothing after this point may assume\n        # the parent arm's values.\n        globals().update(\n            SLOTS=list(E11_SLOTS),\n            N_SLOT=len(E11_SLOTS),\n            CACHE_SLICES=int(E11_CACHE_SLICES),\n            IMG=int(E11_IMG),\n            CACHE_IMG=int(E11_IMG),\n            CROP_MM=float(E11_CROP_MM),\n        )\n        log(\n            f\"E11 recipe: {[s[0] for s in E11_SLOTS]} at {E11_CROP_MM:.0f} mm, \"\n            f\"{E11_IMG} px, {E11_CACHE_SLICES} slices/slot\"\n        )\n\n        train = pd.read_csv(ROOT / \"train.csv\", dtype={\"StudyInstanceUID\": str})\n        train_series = pd.read_csv(\n            ROOT / \"train_series.csv\",\n            dtype={\"StudyInstanceUID\": str, \"SeriesInstanceUID\": str},\n        )\n        if len(train) != 4407:\n            raise RuntimeError(f\"unexpected train study count {len(train)}\")\n        plane = dict(zip(train_series.SeriesInstanceUID, train_series.Anatomical_Plane))\n        headers = annotate(walk(\"train_series\"))\n        audit[\"availability\"] = _v52_e11_availability(headers, plane)\n\n        studies, pixels, slot_mask = build_cache(\n            pick_slots(headers, plane), plane, lat_of(headers, \"train-e11 \"), \"train-e11\"\n        )\n        by_uid = {str(uid): i for i, uid in enumerate(studies)}\n        missing = [uid for uid in train.StudyInstanceUID if uid not in by_uid]\n        if missing:\n            raise RuntimeError(f\"{len(missing)} train studies absent from cache\")\n        order = np.array([by_uid[uid] for uid in train.StudyInstanceUID], dtype=np.int64)\n        pixels, slot_mask = pixels[order], slot_mask[order]\n        fill = float(slot_mask.mean())\n        per_slot = {\n            name: float(slot_mask[:, k].mean())\n            for k, (name, _, _, _) in enumerate(E11_SLOTS)\n        }\n        audit[\"slot_fill\"] = per_slot\n        audit[\"overall_fill\"] = fill\n        for name, value in per_slot.items():\n            log(f\"E11 slot fill {name}: {value:.1%}\")\n        if fill < E11_MIN_FILL:\n            raise RuntimeError(f\"E11 slot fill {fill:.1%} below the {E11_MIN_FILL:.0%} floor\")\n\n        features, token_mask = encode_radimagenet(pixels, slot_mask, device)\n        del pixels, slot_mask, headers\n        gc.collect()\n\n        y, weights, gold = make_targets(train)\n        if int(gold.sum()) != 58:\n            raise RuntimeError(f\"expected 58 fully gold studies, observed {int(gold.sum())}\")\n        groups = report_groups(train)\n        if len(np.unique(groups)) < 4000:\n            raise RuntimeError(\"unexpected report-group collapse\")\n\n        splits = list(GroupKFold(5).split(features, groups=groups))\n        fold_id = np.full(len(train), -1, dtype=np.int8)\n        folds = []\n        oof = np.zeros_like(y, dtype=np.float32)\n        for fold, (tr, va) in enumerate(splits):\n            if set(groups[tr]).intersection(groups[va]):\n                raise RuntimeError(f\"report leakage in fold {fold}\")\n            fold_id[va] = fold\n            state, score = train_fold(\n                features, token_mask, y, weights, tr, va, fold, device\n            )\n            if state is None:\n                raise RuntimeError(f\"fold {fold} produced no checkpoint\")\n            head = FoundationQueryHead().to(device)\n            head.load_state_dict(state, strict=True)\n            oof[va] = predict_head(head, features, token_mask, va, device)\n            folds.append({\"fold\": fold, \"weak_auc\": float(score), \"state_dict\": state})\n            del head\n            torch.cuda.empty_cache()\n        if (fold_id < 0).any() or not np.isfinite(oof).all():\n            raise RuntimeError(\"incomplete E11 OOF\")\n\n        weak_auc = macro_auc(y, oof)\n        gold_auc = macro_auc(y[gold], oof[gold])\n        audit[\"weak_oof_auc\"] = float(weak_auc)\n        audit[\"gold_oof_auc\"] = float(gold_auc)\n        log(f\"E11 OOF weak macro AUC {weak_auc:.5f}\")\n        log(f\"E11 OOF gold macro AUC {gold_auc:.5f} on 58 studies\")\n\n        # The question E11 exists to answer is not whether this arm is strong on its own but\n        # whether it says something the portfolio does not already know. Both halves are\n        # measured against the same 58 rows and the same rank basis the deployed blend uses.\n        base_npz = find_input_file(\"oof.npz\")\n        with np.load(base_npz, allow_pickle=False) as bundle:\n            if bundle[\"targets\"].astype(str).tolist() != TARGETS:\n                raise RuntimeError(\"E11 E2 OOF target order drift\")\n            if not np.array_equal(\n                bundle[\"ids\"].astype(str), train.StudyInstanceUID.astype(str).to_numpy()\n            ):\n                raise RuntimeError(\"E11 E2 OOF study order drift\")\n            base_prediction = bundle[\"pred\"].astype(np.float64)\n        base = _v52_rank_columns(base_prediction[gold])\n        new = _v52_rank_columns(oof[gold].astype(np.float64))\n        gold_y = train.loc[gold, TARGETS].to_numpy(np.float64)\n        reference = _v52_target_auc(gold_y, base)\n        audit[\"e2_base_gold_macro\"] = float(np.mean([reference[t] for t in TARGETS]))\n        ladder = {}\n        for alpha in (0.20, 0.35, 0.50):\n            scores = _v52_target_auc(gold_y, (1.0 - alpha) * base + alpha * new)\n            ladder[f\"{alpha:.2f}\"] = {\n                \"macro\": float(np.mean([scores[t] for t in TARGETS])),\n                \"per_target_delta\": {t: float(scores[t] - reference[t]) for t in TARGETS},\n            }\n            log(f\"E11 blend alpha={alpha:.2f} gold macro {ladder[f'{alpha:.2f}']['macro']:.5f}\")\n        audit[\"blend_vs_e2\"] = ladder\n\n        try:\n            parent_oof = pd.read_csv(\n                find_input_file(\"v52_oof.csv\"), dtype={\"StudyInstanceUID\": str}\n            )\n            aligned = train[[\"StudyInstanceUID\"]].merge(\n                parent_oof, on=\"StudyInstanceUID\", how=\"left\", validate=\"one_to_one\"\n            )\n            parent = _v52_rank_columns(aligned[TARGETS].to_numpy(np.float64)[gold])\n            audit[\"correlation_with_parent_arm\"] = {\n                t: float(np.corrcoef(parent[:, i], new[:, i])[0, 1])\n                for i, t in enumerate(TARGETS)\n            }\n            log(\n                \"E11 mean rank correlation with the parent arm: \"\n                f\"{np.mean(list(audit['correlation_with_parent_arm'].values())):.3f}\"\n            )\n        except FileNotFoundError:\n            audit[\"correlation_with_parent_arm\"] = None\n\n        torch.save(\n            {\n                \"version\": \"e11-radimagenet-resnet50-diverse-1\",\n                \"targets\": TARGETS,\n                \"encoder_sha256\": audit[\"encoder_sha256\"],\n                \"slots\": [list(slot) for slot in E11_SLOTS],\n                \"crop_mm\": E11_CROP_MM,\n                \"img\": E11_IMG,\n                \"slices_per_plane\": E11_CACHE_SLICES,\n                \"feature\": \"global_average_pool\",\n                \"folds\": folds,\n                \"weak_oof_auc\": float(weak_auc),\n                \"gold_oof_auc\": float(gold_auc),\n            },\n            output / \"v52_e11_heads.pt\",\n        )\n        oof_frame = pd.DataFrame(oof, columns=TARGETS)\n        oof_frame.insert(0, \"StudyInstanceUID\", train.StudyInstanceUID)\n        oof_frame[\"fold\"] = fold_id\n        oof_frame[\"is_gold\"] = gold.astype(np.uint8)\n        oof_frame.to_csv(output / \"v52_e11_oof.csv\", index=False)\n        audit[\"status\"] = \"E11_TRAINED_E2_PRESERVED\"\n        audit[\"heads_sha256\"] = _v52_sha256(output / \"v52_e11_heads.pt\")\n        audit[\"oof_sha256\"] = _v52_sha256(output / \"v52_e11_oof.csv\")\n    except Exception as error:\n        audit[\"status\"] = \"ERROR_E2_PRESERVED\"\n        audit[\"error\"] = f\"{type(error).__name__}: {error}\"\n        audit[\"traceback\"] = traceback.format_exc()\n        log(f\"E11 preserves E2: {audit['error']}\")\n    finally:\n        if preserved.is_file():\n            shutil.copy2(preserved, primary)\n        audit[\"primary_sha256\"] = _v52_sha256(primary) if primary.is_file() else None\n        audit_path.write_text(json.dumps(audit, indent=2, sort_keys=True) + \"\\n\")\n\n\n\n# E15: swap the frozen encoder family -- RadImageNet ResNet-50 -> DINOv2-base (already\n# mounted for the parent). Same non-FS 130 mm recipe that made E11 the one arm to pay;\n# what changes is the feature space. Placed immediately before the dispatch so it\n# overrides the notebook's own definitions rather than being overwritten by them.\nTOKEN_DIM = 768\n\ndef load_radimagenet(device):\n    \"\"\"DINOv2-base as a frozen slice encoder, returning the CLS embedding.\"\"\"\n    from transformers import AutoModel\n    path = _rt_find_dino_base()\n    if path is None:\n        raise FileNotFoundError('DINOv2-base not mounted')\n    core = AutoModel.from_pretrained(str(path)).to(device).eval()\n    for p in core.parameters():\n        p.requires_grad_(False)\n\n    class DinoSliceEncoder(nn.Module):\n        def __init__(self, core):\n            super().__init__()\n            self.core = core\n            self.register_buffer('mu', torch.tensor([0.485, 0.456, 0.406]).view(1, 3, 1, 1))\n            self.register_buffer('sd', torch.tensor([0.229, 0.224, 0.225]).view(1, 3, 1, 1))\n\n        def forward(self, x):\n            x = ((x + 1.0) / 2.0 - self.mu) / self.sd\n            return self.core(pixel_values=x).last_hidden_state[:, 0]\n\n    return DinoSliceEncoder(core).to(device).eval()\n\nif ARM_MODE == \"e11\":\n    main_v52_e11()\nelse:\n    try:\n        find_input_file(\"v52_radimagenet_heads.pt\")\n    except FileNotFoundError:\n        main_v52()\n    else:\n        try:\n            find_input_file(\"e10_contract.json\")\n        except FileNotFoundError:\n            main_v52_pinned_e9b()\n        else:\n            main_v52_e10()\n\n","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"code","execution_count":null,"id":"v40-runtime-audit","metadata":{},"outputs":[],"source":"if ARM_MODE == 'e11':\n    print('e11 training mode: V40 submission receipt skipped; artifacts are rad_e11_audit.json + rsna_rad_e11/*')\nelse:\n    # Fail-closed V40 runtime receipt.\n    import hashlib as _v40_hashlib\n    import json as _v40_json\n    from pathlib import Path as _V40Path\n    import numpy as _v40_np\n    import pandas as _v40_pd\n\n    _v40_work = _V40Path('/kaggle/working')\n    _v40_primary = _v40_work / 'submission.csv'\n    _v40_parent = _v40_work / 'submission_e2_preserved.csv'\n    _v40_e10_audit = _v40_work / 'rad_e10_audit.json'\n    _v40_test = _v40_pd.read_csv(COMP / 'test.csv', dtype={'StudyInstanceUID': str})\n    _v40_sub = _v40_pd.read_csv(_v40_primary, dtype={'StudyInstanceUID': str})\n    _v40_expected = ['StudyInstanceUID', *TARGETS]\n    if _v40_sub.columns.tolist() != _v40_expected:\n        raise RuntimeError('V40 submission schema drift')\n    if _v40_sub.StudyInstanceUID.tolist() != _v40_test.StudyInstanceUID.astype(str).tolist():\n        raise RuntimeError('V40 dynamic test identity/order drift')\n    _v40_values = _v40_sub[TARGETS].to_numpy(float)\n    if not _v40_np.isfinite(_v40_values).all() or _v40_values.min() < 0 or _v40_values.max() > 1:\n        raise RuntimeError('V40 invalid submission values')\n    if not _v40_parent.is_file() or not _v40_e10_audit.is_file():\n        raise RuntimeError('V40 missing parent or E10 receipt')\n    _v40_audit = _v40_json.loads(_v40_e10_audit.read_text())\n    if _v40_audit.get('status') != 'CANDIDATE_SELECTED' or _v40_audit.get('configuration') != 'uniform_060':\n        raise RuntimeError('V40 E10 promotion contract failed')\n    _v40_sha = lambda p: _v40_hashlib.sha256(p.read_bytes()).hexdigest()\n    if _v40_sha(_v40_primary) != _v40_audit.get('selected_sha256'):\n        raise RuntimeError('V40 E10 output hash mismatch')\n    _v40_receipt = {\n        'status': 'VALID_DINOV3_E10_HYBRID_ALPHA060',\n        'test_studies': len(_v40_sub),\n        'dynamic_test_ids_exact': True,\n        'schema_exact': True,\n        'finite_in_range': True,\n        'parent': 'DINOv2 rank ensemble plus cross-series DINOv3',\n        'correction': 'E10 RadImageNet uniform 0.60; Baker and Fracture preserved',\n        'parent_sha256': _v40_sha(_v40_parent),\n        'submission_sha256': _v40_sha(_v40_primary),\n        'e10_contract_sha256': _v40_audit.get('contract_sha256'),\n        'e10_heads_sha256': _v40_audit.get('heads_sha256'),\n    }\n    (_v40_work / 'v40_runtime_audit.json').write_text(\n        _v40_json.dumps(_v40_receipt, indent=2, sort_keys=True) + '\\n'\n    )\n    print(_v40_json.dumps(_v40_receipt, indent=2, sort_keys=True))\n"},{"cell_type":"markdown","id":"3e049761","metadata":{},"source":"## License\n\nThis notebook is a derivative work of **prvsiyan**'s *RSNA Knee: read the report,\nthen the knee*, released under the **Apache License 2.0**, and is published under\nthe same licence.\n\nChanges from the original: most of its later staging was removed, a DINOv3\nViT-S/16 was added, and the RadImageNet heads were retrained on our own fold\nsplit.\n\nIf you fork this notebook, Apache 2.0 requires you to keep these notices and the\nattributions above, and to state your own changes."},{"cell_type":"markdown","id":"v40-attribution","metadata":{},"source":"## V40 change notice\n\nV40 preserves the Apache-2.0 notices and author attributions in the parent notebook. Its E10 stage is adapted from fishface's public competition notebook and the public prvsiyan pipeline, with the hash-pinned Antoine Gagnon's v15 heads. The V40 change is limited to replacing the older Rad15 correction with the precommitted E10 uniform-0.35 correction after the DINOv3 parent is produced.\n"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.11.13"},"rsna_optimization":{"candidate":"v41-dinov3-e10-alpha060","parent":"v40-dinov3-e10-hybrid","selection":"uniform alpha 0.60 chosen in 5 of 5 grouped public outer folds","bootstrap_probability_beats_alpha020":0.9596666666666667}},"nbformat":4,"nbformat_minor":4}