{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.12.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"id":"dcb0f2f1","cell_type":"markdown","source":"# RSNA Knee Abnormality Detection — 2D CNN + Report Weak Labels Baseline (Stage A)\n\n**Notebook:** `RSNA_Knee_2D_CNN_ReportWeakLabels_Baseline`\n\n## Where this fits in the roadmap\n\n```\nBaseline (RSNA_Knee_2D_CNN_StudyLevel_Baseline, Public Score 0.537)\n        v\nBaseline Improvement — THIS NOTEBOOK (Stage A)\n        v\nImage Model Improvement / Multimodal (Stage B — pretrained backbone, multi-slot fusion)\n        v\nText Model -> Multimodal -> Ensemble -> Efficiency\n```\n\n## Diagnosis carried over from the previous baseline\n\nThe first baseline scored close to random (Public 0.537; internal OOF macro AUC 0.468 on\nonly 58 labeled studies). Test-set predictions barely varied across different studies,\nindicating the from-scratch CNN had essentially no signal to learn from **58 labeled\nstudies** for a 99K-parameter model. Two independently-verified public notebooks scoring\n0.80-0.89 confirmed the dominant lever is **not** a fancier CNN — it's using the `Report`\ncolumn to derive weak labels for the ~4,349 studies that have no gold labels, turning a\n58-sample problem into a ~4,200-sample one.\n\n## What this notebook changes vs. the first baseline\n\n1. **Report-derived weak labels** (multilingual, rule-based) applied to all training\n   studies, used to train on far more data than the 58 gold-labeled studies.\n2. **A DICOM slice-ordering bug fix.** The first baseline sorted DICOM files by *filename*\n   (SOP Instance UID) and took the \"middle\" one, assuming that approximates the anatomical\n   middle slice. This assumption is checked directly below and found to be close to random\n   — the correct approach uses the DICOM geometry tags.\n3. The 58 gold-labeled studies are **held out entirely** from training and used only as a\n   clean final evaluation set — this is the most trustworthy number in this notebook,\n   since (unlike the weak labels) it was never touched by report-derived label noise.\n\n## What is explicitly deferred to Stage B (not attempted here)\n\nPretrained backbones (e.g. DINOv2) requiring an attached offline weights dataset,\nmulti-series/multi-slot fusion, physical-scale-normalized cropping, laterality\nnormalization, and rank-based ensembling. Per the \"do not overfit the first notebook\"\nprinciple, Stage A isolates the effect of (1) and (2) above on top of the existing simple\nCNN, so their individual contribution can be understood before adding more complexity.\n","metadata":{}},{"id":"a90d4a0b","cell_type":"markdown","source":"## 1. Imports","metadata":{}},{"id":"c29aaedf","cell_type":"code","source":"import os, gc, re, time, random, unicodedata, warnings\nfrom pathlib import Path\n\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\nfrom sklearn.model_selection import KFold\nfrom sklearn.metrics import roc_auc_score\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.utils.data import Dataset, DataLoader\n\ntry:\n    import pydicom\n    PYDICOM_AVAILABLE = True\nexcept ImportError:\n    PYDICOM_AVAILABLE = False\n\nwarnings.filterwarnings(\"ignore\")\nsns.set_style(\"whitegrid\")\nprint(\"PyTorch:\", torch.__version__, \"| CUDA:\", torch.cuda.is_available(),\n      \"| pydicom:\", PYDICOM_AVAILABLE)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-08T13:09:29.561389Z","iopub.execute_input":"2026-08-08T13:09:29.561686Z","iopub.status.idle":"2026-08-08T13:09:43.670817Z","shell.execute_reply.started":"2026-08-08T13:09:29.561662Z","shell.execute_reply":"2026-08-08T13:09:43.670116Z"}},"outputs":[],"execution_count":null},{"id":"c44b30c9","cell_type":"markdown","source":"## 2. Configuration","metadata":{}},{"id":"c8f87d18","cell_type":"code","source":"class CFG:\n    COMP_DIR = \"/kaggle/input/competitions/rsna-knee-abnormality-detection\"\n    TRAIN_CSV = f\"{COMP_DIR}/train.csv\"\n    TRAIN_SERIES_CSV = f\"{COMP_DIR}/train_series.csv\"\n    TEST_CSV = f\"{COMP_DIR}/test.csv\"\n    TEST_SERIES_CSV = f\"{COMP_DIR}/test_series.csv\"\n    SAMPLE_SUB_CSV = f\"{COMP_DIR}/sample_submission.csv\"\n    TRAIN_DICOM_DIR = f\"{COMP_DIR}/train_series\"\n    TEST_DICOM_DIR = f\"{COMP_DIR}/test_series\"\n\n    TARGET_COLS = [\"ACL\", \"MCL\", \"Medial Meniscus\", \"Lateral Meniscus\", \"Medial OA\",\n                    \"Lateral OA\", \"PF OA\", \"Effusion\", \"Synovitis\", \"Baker's\",\n                    \"Contusion\", \"Fracture\"]\n    N_TARGETS = len(TARGET_COLS)\n\n    SEED = 42\n    N_INNER_FOLDS = 5          # used only within the weak-labeled set, for early stopping\n    IMG_SIZE = 224\n    MODEL_NAME = \"RSNA_Knee_2D_CNN_ReportWeakLabels_Baseline\"\n\n    BATCH_SIZE = 32\n    NUM_EPOCHS = 12\n    LEARNING_RATE = 1e-3\n    WEIGHT_DECAY = 1e-4\n    EARLY_STOP_PATIENCE = 3\n    NUM_WORKERS = 2\n    MIN_TARGETS_COVERED = 1     # keep a weakly-labeled study only if >=1 target was extracted\n\n    DEVICE = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n    OUTPUT_SUBMISSION = \"submission.csv\"\n\ncfg = CFG()\nprint(f\"Model name: {cfg.MODEL_NAME} | Device: {cfg.DEVICE}\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-08T13:09:43.671922Z","iopub.execute_input":"2026-08-08T13:09:43.672466Z","iopub.status.idle":"2026-08-08T13:09:43.678888Z","shell.execute_reply.started":"2026-08-08T13:09:43.672442Z","shell.execute_reply":"2026-08-08T13:09:43.678206Z"}},"outputs":[],"execution_count":null},{"id":"7091e675","cell_type":"markdown","source":"## 3. Seed setup","metadata":{}},{"id":"b45b26f0","cell_type":"code","source":"def set_seed(seed):\n    random.seed(seed); np.random.seed(seed)\n    torch.manual_seed(seed); torch.cuda.manual_seed_all(seed)\n    torch.backends.cudnn.deterministic = True\n    torch.backends.cudnn.benchmark = False\n\nset_seed(cfg.SEED)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-08T13:09:43.680428Z","iopub.execute_input":"2026-08-08T13:09:43.681114Z","iopub.status.idle":"2026-08-08T13:09:43.704518Z","shell.execute_reply.started":"2026-08-08T13:09:43.681084Z","shell.execute_reply":"2026-08-08T13:09:43.703761Z"}},"outputs":[],"execution_count":null},{"id":"cb51c1bc","cell_type":"markdown","source":"## 4. Load data and recap the gold-labeled subset","metadata":{}},{"id":"2970aa40","cell_type":"code","source":"train = pd.read_csv(cfg.TRAIN_CSV)\ntrain_series = pd.read_csv(cfg.TRAIN_SERIES_CSV)\ntest = pd.read_csv(cfg.TEST_CSV)\ntest_series = pd.read_csv(cfg.TEST_SERIES_CSV)\nsample_sub = pd.read_csv(cfg.SAMPLE_SUB_CSV)\n\ngold_mask = train[cfg.TARGET_COLS].notnull().any(axis=1)\ngold_df = train.loc[gold_mask].reset_index(drop=True)\nweak_source_df = train.loc[~gold_mask].reset_index(drop=True)\n\nprint(f\"Gold-labeled studies (held out for final evaluation only): {len(gold_df)}\")\nprint(f\"Studies with no gold label (candidates for report weak-labeling): {len(weak_source_df)}\")\nassert set(gold_df[\"StudyInstanceUID\"]).isdisjoint(set(weak_source_df[\"StudyInstanceUID\"]))\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-08T13:10:20.492955Z","iopub.execute_input":"2026-08-08T13:10:20.493411Z","iopub.status.idle":"2026-08-08T13:10:20.821939Z","shell.execute_reply.started":"2026-08-08T13:10:20.493383Z","shell.execute_reply":"2026-08-08T13:10:20.821283Z"}},"outputs":[],"execution_count":null},{"id":"4e66bdc8","cell_type":"markdown","source":"## 5. Report weak-label extractor\n\n**Decision.** Build a rule-based, multilingual (EN/ES/DE/FR/NL/TR/EL/HR-SR/RU-BG) extractor\nthat, for each of the 12 targets, looks for anatomy-specific terms at clause level and\nresolves polarity (positive / negative / normal-but-torn / uncertain) using nearby negation,\nuncertainty and normality cues. Uncertain mentions are skipped rather than guessed. A target\nwith no clause match at all is left as **unknown (`None`)**, not assumed negative — this\nmirrors the \"do not treat missing as negative\" instruction from Step 3 of the original task,\nnow applied to weak labels instead of gold labels.\n\n**Why rule-based instead of a trained NLP model.** No internet access at scoring time, and\nno labeled report-to-target training data exists to train a supervised text classifier in\nthe first place (the 58 gold labels are a per-image annotation, not a report-parsing\ndataset). A transparent, inspectable rule set also makes the extractor's behavior directly\nauditable against the 58 gold studies (next section) — important since these weak labels\nwill directly drive what the image model learns.\n\n**Trade-off.** This is necessarily an approximation of real radiology report understanding\nand does not match the sophistication of dedicated clinical NLP. It is validated explicitly\nbelow rather than trusted blindly.\n","metadata":{}},{"id":"095155f3","cell_type":"code","source":"_PRE = str.maketrans({\"\\u0131\": \"i\", \"\\u0130\": \"i\",\n                       \"\\u00df\": \"ss\", \"\\u0111\": \"d\", \"\\u0110\": \"d\",\n                       \"\\u00f8\": \"o\", \"\\u00d8\": \"o\", \"\\u00e6\": \"ae\", \"\\u00c6\": \"ae\"})\n\ndef normalize_text(text):\n    if not isinstance(text, str):\n        return \"\"\n    text = text.translate(_PRE).lower()\n    text = unicodedata.normalize(\"NFKD\", text)\n    text = \"\".join(ch for ch in text if not unicodedata.combining(ch))\n    text = re.sub(r\"[_\\-/\\\\]+\", \" \", text)\n    text = re.sub(r\"[ \\t]+\", \" \", text)\n    return text\n\ndef split_clauses(text):\n    text = normalize_text(text)\n    parts = re.split(r\"[.;\\n]+\", text)\n    return [p.strip() for p in parts if p.strip()]\n\ndef _rx(*pats):\n    return re.compile(\"|\".join(pats))\n\nNEGATION = _rx(\n    r\"\\bno\\b\", r\"\\bnot\\b\", r\"\\bwithout\\b\", r\"\\bnegative for\\b\", r\"\\babsence\\b\",\n    r\"\\bno evidence\\b\", r\"\\bunremarkable\\b\", r\"\\bfree of\\b\",\n    r\"\\bsin\\b\", r\"\\bno hay\\b\", r\"\\bausencia\\b\", r\"\\bausentes?\\b\",\n    r\"\\bpas de\\b\", r\"\\bsans\\b\", r\"\\baucune?\\b\",\n    r\"\\bgeen\\b\", r\"\\bzonder\\b\", r\"\\bniet\\b\",\n    r\"\\bkeine?\\b\", r\"\\bohne\\b\", r\"\\bnicht\\b\",\n    r\"\\byok\\b\", r\"\\byoktur\\b\", r\"izlenmemekte\", r\"saptanmadi\", r\"\\bdegil\\b\",\n    r\"gozlenmemekte\", r\"mevcut degil\", r\"\\bizlenmedi\\b\",\n    r\"\\bnema\\b\", r\"\\bbez\\b\", r\"\\bnisu\\b\", r\"\\bnije\\b\",\n    r\"\\bdelta\\b|\\bden\\b\", r\"\\bxoris\\b\",\n    r\"\\bбез\\b\", r\"\\bне\\b\", r\"липсва\", r\"\\bняма\\b\",\n)\nNORMALITY = _rx(\n    r\"\\bnormal\", r\"\\bintact\\b\", r\"\\bpreserved\\b\", r\"\\bwithin normal limits\\b\",\n    r\"limites normales\", r\"\\bconservad\", r\"\\bintegr\", r\"\\bnormales\\b\",\n    r\"\\bdogal\\b\", r\"korunmus\", r\"\\bnormaldir\\b\", r\"olagan\",\n    r\"\\buredn\", r\"\\bocuvan\", r\"\\bodrzan\", r\"\\bintakt\",\n    r\"unauffallig\", r\"regelrecht\",\n    r\"normal\", r\"запазен\", r\"съхранен\",\n    r\"\\bgaaf\\b\", r\"\\bnormaal\\b\",\n)\nUNCERTAIN = _rx(\n    r\"\\bpossible\\b\", r\"\\bprobable\\b\", r\"\\bsuspicious\\b\", r\"\\bsuspected\\b\",\n    r\"cannot (be )?exclude\", r\"\\bmay\\b\", r\"\\bquestionable\\b\", r\"\\bequivocal\\b\",\n    r\"\\bposible\\b\", r\"\\bdudos\", r\"peut.?etre\", r\"\\bsuspect\",\n    r\"\\bmogelijk\\b\", r\"\\bmoglich\\b\", r\"\\bfraglich\\b\",\n    r\"\\bolasi\\b\", r\"\\bsupheli\\b\",\n    r\"\\bmoguc\", r\"\\bsumnja\",\n)\nTEAR = _rx(\n    r\"\\btear\", r\"\\btorn\\b\", r\"\\brupture\", r\"\\bdisruption\\b\", r\"discontinuit\", r\"\\bavuls\",\n    r\"\\brotura\\b\", r\"\\broturas\\b\", r\"\\bruptura\", r\"\\bdesgarro\", r\"\\broto\\b\",\n    r\"\\bdechirure\", r\"\\bdechire\",\n    r\"\\bscheur\", r\"\\bruptuur\", r\"gescheurd\",\n    r\"\\briss\\b\", r\"einriss\", r\"\\bruptur\", r\"\\blasion\",\n    r\"\\byirtik\", r\"\\bruptur\",\n)\nANAT = {\n    \"ACL\": _rx(r\"anterior cruciate\", r\"\\bacl\\b\", r\"cruzado anterior\", r\"\\blca\\b\",\n               r\"croise anterieur\", r\"voorste kruisband\", r\"\\bvkb\\b\",\n               r\"vorderes? kreuzband\", r\"vorderen kreuzband\", r\"on capraz\", r\"\\bocb\\b\",\n               r\"prednji krizni\", r\"prednjeg krizn\", r\"cruciate ligaments\",\n               r\"ligamentos cruzados\", r\"ligaments croises\", r\"kruisbanden\",\n               r\"kreuzbander\", r\"capraz baglar\", r\"krizn\\w* ligament\\w*\"),\n    \"MCL\": _rx(r\"medial collateral\", r\"\\bmcl\\b\", r\"tibial collateral\",\n               r\"colateral medial\", r\"colateral interno\", r\"\\blcm\\b\",\n               r\"collateral medial\", r\"collateral interne\", r\"mediale collaterale\",\n               r\"binnenband\", r\"mediale banden\", r\"innenband\", r\"ic yan bag\",\n               r\"medijalni kolateral\"),\n    \"Medial Meniscus\": _rx(r\"medial meniscus\", r\"menisco medial\", r\"menisco interno\",\n                            r\"menisque interne\", r\"mediale meniscus\", r\"innenmeniskus\",\n                            r\"medial menisk\", r\"ic menisk\", r\"medijalni meniskus\"),\n    \"Lateral Meniscus\": _rx(r\"lateral meniscus\", r\"menisco lateral\", r\"menisco externo\",\n                             r\"menisque externe\", r\"laterale meniscus\", r\"aussenmeniskus\",\n                             r\"lateral menisk\", r\"dis menisk\", r\"lateralni meniskus\"),\n    \"Medial OA\": _rx(r\"medial.{0,20}(osteoarthritis|oa\\b|compartment)\",\n                      r\"medial.{0,20}artrosis\", r\"artrosis.{0,20}medial\",\n                      r\"femorotibial medial\", r\"gonarthrose.{0,15}medial\",\n                      r\"medial.{0,15}gonarthrose\", r\"medial.{0,15}arthrose\",\n                      r\"medial compartment.{0,20}(arthros|degenerat|chondr)\",\n                      r\"ic bolme.{0,15}artroz\", r\"medijaln\\w*.{0,15}artroz\"),\n    \"Lateral OA\": _rx(r\"lateral.{0,20}(osteoarthritis|oa\\b|compartment)\",\n                       r\"lateral.{0,20}artrosis\", r\"artrosis.{0,20}lateral\",\n                       r\"femorotibial lateral\", r\"gonarthrose.{0,15}lateral\",\n                       r\"lateral.{0,15}gonarthrose\", r\"lateral.{0,15}arthrose\",\n                       r\"lateral compartment.{0,20}(arthros|degenerat|chondr)\",\n                       r\"dis bolme.{0,15}artroz\", r\"lateralni\\w*.{0,15}artroz\"),\n    \"PF OA\": _rx(r\"patellofemoral.{0,20}(osteoarthritis|oa\\b|arthros|chondr|degenerat)\",\n                 r\"femoropatelar\", r\"retropatelar\", r\"patelofemoral\",\n                 r\"chondropathie retropatellaire\", r\"chondropatia retropatelar\",\n                 r\"femoropatellaire\", r\"patellofemorale.{0,15}arthrose\",\n                 r\"retropatellar\\w*\", r\"patellofemoral\\w*\"),\n    \"Effusion\": _rx(r\"\\beffusion\", r\"\\bderrame\\b\", r\"\\berguss\\b\", r\"epanchement\",\n                     r\"gewrichtsvocht\", r\"\\bvocht\\b\", r\"\\bsivi\\b\", r\"efuzyon\",\n                     r\"joint fluid\", r\"\\bhydrarthros\"),\n    \"Synovitis\": _rx(r\"synovitis\", r\"sinovitis\", r\"synoviaal\", r\"synovialitis\", r\"sinovit\"),\n    \"Baker's\": _rx(r\"baker.?s? cyst\", r\"quiste de baker\", r\"bakerzyste\", r\"kyste de baker\",\n                    r\"bakercyste\", r\"baker kisti\", r\"popliteal cyst\", r\"quiste poplite\"),\n    \"Contusion\": _rx(r\"\\bcontusion\", r\"kontusion\", r\"bone bruise\", r\"kneuzing\",\n                      r\"beenkneuzing\", r\"kemik kontuzyon\", r\"kontuzyon\",\n                      r\"bone marrow edema\", r\"edema osseo\", r\"medulla.{0,10}edema\"),\n    \"Fracture\": _rx(r\"\\bfracture\", r\"\\bfractura\\b\", r\"\\bfraktur\", r\"\\bfractuur\",\n                     r\"\\bkirik\\b\", r\"\\bkirik\\b\"),\n}\nTARGET_COLS_LOCAL = list(ANAT.keys())\n\ndef _polarity(clause):\n    if UNCERTAIN.search(clause):\n        return \"uncertain\"\n    if NEGATION.search(clause):\n        return \"negative\"\n    if NORMALITY.search(clause):\n        return \"positive\" if TEAR.search(clause) else \"negative\"\n    return \"positive\"\n\ndef extract_report_labels(report_text):\n    result = {t: None for t in TARGET_COLS_LOCAL}\n    for cl in split_clauses(report_text):\n        for t in TARGET_COLS_LOCAL:\n            if ANAT[t].search(cl):\n                pol = _polarity(cl)\n                if pol == \"uncertain\":\n                    continue\n                val = 1 if pol == \"positive\" else 0\n                if result[t] is None:\n                    result[t] = val\n                elif result[t] == 0 and val == 1:\n                    result[t] = 1\n    return result\n\nprint(f\"Extractor covers {len(ANAT)} targets across 9 languages (EN/ES/DE/FR/NL/TR/EL/HR-SR/RU-BG).\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-08T13:10:24.813962Z","iopub.execute_input":"2026-08-08T13:10:24.81441Z","iopub.status.idle":"2026-08-08T13:10:24.836147Z","shell.execute_reply.started":"2026-08-08T13:10:24.814382Z","shell.execute_reply":"2026-08-08T13:10:24.835377Z"}},"outputs":[],"execution_count":null},{"id":"3225b2fd","cell_type":"markdown","source":"## 6. Validate the extractor against the 58 gold-labeled studies\n\nBefore trusting these weak labels to train anything, we measure how often the extractor's\ncall agrees with the real radiologist-derived labels, and how much coverage it achieves,\n**per target**. This uses the *training* reports of the gold-labeled studies purely as a\nvalidation set for the extractor itself (not for training the extractor, since it has no\nlearned parameters).\n","metadata":{}},{"id":"da30b189","cell_type":"code","source":"gold_extracted = pd.DataFrame([extract_report_labels(r) for r in gold_df[\"Report\"]])\n\nrows = []\nfor t in cfg.TARGET_COLS:\n    gold_col = gold_df[t]\n    pred_col = gold_extracted[t]\n    covered = pred_col.notnull()\n    coverage = covered.mean()\n    agreement = (gold_col[covered] == pred_col[covered]).mean() if covered.sum() > 0 else np.nan\n    rows.append({\"target\": t, \"coverage\": coverage, \"agreement_when_covered\": agreement,\n                 \"n_gold_positive\": int(gold_col.sum())})\n\nextractor_eval = pd.DataFrame(rows).set_index(\"target\")\nprint(extractor_eval)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-08T13:10:31.155623Z","iopub.execute_input":"2026-08-08T13:10:31.156232Z","iopub.status.idle":"2026-08-08T13:10:31.265219Z","shell.execute_reply.started":"2026-08-08T13:10:31.156201Z","shell.execute_reply":"2026-08-08T13:10:31.264505Z"}},"outputs":[],"execution_count":null},{"id":"6f806d8b","cell_type":"code","source":"fig, ax = plt.subplots(figsize=(9, 5))\nextractor_eval[[\"coverage\", \"agreement_when_covered\"]].plot(kind=\"bar\", ax=ax,\n                                                              color=[\"steelblue\", \"darkorange\"])\nax.axhline(0.5, color=\"red\", linestyle=\"--\", linewidth=1)\nax.set_ylim(0, 1)\nax.set_title(\"Report weak-label extractor: coverage and agreement vs. gold labels (n=58)\")\nplt.xticks(rotation=60)\nplt.tight_layout()\nplt.show()\n\nweak_targets = extractor_eval[extractor_eval[\"agreement_when_covered\"] < 0.6].index.tolist()\nprint(\n    f\"\\nHONEST CAVEAT: agreement is weakest for {weak_targets if weak_targets else '(none below 60%)'} \"\n    \"— these targets' weak labels are noisier and the resulting model quality for them should \"\n    \"be trusted less than for high-agreement targets like Fracture or Lateral Meniscus. This is \"\n    \"expected from a compact rule-based lexicon and is a direct target for future iteration \"\n    \"(e.g. expanding the term lists, or down-weighting low-agreement targets in the loss).\"\n)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-08T13:10:33.746301Z","iopub.execute_input":"2026-08-08T13:10:33.746884Z","iopub.status.idle":"2026-08-08T13:10:34.22445Z","shell.execute_reply.started":"2026-08-08T13:10:33.746857Z","shell.execute_reply":"2026-08-08T13:10:34.223778Z"}},"outputs":[],"execution_count":null},{"id":"42dd9fbd","cell_type":"markdown","source":"## 7. Build the weak-labeled training set\n\n**Decision.** Train only on the ~4,349 studies with **no** gold label, using the extractor's\nper-target calls with `None` treated as missing and masked out of the loss (never imputed as\nnegative). The 58 gold-labeled studies are **not** added into training, even partially — they\nare reserved entirely for the final evaluation in Section 13, so that number reflects real\nlabels only, uncontaminated by weak-label noise or the model having seen them during fitting.\n","metadata":{}},{"id":"ef476275","cell_type":"code","source":"weak_rows = []\nfor _, r in weak_source_df.iterrows():\n    labs = extract_report_labels(r[\"Report\"])\n    labs[\"StudyInstanceUID\"] = r[\"StudyInstanceUID\"]\n    weak_rows.append(labs)\n\nweak_labels_df = pd.DataFrame(weak_rows)\nn_targets_covered = weak_labels_df[cfg.TARGET_COLS].notnull().sum(axis=1)\nweak_labels_df = weak_labels_df[n_targets_covered >= cfg.MIN_TARGETS_COVERED].reset_index(drop=True)\n\nprint(f\"Weakly-labeled studies retained for training: {len(weak_labels_df)} / {len(weak_source_df)}\")\nprint(f\"Average targets covered per retained study: {n_targets_covered[n_targets_covered >= cfg.MIN_TARGETS_COVERED].mean():.2f} / {cfg.N_TARGETS}\")\nprint(\"\\nPer-target coverage on the training set actually used:\")\nprint((weak_labels_df[cfg.TARGET_COLS].notnull().mean() * 100).round(1))\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-08T13:10:37.160966Z","iopub.execute_input":"2026-08-08T13:10:37.161336Z","iopub.status.idle":"2026-08-08T13:10:42.585487Z","shell.execute_reply.started":"2026-08-08T13:10:37.161309Z","shell.execute_reply":"2026-08-08T13:10:42.584765Z"}},"outputs":[],"execution_count":null},{"id":"b6ddc60e","cell_type":"markdown","source":"## 8. DICOM slice-ordering fix\n\n**The bug being fixed.** The first baseline listed a series' `.dcm` files with\n`sorted(glob(\"*.dcm\"))`, which sorts by **filename** (commonly the SOP Instance UID), and\ntook the middle *file* as a proxy for the middle *anatomical slice*. Filenames are not\nguaranteed to correlate with slice position — this section checks that directly against\nreal DICOM headers and fixes it if confirmed.\n\n**The fix.** For each file in a series, project `ImagePositionPatient` onto the series'\nslice-normal direction (the cross product of the two `ImageOrientationPatient` row/column\nvectors). Sorting by this scalar recovers the true anatomical order regardless of filename.\nFalls back to the DICOM `InstanceNumber` tag if orientation/position tags are missing, and\nfinally to filename order only as a last resort.\n\n**Why this matters.** For any series with meaningful anatomical structure (e.g. a knee joint\nimaged sagittally), the \"middle\" slice is where the joint is best visualized. Picking a\nfilename-middle slice that is actually near an edge of the volume can mean training on\nmostly background/muscle instead of the joint itself — silently degrading signal without\nraising any error.\n","metadata":{}},{"id":"293d53a8","cell_type":"code","source":"def list_series_files(study_uid, series_uid, dicom_root):\n    d = Path(dicom_root) / study_uid / series_uid\n    if not d.exists():\n        return []\n    return sorted(d.glob(\"*.dcm\"))\n\n\ndef geometric_slice_key(ds):\n    try:\n        iop = [float(x) for x in ds.ImageOrientationPatient]\n        ipp = [float(x) for x in ds.ImagePositionPatient]\n        row = np.array(iop[:3]); col = np.array(iop[3:6])\n        normal = np.cross(row, col)\n        return float(np.dot(normal, ipp))\n    except Exception:\n        return None\n\n\ndef order_series_files(files):\n    '''Return files sorted into true anatomical slice order (geometry > InstanceNumber > filename).'''\n    if not PYDICOM_AVAILABLE or len(files) == 0:\n        return files, \"no_pydicom_or_empty\"\n\n    keyed, method = [], \"geometry\"\n    for f in files:\n        try:\n            ds = pydicom.dcmread(str(f), stop_before_pixels=True, force=True)\n            k = geometric_slice_key(ds)\n            if k is None:\n                k = getattr(ds, \"InstanceNumber\", None)\n                k = float(k) if k is not None else None\n                method = \"instance_number\" if method == \"geometry\" else method\n        except Exception:\n            k = None\n        keyed.append((k, f))\n\n    if all(k is None for k, _ in keyed):\n        return sorted(files), \"filename_fallback\"\n\n    # Any remaining None keys (a few unreadable headers) go to the end, stable otherwise.\n    keyed_sorted = sorted(keyed, key=lambda x: (x[0] is None, x[0]))\n    return [f for _, f in keyed_sorted], method\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-08T13:10:47.476246Z","iopub.execute_input":"2026-08-08T13:10:47.476508Z","iopub.status.idle":"2026-08-08T13:10:47.484778Z","shell.execute_reply.started":"2026-08-08T13:10:47.476487Z","shell.execute_reply":"2026-08-08T13:10:47.484239Z"}},"outputs":[],"execution_count":null},{"id":"d40d2ede","cell_type":"code","source":"# Quick empirical check (runs only if real DICOM files are present in this environment):\n# compare filename order vs. geometric order for a small sample of series, to confirm the\n# bug actually manifests before relying on the fix.\nsample_pairs = train_series[[\"StudyInstanceUID\", \"SeriesInstanceUID\"]].drop_duplicates().head(20)\ncorrelations = []\nfor _, r in sample_pairs.iterrows():\n    files = list_series_files(r[\"StudyInstanceUID\"], r[\"SeriesInstanceUID\"], cfg.TRAIN_DICOM_DIR)\n    if len(files) < 5:\n        continue\n    filename_order = {f: i for i, f in enumerate(sorted(files))}\n    ordered, method = order_series_files(files)\n    geom_order = {f: i for i, f in enumerate(ordered)}\n    fn_ranks = [filename_order[f] for f in files]\n    geom_ranks = [geom_order[f] for f in files]\n    if len(set(fn_ranks)) > 1 and len(set(geom_ranks)) > 1:\n        corr = np.corrcoef(fn_ranks, geom_ranks)[0, 1]\n        correlations.append(corr)\n\nif correlations:\n    print(f\"Filename-order vs. geometric-order rank correlation \"\n          f\"(n={len(correlations)} series): mean={np.mean(correlations):.3f}, \"\n          f\"min={np.min(correlations):.3f}, max={np.max(correlations):.3f}\")\n    print(\"A correlation well below 1.0 confirms filename order is not a reliable proxy \"\n          \"for anatomical order, justifying the fix above.\")\nelse:\n    print(\"No DICOM files found in this environment to run the empirical check \"\n          \"(expected outside the Kaggle competition environment).\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-08T13:10:51.012396Z","iopub.execute_input":"2026-08-08T13:10:51.012646Z","iopub.status.idle":"2026-08-08T13:10:57.419881Z","shell.execute_reply.started":"2026-08-08T13:10:51.012626Z","shell.execute_reply":"2026-08-08T13:10:57.418944Z"}},"outputs":[],"execution_count":null},{"id":"89b8311f","cell_type":"markdown","source":"## 9. Series selection (same EDA-derived rule as the first baseline)","metadata":{}},{"id":"57f780d5","cell_type":"code","source":"def select_representative_series(study_uid, series_df, dicom_root):\n    study_series = series_df[series_df[\"StudyInstanceUID\"] == study_uid]\n    if len(study_series) == 0:\n        return None\n\n    def pick_most_slices(candidates):\n        best_row, best_files = None, []\n        for _, row in candidates.iterrows():\n            files = list_series_files(study_uid, row[\"SeriesInstanceUID\"], dicom_root)\n            if len(files) > len(best_files):\n                best_row, best_files = row, files\n        return best_row, best_files\n\n    rule1 = study_series[(study_series[\"Anatomical_Plane\"] == \"Sagittal\") &\n                          (study_series[\"Fluid_Sensitive\"] == 1)]\n    if len(rule1) > 0:\n        row, files = pick_most_slices(rule1)\n        if row is not None:\n            return {\"series_uid\": row[\"SeriesInstanceUID\"], \"rule\": \"sagittal_fluid_sensitive\",\n                    \"files\": files}\n\n    rule2 = study_series[study_series[\"Anatomical_Plane\"] == \"Sagittal\"]\n    if len(rule2) > 0:\n        row, files = pick_most_slices(rule2)\n        if row is not None:\n            return {\"series_uid\": row[\"SeriesInstanceUID\"], \"rule\": \"sagittal_any\", \"files\": files}\n\n    row, files = pick_most_slices(study_series)\n    if row is not None:\n        return {\"series_uid\": row[\"SeriesInstanceUID\"], \"rule\": \"any_series\", \"files\": files}\n    return None\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"6ca68c4d","cell_type":"markdown","source":"## 10. Precompute the representative slice path per study (once, not per epoch)\n\n**Decision.** Series selection + geometric slice ordering both require reading DICOM\nheaders, which is far more expensive than the filename-sort used in the first baseline. We\ntherefore compute the final **middle-slice file path** for every study exactly once up\nfront and cache it in a DataFrame, instead of recomputing it inside `Dataset.__getitem__`\non every access across every training epoch. This keeps per-epoch cost to \"decode one\nalready-known DICOM file per study\" rather than \"re-list and re-order a whole series\".\n","metadata":{}},{"id":"dc1938bf","cell_type":"code","source":"def build_slice_index(study_ids, series_df, dicom_root):\n    rows = []\n    for study_uid in study_ids:\n        sel = select_representative_series(study_uid, series_df, dicom_root)\n        if sel is None or len(sel[\"files\"]) == 0:\n            rows.append({\"StudyInstanceUID\": study_uid, \"series_uid\": None,\n                         \"rule\": \"no_series_found\", \"slice_path\": None, \"order_method\": None})\n            continue\n        ordered_files, order_method = order_series_files(sel[\"files\"])\n        mid_path = ordered_files[len(ordered_files) // 2]\n        rows.append({\"StudyInstanceUID\": study_uid, \"series_uid\": sel[\"series_uid\"],\n                     \"rule\": sel[\"rule\"], \"slice_path\": str(mid_path),\n                     \"order_method\": order_method})\n    return pd.DataFrame(rows)\n\nt0 = time.time()\ntrain_slice_index = build_slice_index(weak_labels_df[\"StudyInstanceUID\"], train_series, cfg.TRAIN_DICOM_DIR)\ngold_slice_index = build_slice_index(gold_df[\"StudyInstanceUID\"], train_series, cfg.TRAIN_DICOM_DIR)\nprint(f\"Slice-index build time: {time.time() - t0:.1f}s\")\nprint(\"\\nSeries-selection rule usage (weak-labeled training set):\")\nprint(train_slice_index[\"rule\"].value_counts())\nprint(\"\\nSlice-ordering method usage (weak-labeled training set):\")\nprint(train_slice_index[\"order_method\"].value_counts())\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"79da060d","cell_type":"markdown","source":"## 11. Slice loading, dataset and masked multi-label collate","metadata":{}},{"id":"adf249c4","cell_type":"code","source":"def load_slice(slice_path, img_size):\n    if slice_path is None:\n        return np.zeros((img_size, img_size), dtype=np.float32), False\n    try:\n        ds = pydicom.dcmread(slice_path)\n        arr = ds.pixel_array.astype(np.float32)\n        slope = float(getattr(ds, \"RescaleSlope\", 1.0))\n        intercept = float(getattr(ds, \"RescaleIntercept\", 0.0))\n        arr = arr * slope + intercept\n        lo, hi = arr.min(), arr.max()\n        arr = (arr - lo) / (hi - lo) if hi > lo else np.zeros_like(arr)\n        t = torch.from_numpy(arr).unsqueeze(0).unsqueeze(0)\n        t = F.interpolate(t, size=(img_size, img_size), mode=\"bilinear\", align_corners=False)\n        return t.squeeze(0).squeeze(0).numpy().astype(np.float32), True\n    except Exception:\n        return np.zeros((img_size, img_size), dtype=np.float32), False\n\n\nclass KneeMaskedDataset(Dataset):\n    '''One sample = one study. Labels may contain NaN (unknown) which is converted into a\n    parallel mask tensor so the loss can ignore unknown targets per-sample, per-target.'''\n\n    def __init__(self, slice_index, img_size, labels_df=None, target_cols=None):\n        self.slice_index = slice_index.reset_index(drop=True)\n        self.img_size = img_size\n        self.target_cols = target_cols or cfg.TARGET_COLS\n        if labels_df is not None:\n            lab = (labels_df.set_index(\"StudyInstanceUID\")[self.target_cols]\n                   .reindex(self.slice_index[\"StudyInstanceUID\"]))\n            self.mask = lab.notnull().values.astype(np.float32)\n            self.labels = lab.fillna(0.0).values.astype(np.float32)\n        else:\n            self.labels, self.mask = None, None\n\n    def __len__(self):\n        return len(self.slice_index)\n\n    def __getitem__(self, idx):\n        row = self.slice_index.iloc[idx]\n        img, _ = load_slice(row[\"slice_path\"], self.img_size)\n        img_t = torch.from_numpy(img).unsqueeze(0)\n        if self.labels is not None:\n            return img_t, torch.from_numpy(self.labels[idx]), torch.from_numpy(self.mask[idx])\n        return img_t, row[\"StudyInstanceUID\"]\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"179ae11b","cell_type":"markdown","source":"## 12. Model, masked loss, and training loop","metadata":{}},{"id":"a92a1f1f","cell_type":"code","source":"class SimpleKneeCNN(nn.Module):\n    def __init__(self, n_targets, in_channels=1):\n        super().__init__()\n        self.features = nn.Sequential(\n            nn.Conv2d(in_channels, 16, 3, padding=1), nn.BatchNorm2d(16), nn.ReLU(), nn.MaxPool2d(2),\n            nn.Conv2d(16, 32, 3, padding=1), nn.BatchNorm2d(32), nn.ReLU(), nn.MaxPool2d(2),\n            nn.Conv2d(32, 64, 3, padding=1), nn.BatchNorm2d(64), nn.ReLU(), nn.MaxPool2d(2),\n            nn.Conv2d(64, 128, 3, padding=1), nn.BatchNorm2d(128), nn.ReLU(), nn.MaxPool2d(2),\n            nn.Conv2d(128, 128, 3, padding=1), nn.BatchNorm2d(128), nn.ReLU(),\n            nn.AdaptiveAvgPool2d(1),\n        )\n        self.dropout = nn.Dropout(0.3)\n        self.head = nn.Linear(128, n_targets)\n\n    def forward(self, x):\n        x = self.features(x).flatten(1)\n        return self.head(self.dropout(x))\n\n\ndef masked_bce_loss(logits, targets, mask, pos_weight=None):\n    loss = F.binary_cross_entropy_with_logits(logits, targets, pos_weight=pos_weight, reduction=\"none\")\n    loss = loss * mask\n    denom = mask.sum().clamp(min=1.0)\n    return loss.sum() / denom\n\n\n# Per-target pos_weight from the weak-labeled training set's observed positive rate\n# (helps counter imbalance and low-coverage targets contributing little gradient signal).\npos_counts = np.nansum(np.where(weak_labels_df[cfg.TARGET_COLS].values == 1, 1, 0), axis=0)\nneg_counts = np.nansum(np.where(weak_labels_df[cfg.TARGET_COLS].values == 0, 1, 0), axis=0)\npos_weight_vals = np.clip(neg_counts / np.maximum(pos_counts, 1), 1.0, 10.0)\npos_weight_t = torch.tensor(pos_weight_vals, dtype=torch.float32, device=cfg.DEVICE)\nprint(\"Per-target pos_weight (clipped to [1, 10]):\")\nprint(pd.Series(pos_weight_vals, index=cfg.TARGET_COLS).round(2))\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"bac516d3","cell_type":"code","source":"# Inner split of the weak-labeled set purely for early-stopping / model selection.\n# (The number that matters for reporting is the Section 13 evaluation on the 58 gold studies.)\nkf = KFold(n_splits=cfg.N_INNER_FOLDS, shuffle=True, random_state=cfg.SEED)\ntrain_slice_index = train_slice_index.reset_index(drop=True)\ninner_idx = next(kf.split(train_slice_index))\ntr_idx, va_idx = inner_idx\n\ntrain_ds = KneeMaskedDataset(train_slice_index.iloc[tr_idx], cfg.IMG_SIZE,\n                              labels_df=weak_labels_df, target_cols=cfg.TARGET_COLS)\nval_ds = KneeMaskedDataset(train_slice_index.iloc[va_idx], cfg.IMG_SIZE,\n                            labels_df=weak_labels_df, target_cols=cfg.TARGET_COLS)\n\ntrain_loader = DataLoader(train_ds, batch_size=cfg.BATCH_SIZE, shuffle=True,\n                           num_workers=cfg.NUM_WORKERS, drop_last=True)\nval_loader = DataLoader(val_ds, batch_size=cfg.BATCH_SIZE, shuffle=False, num_workers=cfg.NUM_WORKERS)\n\nprint(f\"Inner train studies: {len(train_ds)} | inner val studies: {len(val_ds)}\")\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"73906b1b","cell_type":"code","source":"t_start = time.time()\nmodel = SimpleKneeCNN(cfg.N_TARGETS).to(cfg.DEVICE)\noptimizer = torch.optim.AdamW(model.parameters(), lr=cfg.LEARNING_RATE, weight_decay=cfg.WEIGHT_DECAY)\n\nbest_val_loss, best_state, epochs_no_improve = float(\"inf\"), None, 0\n\nfor epoch in range(cfg.NUM_EPOCHS):\n    model.train()\n    tr_loss = 0.0\n    for imgs, labels, mask in train_loader:\n        imgs, labels, mask = imgs.to(cfg.DEVICE), labels.to(cfg.DEVICE), mask.to(cfg.DEVICE)\n        if random.random() < 0.5:\n            imgs = torch.flip(imgs, dims=[3])\n        optimizer.zero_grad()\n        logits = model(imgs)\n        loss = masked_bce_loss(logits, labels, mask, pos_weight=pos_weight_t)\n        loss.backward()\n        optimizer.step()\n        tr_loss += loss.item() * imgs.size(0)\n    tr_loss /= len(train_ds)\n\n    model.eval()\n    va_loss = 0.0\n    with torch.no_grad():\n        for imgs, labels, mask in val_loader:\n            imgs, labels, mask = imgs.to(cfg.DEVICE), labels.to(cfg.DEVICE), mask.to(cfg.DEVICE)\n            logits = model(imgs)\n            va_loss += masked_bce_loss(logits, labels, mask, pos_weight=pos_weight_t).item() * imgs.size(0)\n    va_loss /= max(len(val_ds), 1)\n\n    if va_loss < best_val_loss:\n        best_val_loss, best_state, epochs_no_improve = va_loss, {k: v.cpu().clone() for k, v in model.state_dict().items()}, 0\n    else:\n        epochs_no_improve += 1\n\n    print(f\"epoch {epoch:2d} | train_loss={tr_loss:.4f} | val_loss={va_loss:.4f}\")\n    if epochs_no_improve >= cfg.EARLY_STOP_PATIENCE:\n        print(f\"Early stopping at epoch {epoch}\")\n        break\n\nmodel.load_state_dict(best_state)\ntrain_time_sec = time.time() - t_start\nprint(f\"\\nTraining time: {train_time_sec:.1f}s ({train_time_sec/60:.2f} min)\")\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"16027683","cell_type":"markdown","source":"## 13. Final, honest evaluation on the 58 gold-labeled studies\n\nThis is the number that should actually be trusted: it uses only real radiologist-derived\nlabels, on studies the model never saw (directly or via weak labels) during training.\n","metadata":{}},{"id":"259f1fc1","cell_type":"code","source":"gold_ds = KneeMaskedDataset(gold_slice_index, cfg.IMG_SIZE, labels_df=gold_df, target_cols=cfg.TARGET_COLS)\ngold_loader = DataLoader(gold_ds, batch_size=cfg.BATCH_SIZE, shuffle=False, num_workers=cfg.NUM_WORKERS)\n\nmodel.eval()\nall_logits = []\nwith torch.no_grad():\n    for imgs, labels, mask in gold_loader:\n        imgs = imgs.to(cfg.DEVICE)\n        all_logits.append(model(imgs).cpu().numpy())\ngold_logits = np.concatenate(all_logits, axis=0)\ngold_probs = 1 / (1 + np.exp(-gold_logits))\ngold_y = gold_df[cfg.TARGET_COLS].values.astype(np.float32)\n\n\ndef compute_auc_report(y_true, y_pred, target_cols):\n    rows, aucs = [], []\n    for i, t in enumerate(target_cols):\n        yt, yp = y_true[:, i], y_pred[:, i]\n        if len(np.unique(yt)) < 2:\n            rows.append({\"target\": t, \"auc\": np.nan, \"n_positive\": int(yt.sum())})\n            continue\n        auc = roc_auc_score(yt, yp)\n        rows.append({\"target\": t, \"auc\": auc, \"n_positive\": int(yt.sum())})\n        aucs.append(auc)\n    return pd.DataFrame(rows).set_index(\"target\"), (float(np.mean(aucs)) if aucs else float(\"nan\"))\n\n\ngold_auc_report, gold_macro_auc = compute_auc_report(gold_y, gold_probs, cfg.TARGET_COLS)\nprint(gold_auc_report)\nprint(f\"\\nMacro ROC AUC on the 58 held-out GOLD-labeled studies: {gold_macro_auc:.4f}\")\nprint(\"(Compare to the first baseline's OOF macro AUC of 0.468 and Public LB of 0.537.)\")\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"5db103df","cell_type":"code","source":"fig, ax = plt.subplots(figsize=(9, 5))\ngold_auc_report[\"auc\"].plot(kind=\"bar\", ax=ax, color=\"darkslateblue\")\nax.axhline(0.5, color=\"red\", linestyle=\"--\", label=\"random (0.5)\")\nax.axhline(gold_macro_auc, color=\"green\", linestyle=\"--\", label=f\"macro AUC = {gold_macro_auc:.3f}\")\nax.set_ylim(0, 1); ax.legend()\nax.set_title(f\"{cfg.MODEL_NAME} — per-target AUC on 58 gold studies\")\nplt.xticks(rotation=60); plt.tight_layout(); plt.show()\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"854c477e","cell_type":"markdown","source":"## 14. Runtime summary","metadata":{}},{"id":"0a549c80","cell_type":"code","source":"print(f\"Training time: {train_time_sec:.1f}s\")\nprint(f\"Weak-labeled training studies: {len(weak_labels_df)}\")\nprint(f\"Device: {cfg.DEVICE}\")\nprint(\n    \"Slice-index construction (Section 10) is the main added cost vs. the first baseline, \"\n    \"since it reads DICOM headers for every file in the selected series rather than trusting \"\n    \"filename order. It is run once, not per epoch.\"\n)\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"8d6cfd98","cell_type":"markdown","source":"## 15. Inference on the test set and submission","metadata":{}},{"id":"dff7b59b","cell_type":"code","source":"t_inf = time.time()\ntest_slice_index = build_slice_index(test[\"StudyInstanceUID\"], test_series, cfg.TEST_DICOM_DIR)\nprint(test_slice_index[\"rule\"].value_counts())\n\ntest_ds = KneeMaskedDataset(test_slice_index, cfg.IMG_SIZE, labels_df=None)\ntest_loader = DataLoader(test_ds, batch_size=cfg.BATCH_SIZE, shuffle=False, num_workers=cfg.NUM_WORKERS)\n\nmodel.eval()\ntest_probs, test_ids = [], []\nwith torch.no_grad():\n    for imgs, study_ids in test_loader:\n        imgs = imgs.to(cfg.DEVICE)\n        probs = 1 / (1 + np.exp(-model(imgs).cpu().numpy()))\n        test_probs.append(probs)\n        test_ids.extend(study_ids)\ntest_probs = np.concatenate(test_probs, axis=0)\nprint(f\"Inference time: {time.time() - t_inf:.2f}s for {len(test_ids)} studies\")\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"0fae7699","cell_type":"code","source":"pred_df = pd.DataFrame(test_probs, columns=cfg.TARGET_COLS)\npred_df.insert(0, \"StudyInstanceUID\", test_ids)\nsubmission = test[[\"StudyInstanceUID\"]].merge(pred_df, on=\"StudyInstanceUID\", how=\"left\")\n\nassert list(submission.columns) == list(sample_sub.columns), \"Column mismatch vs sample_submission.csv\"\nassert len(submission) == len(test)\nassert submission[\"StudyInstanceUID\"].is_unique\nassert set(submission[\"StudyInstanceUID\"]) == set(test[\"StudyInstanceUID\"])\nvals = submission[cfg.TARGET_COLS].values\nassert np.isfinite(vals).all() and (vals >= 0).all() and (vals <= 1).all()\nprint(\"All submission validation checks passed.\")\n\nsubmission.to_csv(cfg.OUTPUT_SUBMISSION, index=False)\nprint(f\"Saved {cfg.OUTPUT_SUBMISSION} with shape {submission.shape}\")\nsubmission.head()\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"718807cf","cell_type":"markdown","source":"## 16. Summary and what's next (Stage B)\n\n### What changed and what to expect\n- Training data expanded from 58 gold-labeled studies to a few thousand weakly-labeled\n  studies via report parsing, with an explicit, measured coverage/agreement audit against\n  the gold set (Section 6) rather than blind trust.\n- Fixed the filename-vs-anatomical slice-ordering bug from the first baseline.\n- Reported a macro AUC computed **only** on the untouched 58 gold studies — the most honest\n  number available, comparable across future iterations.\n\n### Still not done (Stage B)\n1. Pretrained image backbone (e.g. DINOv2) attached as an offline Kaggle dataset, partial\n   fine-tuning.\n2. Multi-series / multi-slot aggregation instead of one slice from one series.\n3. Physical-scale-normalized cropping so small structures (e.g. meniscal tears) are not\n   under-sampled across scanners with different pixel spacing.\n4. Laterality normalization (medial/lateral consistent orientation).\n5. Rank-based ensembling across folds/models instead of simple probability averaging.\n6. Expanding/tuning the weak-label lexicon for the lowest-agreement targets identified in\n   Section 6 (e.g. Lateral OA, Synovitis in this run).\n","metadata":{}}]}