I have seen a few comments on this discussion board about various difficulties in loading DICOM data into 3D image arrays. I wanted to provide my approach for this in case it is helpful for someone. However, important to acknowledge that there are many ways to do this, and mine is not necessarily the best.
\nDICOM is a complex format with many different ways of structuring the underlying imaging data. In my opinion, its a terrible format for data analysis; however, it is the reality of current clinical practice, which is why we chose to provide DICOM data instead of a more data analysis friendly format.
\nWhen I am trying to process DICOM image data, the first thing I do is convert to NIfTI format using dcm2niix. This is publicly available software (https://github.com/rordenlab/dcm2niix) that automates the process of DICOM to NIfTI conversion including graceful handling of a large majority of edge cases. The dcm2niix binary can be easily interfaced/pipelined with nipype see docs here. There is also a pip package available (https://pypi.org/project/dcm2niix/), though I have limited experience with this.
\nWhen using dcm2niix, the key conversion flags that you will want to consider are:
\n-b, which allows you to output DICOM header variables as JSON alongside your images
\n-i, which allows you to ignore stacked derived/localizer images (see this thread)
\n-z, which allows you gzip outputs to save considerable disk space
\nThe main benefit of dcm2niix is that it completely automates the otherwise complex process of extracting 3D numpy arrays from DICOM datasets. While it lacks the customizability of approaches using pydicom, it is much simpler, particularly for those without prior DICOM experience.
NIfTI is a much simpler image format that can handle a large majority of cross-sectional medical imaging data. It is also extremely easy to load and manipulate using python and other image analysis softwares. nibabel is the essential python package for NIfTI file manipulation, and loading is a simple as nibabel.load('/path/to/file.nii.gz').
I have seen a few comments on this discussion board about various difficulties in loading DICOM data into 3D image arrays. I wanted to provide my approach for this in case it is helpful for someone. However, important to acknowledge that there are many ways to do this, and mine is not necessarily the best.
\nDICOM is a complex format with many different ways of structuring the underlying imaging data. In my opinion, its a terrible format for data analysis; however, it is the reality of current clinical practice, which is why we chose to provide DICOM data instead of a more data analysis friendly format.
\nWhen I am trying to process DICOM image data, the first thing I do is convert to NIfTI format using dcm2niix. This is publicly available software (https://github.com/rordenlab/dcm2niix) that automates the process of DICOM to NIfTI conversion including graceful handling of a large majority of edge cases. The dcm2niix binary can be easily interfaced/pipelined with nipype see docs here. There is also a pip package available (https://pypi.org/project/dcm2niix/), though I have limited experience with this.
\nWhen using dcm2niix, the key conversion flags that you will want to consider are:
\n-b, which allows you to output DICOM header variables as JSON alongside your images
\n-i, which allows you to ignore stacked derived/localizer images (see this thread)
\n-z, which allows you gzip outputs to save considerable disk space
\nThe main benefit of dcm2niix is that it completely automates the otherwise complex process of extracting 3D numpy arrays from DICOM datasets. While it lacks the customizability of approaches using pydicom, it is much simpler, particularly for those without prior DICOM experience.
NIfTI is a much simpler image format that can handle a large majority of cross-sectional medical imaging data. It is also extremely easy to load and manipulate using python and other image analysis softwares. nibabel is the essential python package for NIfTI file manipulation, and loading is a simple as nibabel.load('/path/to/file.nii.gz').
Here’s a compact field-note I compiled from our work (joint with a ChatGPT-5 ); since this post is published under my name, a heads-up that most of the hands-on digging and write-up here is on my side. Hope it saves someone a few days. Wy fight we converting all dcms to NIfTI is about 5 days. But still in progess.
\nDICOM → NIfTI for brain aneurysm data: pitfalls & fixes
\nBaseline tools: we convert with dcm2niix and keep the JSON sidecar (-b y), gzip outputs (-z y). When needed, we first “normalize” raw DICOM with gdcmconv --raw and then run dcm2niix.
The traps we hit (and what worked)
\nMixed slice sizes / localizers in one folder
\nSymptoms: a series folder has scouts and the actual scan → different Rows/Cols.
\nFix: pre-audit and drop files whose matrix ≠ the mode; sort by InstanceNumber. Log the UID.
Enhanced/multiframe DICOM & hidden spacing
\nPixel spacing lives in FunctionalGroups (0028,9110); naïve pydicom reads miss it.
\nFix: prefer dcm2niix (handles this). If you must roll your own, parse functional groups.
DERIVED / MPR vs “ignore derived”
\n-i y can throw away everything; -i n lets MPRs through.
\nFix: run with -i n, then filter post-hoc: keep isotropic, long volumes (e.g., Nz≥100, dz≤1.2 mm) even if DERIVED; send true slabs to a 2D/2.5D branch.
RescaleSlope/Intercept after anonymization
\nBad slope/intercept → “grey rectangle” CTs.
\nFix: don’t pre-apply rescale yourself; let dcm2niix write scl_slope/inter into NIfTI. Apply HU on load only when slope/intercept ≠ identity; sanity-check with robust window percentiles.
Ovals instead of circles (affine / spacing mix-ups)
\nFix: trust dcm2niix’s affine (from IOP/IPP). Verify dx≈dy and diagonal signs (LPS→RAS). If a head looks squashed, check pixdim first, not your viewer.
RLE / uncommon transfer syntaxes
\nDecoder errors, “wraps”, corrupted offsets.
\nFix: gdcmconv --raw to a temp folder → then dcm2niix on the unpacked files.
Throughput & I/O pain (millions of files)
\nFix: parallelize (16–20 workers), write NIfTI with -z y, mount a tmpfs for intermediates, and write outputs to a separate fast disk.
Quality control & quarantine
\nFix: after conversion, compute per-volume: Nx,Ny,Nz,dx,dy,dz, intensity stats, and flags like derived_only, json_missing, int16_fullspan, slab. Keep a meta_summary.csv and suspects.csv; maintain a blacklist for truly broken cases.
Brain masking (less junk, faster training)
\nFix: run SynthStrip; zero out outside the mask but dilate 3–5 vox and give 10–15 mm padding to avoid cutting cortex/foramen magnum. If SynthStrip fails, fallback = ellipse in XY ∪ largest head component (for CT via HU threshold). QC: brain_frac, brain_vol_ml, touch_border.
Minimal flag cheat-sheet
\nDefault: dcm2niix -z y -b y -i n -o -f %s
If decoder complains: gdcmconv --raw /*.dcm // && dcm2niix … //
\nTakeaways
\ndcm2niix handles 90% of the weirdness; the rest is audit + simple rules.
Always keep the JSON sidecar—it prevents guesswork later.
", "rawMarkdown": "##Experience with converting\n\nHere’s a compact field-note I compiled from our work (joint with a ChatGPT-5 ); since this post is published under my name, a heads-up that most of the hands-on digging and write-up here is on my side. Hope it saves someone a few days. Wy fight we converting all dcms to NIfTI is about 5 days. But still in progess.\n\n\nDICOM → NIfTI for brain aneurysm data: pitfalls & fixes\nBaseline tools: we convert with dcm2niix and keep the JSON sidecar (-b y), gzip outputs (-z y). When needed, we first “normalize” raw DICOM with gdcmconv --raw and then run dcm2niix.\n\nThe traps we hit (and what worked)\nMixed slice sizes / localizers in one folder\nSymptoms: a series folder has scouts and the actual scan → different Rows/Cols.\nFix: pre-audit and drop files whose matrix ≠ the mode; sort by InstanceNumber. Log the UID.\n\nEnhanced/multiframe DICOM & hidden spacing\nPixel spacing lives in FunctionalGroups (0028,9110); naïve pydicom reads miss it.\nFix: prefer dcm2niix (handles this). If you must roll your own, parse functional groups.\n\nDERIVED / MPR vs “ignore derived”\n-i y can throw away everything; -i n lets MPRs through.\nFix: run with -i n, then filter post-hoc: keep isotropic, long volumes (e.g., Nz≥100, dz≤1.2 mm) even if DERIVED; send true slabs to a 2D/2.5D branch.\n\nRescaleSlope/Intercept after anonymization\nBad slope/intercept → “grey rectangle” CTs.\nFix: don’t pre-apply rescale yourself; let dcm2niix write scl_slope/inter into NIfTI. Apply HU on load only when slope/intercept ≠ identity; sanity-check with robust window percentiles.\n\nOvals instead of circles (affine / spacing mix-ups)\nFix: trust dcm2niix’s affine (from IOP/IPP). Verify dx≈dy and diagonal signs (LPS→RAS). If a head looks squashed, check pixdim first, not your viewer.\n\nRLE / uncommon transfer syntaxes\nDecoder errors, “wraps”, corrupted offsets.\nFix: gdcmconv --raw to a temp folder → then dcm2niix on the unpacked files.\n\nThroughput & I/O pain (millions of files)\nFix: parallelize (16–20 workers), write NIfTI with -z y, mount a tmpfs for intermediates, and write outputs to a separate fast disk.\n\nQuality control & quarantine\nFix: after conversion, compute per-volume: Nx,Ny,Nz,dx,dy,dz, intensity stats, and flags like derived_only, json_missing, int16_fullspan, slab. Keep a meta_summary.csv and suspects.csv; maintain a blacklist for truly broken cases.\n\nBrain masking (less junk, faster training)\nFix: run SynthStrip; zero out outside the mask but dilate 3–5 vox and give 10–15 mm padding to avoid cutting cortex/foramen magnum. If SynthStrip fails, fallback = ellipse in XY ∪ largest head component (for CT via HU threshold). QC: brain_frac, brain_vol_ml, touch_border.\n\nMinimal flag cheat-sheet\nDefault: dcm2niix -z y -b y -i n -oGreat advice!
", "rawMarkdown": "Great advice!" } ] } ], "comments": [ { "id": 3266814, "author_name": "Anton Sibilev", "author_url": "", "post_date": "2025-08-09T18:18:18.893000", "content": "Here’s a compact field-note I compiled from our work (joint with a ChatGPT-5 ); since this post is published under my name, a heads-up that most of the hands-on digging and write-up here is on my side. Hope it saves someone a few days. Wy fight we converting all dcms to NIfTI is about 5 days. But still in progess.
\nDICOM → NIfTI for brain aneurysm data: pitfalls & fixes
\nBaseline tools: we convert with dcm2niix and keep the JSON sidecar (-b y), gzip outputs (-z y). When needed, we first “normalize” raw DICOM with gdcmconv --raw and then run dcm2niix.
The traps we hit (and what worked)
\nMixed slice sizes / localizers in one folder
\nSymptoms: a series folder has scouts and the actual scan → different Rows/Cols.
\nFix: pre-audit and drop files whose matrix ≠ the mode; sort by InstanceNumber. Log the UID.
Enhanced/multiframe DICOM & hidden spacing
\nPixel spacing lives in FunctionalGroups (0028,9110); naïve pydicom reads miss it.
\nFix: prefer dcm2niix (handles this). If you must roll your own, parse functional groups.
DERIVED / MPR vs “ignore derived”
\n-i y can throw away everything; -i n lets MPRs through.
\nFix: run with -i n, then filter post-hoc: keep isotropic, long volumes (e.g., Nz≥100, dz≤1.2 mm) even if DERIVED; send true slabs to a 2D/2.5D branch.
RescaleSlope/Intercept after anonymization
\nBad slope/intercept → “grey rectangle” CTs.
\nFix: don’t pre-apply rescale yourself; let dcm2niix write scl_slope/inter into NIfTI. Apply HU on load only when slope/intercept ≠ identity; sanity-check with robust window percentiles.
Ovals instead of circles (affine / spacing mix-ups)
\nFix: trust dcm2niix’s affine (from IOP/IPP). Verify dx≈dy and diagonal signs (LPS→RAS). If a head looks squashed, check pixdim first, not your viewer.
RLE / uncommon transfer syntaxes
\nDecoder errors, “wraps”, corrupted offsets.
\nFix: gdcmconv --raw to a temp folder → then dcm2niix on the unpacked files.
Throughput & I/O pain (millions of files)
\nFix: parallelize (16–20 workers), write NIfTI with -z y, mount a tmpfs for intermediates, and write outputs to a separate fast disk.
Quality control & quarantine
\nFix: after conversion, compute per-volume: Nx,Ny,Nz,dx,dy,dz, intensity stats, and flags like derived_only, json_missing, int16_fullspan, slab. Keep a meta_summary.csv and suspects.csv; maintain a blacklist for truly broken cases.
Brain masking (less junk, faster training)
\nFix: run SynthStrip; zero out outside the mask but dilate 3–5 vox and give 10–15 mm padding to avoid cutting cortex/foramen magnum. If SynthStrip fails, fallback = ellipse in XY ∪ largest head component (for CT via HU threshold). QC: brain_frac, brain_vol_ml, touch_border.
Minimal flag cheat-sheet
\nDefault: dcm2niix -z y -b y -i n -o -f %s
If decoder complains: gdcmconv --raw /*.dcm // && dcm2niix … //
\nTakeaways
\ndcm2niix handles 90% of the weirdness; the rest is audit + simple rules.
Always keep the JSON sidecar—it prevents guesswork later.
", "votes": 6, "replies": [ { "id": 3267313, "author_name": "Evan Calabrese", "author_url": "", "post_date": "2025-08-10T22:21:27.733000", "content": "Great advice!
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "3266199": "I have seen a few comments on this discussion board about various difficulties in loading DICOM data into 3D image arrays. I wanted to provide my approach for this in case it is helpful for someone. However, important to acknowledge that there are many ways to do this, and mine is not necessarily the best.\n\nDICOM is a complex format with many different ways of structuring the underlying imaging data. In my opinion, its a terrible format for data analysis; however, it is the reality of current clinical practice, which is why we chose to provide DICOM data instead of a more data analysis friendly format.\n\nWhen I am trying to process DICOM image data, the first thing I do is convert to NIfTI format using ```dcm2niix```. This is publicly available software (https://github.com/rordenlab/dcm2niix) that automates the process of DICOM to NIfTI conversion including graceful handling of a large majority of edge cases. The ```dcm2niix``` binary can be easily interfaced/pipelined with ```nipype``` see [docs here](https://nipype.readthedocs.io/en/latest/api/generated/nipype.interfaces.dcm2nii.html#dcm2niix). There is also a pip package available (https://pypi.org/project/dcm2niix/), though I have limited experience with this. \nWhen using ```dcm2niix```, the key conversion flags that you will want to consider are:\n```-b```, which allows you to output DICOM header variables as JSON alongside your images\n```-i```, which allows you to ignore stacked derived/localizer images (see [this thread](https://www.kaggle.com/competitions/rsna-intracranial-aneurysm-detection/discussion/596183))\n```-z```, which allows you gzip outputs to save considerable disk space\nThe main benefit of ```dcm2niix``` is that it completely automates the otherwise complex process of extracting 3D ```numpy``` arrays from DICOM datasets. While it lacks the customizability of approaches using ```pydicom```, it is much simpler, particularly for those without prior DICOM experience.\n\nNIfTI is a much simpler image format that can handle a large majority of cross-sectional medical imaging data. It is also extremely easy to load and manipulate using python and other image analysis softwares. ```nibabel``` is the essential python package for NIfTI file manipulation, and loading is a simple as ```nibabel.load('/path/to/file.nii.gz')```.", "3266814": "##Experience with converting\n\nHere’s a compact field-note I compiled from our work (joint with a ChatGPT-5 ); since this post is published under my name, a heads-up that most of the hands-on digging and write-up here is on my side. Hope it saves someone a few days. Wy fight we converting all dcms to NIfTI is about 5 days. But still in progess.\n\n\nDICOM → NIfTI for brain aneurysm data: pitfalls & fixes\nBaseline tools: we convert with dcm2niix and keep the JSON sidecar (-b y), gzip outputs (-z y). When needed, we first “normalize” raw DICOM with gdcmconv --raw and then run dcm2niix.\n\nThe traps we hit (and what worked)\nMixed slice sizes / localizers in one folder\nSymptoms: a series folder has scouts and the actual scan → different Rows/Cols.\nFix: pre-audit and drop files whose matrix ≠ the mode; sort by InstanceNumber. Log the UID.\n\nEnhanced/multiframe DICOM & hidden spacing\nPixel spacing lives in FunctionalGroups (0028,9110); naïve pydicom reads miss it.\nFix: prefer dcm2niix (handles this). If you must roll your own, parse functional groups.\n\nDERIVED / MPR vs “ignore derived”\n-i y can throw away everything; -i n lets MPRs through.\nFix: run with -i n, then filter post-hoc: keep isotropic, long volumes (e.g., Nz≥100, dz≤1.2 mm) even if DERIVED; send true slabs to a 2D/2.5D branch.\n\nRescaleSlope/Intercept after anonymization\nBad slope/intercept → “grey rectangle” CTs.\nFix: don’t pre-apply rescale yourself; let dcm2niix write scl_slope/inter into NIfTI. Apply HU on load only when slope/intercept ≠ identity; sanity-check with robust window percentiles.\n\nOvals instead of circles (affine / spacing mix-ups)\nFix: trust dcm2niix’s affine (from IOP/IPP). Verify dx≈dy and diagonal signs (LPS→RAS). If a head looks squashed, check pixdim first, not your viewer.\n\nRLE / uncommon transfer syntaxes\nDecoder errors, “wraps”, corrupted offsets.\nFix: gdcmconv --raw to a temp folder → then dcm2niix on the unpacked files.\n\nThroughput & I/O pain (millions of files)\nFix: parallelize (16–20 workers), write NIfTI with -z y, mount a tmpfs for intermediates, and write outputs to a separate fast disk.\n\nQuality control & quarantine\nFix: after conversion, compute per-volume: Nx,Ny,Nz,dx,dy,dz, intensity stats, and flags like derived_only, json_missing, int16_fullspan, slab. Keep a meta_summary.csv and suspects.csv; maintain a blacklist for truly broken cases.\n\nBrain masking (less junk, faster training)\nFix: run SynthStrip; zero out outside the mask but dilate 3–5 vox and give 10–15 mm padding to avoid cutting cortex/foramen magnum. If SynthStrip fails, fallback = ellipse in XY ∪ largest head component (for CT via HU threshold). QC: brain_frac, brain_vol_ml, touch_border.\n\nMinimal flag cheat-sheet\nDefault: dcm2niix -z y -b y -i n -o