\nMedicine. It is a set or sequence of X-ray images that CT scan is comprised of providing details on organ
\nhealth.
\nexamples of DICOM images:
\n
![\"Patient](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16972845%2F7ffb464d82e78e29bca3dafe437a1532%2FDICOM1.png?generation=1699299484698499&alt=media\")
![\"Patient](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16972845%2F47a42e18f96eda654081f04d8eba4cc3%2FDICOM2.png?generation=1699299601100767&alt=media\")
\nPatient 10004 – record 21057 – IMG 1000.dcm, Patient 10004 – record 21057 – IMG 1029.dcm
Dataset provided for the contest consists of 3 main data sources:
\n.dcm stands as an extension for DICOM files, an abbreviation for Digital Imaging and Communications in
\nMedicine. It is a set or sequence of X-ray images that CT scan is comprised of providing details on organ
\nhealth.
\nexamples of DICOM images:
\n
\nPatient 10004 – record 21057 – IMG 1000.dcm, Patient 10004 – record 21057 – IMG 1029.dcm
The other set of data present along with the training set was .ni file. During initial procedures we tried to
\nleverage this data to train model as well while actual model conditions and result deviations are discussed at
\na later part.
\nThe .ni files stand as an extension for Neuroimaging Informatics Technology Initiative, the organisation that
\ncreated .nii file format. It is commonly used to store magnetic resonance imaging (MRI) data.
\nThe MRI contains a 3D representation of full middle body section for analysis of bowel structure. These are
\nparticularly difficult to process to the ML model due to its nature. For this problem, we decided to take an
\nalternate route to the problem by converting the 3D lattice into 3 sets of lateral snapshots each having 2 axes
\nfixed at 0 and 1 available for lateral traversal.
\nTo explain it in simple terms. We changed value of z while keeping x and y at 0. This produced slices parallel
\nto x-y plane at regular intervals in z axis from z = 0 to z = max.
\nA visualisation of this lateral segment can be seen in the screenshot below. The screenshot is captured on a
\nweb-app available to public access via this link
\nexamples of NII files:
\n![\"Patient](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16972845%2Fde92fe3838a9035bcb54aa3b1ec46d33%2FNII_1.jpg?generation=1699299895059718&alt=media\")
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16972845%2Fa1b52b75588e1468665c0f2c5d578695%2FNII_2.jpg?generation=1699300536859629&alt=media\")
The data cleaning step included checking for any missing values and removing the rows altogether in the
\nmetadata of train dataset. Fortunately, we found no missing values. The only value present in the meta data
\nfor each patient was the aortic_hu that was absolute value.
\nFor the purpose of a uniform distribution for the conditional model, we normalised the value of aortic_hu to
\nlimited from 0-1 where 0 indicated lowest observed value and 1 represented highest value observed.
\nThis was done using the mathematical expression:
\nnormalised-aortic-hu = (𝐚𝐨𝐫𝐭𝐢𝐜-𝐡𝐮 − 𝐥𝐨𝐰𝐞𝐬𝐭-𝐚𝐨𝐫𝐭𝐢𝐜-𝐡𝐮𝐞)/(𝐡𝐢𝐠𝐡𝐞𝐬𝐭-𝐚𝐨𝐫𝐭𝐢𝐜-𝐡𝐮𝐞 − 𝐥𝐨𝐰𝐞𝐬𝐭-𝐚𝐨𝐫𝐭𝐢𝐜-𝐡𝐮𝐞)
Redundant Full Black image cleanup
\nThe cleaning operation was primarily to remove the full black images from the nii file generated dataset. This
\nwas particularly difficult when we see that the number of such images in the dataset of nii file output was
\nnot the same. In some MRI images, first 12 images were fully black, in others first 16 were. This created an
\nirregular sized set of images if we proceeded with this tactic. This required an alternate route of first finding
\nthe minimum such count of fully black images and then reduce that size from both sides of an image thus
\nreducing the dataset, although keeping it uniform in all patient MRI images and in an optimised set size
\nreducing irregularity and difficulty for model to process each frame.
\nSo to talk numerically,
\nWe took 100 snapshots per axis of each MRI scan or each NII file.
\nSo, each NII file contributed along x = 100, y = 100, z = 100 => total 300 images.
\nWe found minimum non-full black image at 9th position. i.e. we decided to remove 8 images from each side
\non all axes. So, new dataset => x = 84 (100 - 8 - 8), y = 84, z = 84. => total = 252 images per NII file
\nSo effective dataset reduced by 16% after removal of redundant images.
\nNote: Feature Extraction could not be done well due to inaccurate slice ranges and image rotation in NII files unlike DICOM files.
Redundant Full Black image cleanup
\nInitial Approach of the dicom file processing was like NII file about removing redundant images, but it turned
\nout that nearly all of the images were significant and unskipable. This was great because now we had an
\noption to utilise full dataset and each individual image training was meaningful.
The second exploratory analysis process that we implemented was core feature extraction from images by
\nlocalisation of organs. How exactly? As seen from the image below, the organs are localised to certain
\npositions of CT scan or alternatively DICOM images. We tried to extract specific location of each organ with
\na 20% buffer border around each organ to adjust for any dislocation of organ due to natural causes like
\ngenetics or in body fat layers. This buffer was also to account for the organ movement due to diaphragm
\ncompression and relaxation during breathing.
\nWe tried to localise this subsection of each organ and train it individually for each organ health splitting the
\nDICOM based ML model into its subsections. Although this was successfully executed to separate specific
\norgans from images separately with 20 buffers in each axis (10% on each border), there was still overlap.
\nWe tried to localise this subsection of each organ and train it individually for each organ health splitting the
\nDICOM based ML model into its subsections. Although this was successfully executed to separate specific
\norgans from images separately with 20 buffers in each axis (10% on each border), there was still overlap.
\nIt so happened that the considered organs overlapped over each other’s specific sub-region images. So, an
\nimage for kidney health analysis contained a significant part of spleen as well, so if we went forward with this
\nimplementation, It was possible that damage on spleen could be reflected in damage on kidney by model
\npredictions due to spleen sharing a significant portion in images.
\nThis could be solved by removing the restriction of keeping the image sub-regions to be explicitly a rectangle
\nin shape and keep it as a free body line, though, this would be much difficult to proceed with given that free
\nbody line detection, isolation and processing for this model was quite a difficult task which we could not
\ncomplete in time. Thus, we dropped the idea and added it in the Future possibilities/improvements section
For testing of data on the dataset, we used 2 methods of testing excluding the public dataset-based testing.
\nInitial Approach:
\nAs discussed earlier we planned to utilise all the 3 types of data together. But there was a problem with this
\napproach. The 3 datasets showed a lot of variations. If we were to treat MRI images and CT-SCAN images as
\na single input to the model, we were bound to face issues with training and model accuracy plunging down.
\nTo solve this issue, we decided to take an ensemble model like approach to the model where we would be
\ntreating each dataset separately with their own model and then we would combine the generated result
\nfrom each model with appropriate weights to decide on the best output to be returned as a result.
\nAn illustration of the same can be seen as:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16972845%2F47fae49205d7d037c21eadfb3f126f25%2FMLDA_Model.jpg?generation=1699305429235805&alt=media\")
\nThe data had disparity for each patient and not every patient had both CT scan (aka CAT scan in US) as well
\nas MRI done while diagnostics. In such cases, we simply changed the weights for such patients and distributed the other weights proportionally.
\nFor example:
\nIf the decided ideal weights were: α=0.4, β=0.4, γ=0.2
\nIf a patient A only performed CT-SCAN and not MRI, we could simply set value of β=0 and redistribute α and γ
\nproportionally as α = α/(α+ γ), γ = γ/(α+ γ)
\nAlthough this was our initial plan, we observed that the predicted values by the NII file model had a lot of
\ndiscrepancy. Due to this, the idealistic β value would have been near 0. Thus, to save processing time, we eliminated
To build this model, we have taken help of EfficientNet_B4 prebuilt model.
\nEfficientNet is a convolutional neural network architecture and scaling method that uniformly scales all
\ndimensions of depth/width/resolution using a compound coefficient. Unlike conventional practice that
\narbitrary scales these factors, the EfficientNet scaling method uniformly scales network width, depth, and
\nresolution with a set of fixed scaling coefficients.
\nFor example, if we want to use 2^N times more computational resources, then we can simply increase the
\nnetwork depth by a^N, width by b^N, and image size by c^N, where a,b,c are constant coefficients
\ndetermined by a small grid search on the original small model. EfficientNet uses a compound
\ncoefficient phi to uniformly scales network width, depth, and resolution in a principled way.
\nThe compound scaling method is justified by the intuition that if the input image is bigger, then the network
\nneeds more layers to increase the receptive field and more channels to capture more fine-grained patterns
\non the bigger image. EfficientNets also transfer well and achieve state-of-the-art accuracy on CIFAR-100
\n(91.7%), Flowers (98.8%), and 3 other transfer learning datasets, with an order of magnitude fewer
\nparameters.
model = create_model(, =)\ntorch.save(model.state_dict(), )\nweights_path = \ndef build_model(num_classes):\n model = create_model(, =)\n\n os.path.exists(weights_path):\n model.load_state_dict(torch.load(weights_path, =), =) \n :\n (f)\n\n model.classifier = nn.Linear(model.classifier.in_features, num_classes)\n\n return model\ndevice = torch.device( torch.cuda.is_available() )\nmodel = build_model(len(train_df.columns) - 1).(device)\ncriterion = torch.nn.BCEWithLogitsLoss()\noptimizer = optim.Adam(model.parameters(), =0.001)\nnum_epochs = 3\n epoch range(num_epochs):\n model.train()\n running_loss = 0.0\n i, (inputs, labels) enumerate(train_loader):\n (i)\n inputs, labels = inputs.(device), labels.(device)\n optimizer.zero_grad()\n outputs = model(inputs)\n loss = criterion(outputs, labels)\n loss.backward()\n optimizer.()\n running_loss += loss.item()\n (f)\nPart 2 contains Weighted Baseline Model implementation, Model Reasoning, HyperParameteric Tuning and Results Section. You can find a link to the article here.
\nThank you for reading through.
Dataset provided for the contest consists of 3 main data sources:
\n.dcm stands as an extension for DICOM files, an abbreviation for Digital Imaging and Communications in
\nMedicine. It is a set or sequence of X-ray images that CT scan is comprised of providing details on organ
\nhealth.
\nexamples of DICOM images:
\n
\nPatient 10004 – record 21057 – IMG 1000.dcm, Patient 10004 – record 21057 – IMG 1029.dcm
The other set of data present along with the training set was .ni file. During initial procedures we tried to
\nleverage this data to train model as well while actual model conditions and result deviations are discussed at
\na later part.
\nThe .ni files stand as an extension for Neuroimaging Informatics Technology Initiative, the organisation that
\ncreated .nii file format. It is commonly used to store magnetic resonance imaging (MRI) data.
\nThe MRI contains a 3D representation of full middle body section for analysis of bowel structure. These are
\nparticularly difficult to process to the ML model due to its nature. For this problem, we decided to take an
\nalternate route to the problem by converting the 3D lattice into 3 sets of lateral snapshots each having 2 axes
\nfixed at 0 and 1 available for lateral traversal.
\nTo explain it in simple terms. We changed value of z while keeping x and y at 0. This produced slices parallel
\nto x-y plane at regular intervals in z axis from z = 0 to z = max.
\nA visualisation of this lateral segment can be seen in the screenshot below. The screenshot is captured on a
\nweb-app available to public access via this link
\nexamples of NII files:
\n
The data cleaning step included checking for any missing values and removing the rows altogether in the
\nmetadata of train dataset. Fortunately, we found no missing values. The only value present in the meta data
\nfor each patient was the aortic_hu that was absolute value.
\nFor the purpose of a uniform distribution for the conditional model, we normalised the value of aortic_hu to
\nlimited from 0-1 where 0 indicated lowest observed value and 1 represented highest value observed.
\nThis was done using the mathematical expression:
\nnormalised-aortic-hu = (𝐚𝐨𝐫𝐭𝐢𝐜-𝐡𝐮 − 𝐥𝐨𝐰𝐞𝐬𝐭-𝐚𝐨𝐫𝐭𝐢𝐜-𝐡𝐮𝐞)/(𝐡𝐢𝐠𝐡𝐞𝐬𝐭-𝐚𝐨𝐫𝐭𝐢𝐜-𝐡𝐮𝐞 − 𝐥𝐨𝐰𝐞𝐬𝐭-𝐚𝐨𝐫𝐭𝐢𝐜-𝐡𝐮𝐞)
Redundant Full Black image cleanup
\nThe cleaning operation was primarily to remove the full black images from the nii file generated dataset. This
\nwas particularly difficult when we see that the number of such images in the dataset of nii file output was
\nnot the same. In some MRI images, first 12 images were fully black, in others first 16 were. This created an
\nirregular sized set of images if we proceeded with this tactic. This required an alternate route of first finding
\nthe minimum such count of fully black images and then reduce that size from both sides of an image thus
\nreducing the dataset, although keeping it uniform in all patient MRI images and in an optimised set size
\nreducing irregularity and difficulty for model to process each frame.
\nSo to talk numerically,
\nWe took 100 snapshots per axis of each MRI scan or each NII file.
\nSo, each NII file contributed along x = 100, y = 100, z = 100 => total 300 images.
\nWe found minimum non-full black image at 9th position. i.e. we decided to remove 8 images from each side
\non all axes. So, new dataset => x = 84 (100 - 8 - 8), y = 84, z = 84. => total = 252 images per NII file
\nSo effective dataset reduced by 16% after removal of redundant images.
\nNote: Feature Extraction could not be done well due to inaccurate slice ranges and image rotation in NII files unlike DICOM files.
Redundant Full Black image cleanup
\nInitial Approach of the dicom file processing was like NII file about removing redundant images, but it turned
\nout that nearly all of the images were significant and unskipable. This was great because now we had an
\noption to utilise full dataset and each individual image training was meaningful.
The second exploratory analysis process that we implemented was core feature extraction from images by
\nlocalisation of organs. How exactly? As seen from the image below, the organs are localised to certain
\npositions of CT scan or alternatively DICOM images. We tried to extract specific location of each organ with
\na 20% buffer border around each organ to adjust for any dislocation of organ due to natural causes like
\ngenetics or in body fat layers. This buffer was also to account for the organ movement due to diaphragm
\ncompression and relaxation during breathing.
\nWe tried to localise this subsection of each organ and train it individually for each organ health splitting the
\nDICOM based ML model into its subsections. Although this was successfully executed to separate specific
\norgans from images separately with 20 buffers in each axis (10% on each border), there was still overlap.
\nWe tried to localise this subsection of each organ and train it individually for each organ health splitting the
\nDICOM based ML model into its subsections. Although this was successfully executed to separate specific
\norgans from images separately with 20 buffers in each axis (10% on each border), there was still overlap.
\nIt so happened that the considered organs overlapped over each other’s specific sub-region images. So, an
\nimage for kidney health analysis contained a significant part of spleen as well, so if we went forward with this
\nimplementation, It was possible that damage on spleen could be reflected in damage on kidney by model
\npredictions due to spleen sharing a significant portion in images.
\nThis could be solved by removing the restriction of keeping the image sub-regions to be explicitly a rectangle
\nin shape and keep it as a free body line, though, this would be much difficult to proceed with given that free
\nbody line detection, isolation and processing for this model was quite a difficult task which we could not
\ncomplete in time. Thus, we dropped the idea and added it in the Future possibilities/improvements section
For testing of data on the dataset, we used 2 methods of testing excluding the public dataset-based testing.
\nInitial Approach:
\nAs discussed earlier we planned to utilise all the 3 types of data together. But there was a problem with this
\napproach. The 3 datasets showed a lot of variations. If we were to treat MRI images and CT-SCAN images as
\na single input to the model, we were bound to face issues with training and model accuracy plunging down.
\nTo solve this issue, we decided to take an ensemble model like approach to the model where we would be
\ntreating each dataset separately with their own model and then we would combine the generated result
\nfrom each model with appropriate weights to decide on the best output to be returned as a result.
\nAn illustration of the same can be seen as:
\n
\nThe data had disparity for each patient and not every patient had both CT scan (aka CAT scan in US) as well
\nas MRI done while diagnostics. In such cases, we simply changed the weights for such patients and distributed the other weights proportionally.
\nFor example:
\nIf the decided ideal weights were: α=0.4, β=0.4, γ=0.2
\nIf a patient A only performed CT-SCAN and not MRI, we could simply set value of β=0 and redistribute α and γ
\nproportionally as α = α/(α+ γ), γ = γ/(α+ γ)
\nAlthough this was our initial plan, we observed that the predicted values by the NII file model had a lot of
\ndiscrepancy. Due to this, the idealistic β value would have been near 0. Thus, to save processing time, we eliminated
To build this model, we have taken help of EfficientNet_B4 prebuilt model.
\nEfficientNet is a convolutional neural network architecture and scaling method that uniformly scales all
\ndimensions of depth/width/resolution using a compound coefficient. Unlike conventional practice that
\narbitrary scales these factors, the EfficientNet scaling method uniformly scales network width, depth, and
\nresolution with a set of fixed scaling coefficients.
\nFor example, if we want to use 2^N times more computational resources, then we can simply increase the
\nnetwork depth by a^N, width by b^N, and image size by c^N, where a,b,c are constant coefficients
\ndetermined by a small grid search on the original small model. EfficientNet uses a compound
\ncoefficient phi to uniformly scales network width, depth, and resolution in a principled way.
\nThe compound scaling method is justified by the intuition that if the input image is bigger, then the network
\nneeds more layers to increase the receptive field and more channels to capture more fine-grained patterns
\non the bigger image. EfficientNets also transfer well and achieve state-of-the-art accuracy on CIFAR-100
\n(91.7%), Flowers (98.8%), and 3 other transfer learning datasets, with an order of magnitude fewer
\nparameters.
model = create_model(, =)\ntorch.save(model.state_dict(), )\nweights_path = \ndef build_model(num_classes):\n model = create_model(, =)\n\n os.path.exists(weights_path):\n model.load_state_dict(torch.load(weights_path, =), =) \n :\n (f)\n\n model.classifier = nn.Linear(model.classifier.in_features, num_classes)\n\n return model\ndevice = torch.device( torch.cuda.is_available() )\nmodel = build_model(len(train_df.columns) - 1).(device)\ncriterion = torch.nn.BCEWithLogitsLoss()\noptimizer = optim.Adam(model.parameters(), =0.001)\nnum_epochs = 3\n epoch range(num_epochs):\n model.train()\n running_loss = 0.0\n i, (inputs, labels) enumerate(train_loader):\n (i)\n inputs, labels = inputs.(device), labels.(device)\n optimizer.zero_grad()\n outputs = model(inputs)\n loss = criterion(outputs, labels)\n loss.backward()\n optimizer.()\n running_loss += loss.item()\n (f)\nPart 2 contains Weighted Baseline Model implementation, Model Reasoning, HyperParameteric Tuning and Results Section. You can find a link to the article here.
\nThank you for reading through.