{ "id": 420629, "title": "One month to go! Summary of everything that happened.", "url": "/competitions/google-research-identify-contrails-reduce-global-warming/discussion/420629", "author_name": "The Devastator", "post_date": "2023-07-01T18:42:56.238000", "votes": 110, "comment_count": 2, "views": 0, "content": "

One month to go: Summary of everything that happened

\n

Happy last month to everyone!
\nA good time to take a look back and summarize everything we know so far.

\n

Starting from the basics: In this competition we are tasked with detecting contrails in satelite images. (segmentation)

\n

Evaluation

\n

As for the metric, we are being evaluated using Dice coefficient.

\n
\n

Dice Coefficient

\n

$$\\frac{2 * |X \\cap Y|}{|X| + |Y|}$$

\n

Torch

\n
def dice_coefficient(y_true, y_pred,  = 1e-6):\n     (2. * (y_true.(-1) * y_pred.(-1)).() + ) / ((y_true.(-1).() + y_pred.(-1).()) + )\n
\n

Tensorflow

\n
def dice:\n    y_true_f = flatten(y_true)\n    y_pred_f = flatten(y_pred)\n    intersection = tf.reduce\n    return (intersection + smooth)(tf.reduce + tf.reduce + smooth)\n
\n

Numpy

\n
def dice_coefficient(y_true, y_pred, smooth=):\n   y_true_f = y_true.()\n   y_pred_f = y_pred.()\n    = .(y_true_f * y_pred_f)\n    (. *  + smooth) / (.(y_true_f) + .(y_pred_f) + smooth)\n
\n
\n
\n

Large Dataset

\n\n
\n

Previous Competitions Winning Solutions By Dewei Chen

\n\n
\n\n
\n\n
\n

Questions

\n
\n\n

Answer (By Ericka42)

\n
\n

In the preprint, the Advected Flight Density is shown to labelers as a heat map with a color scale ranging from blue to red. The blue color represents low flight density, while the red color represents high flight density.
\n To replicate the reddish false color scheme shown in Fig 3, you could try adjusting the color scale in your heatmap visualization to highlight the higher density regions with a reddish hue. You could also experiment with different color palettes and custom color schemes to find what works best for your specific use case.

\n
\n
\n\n

Pointing out that out of the first 120 entries (ordered by recordId), there are 49 with empty masks.

\n

Answer (From the host):

\n
\n

What you're seeing is expected. Most scenes do not contain any contrails, so most masks are all 0's. The distribution of empty masks should roughly reflect what is seen in the real world for the space-time region that the dataset covers. The goal is to train a model that can eventually be run on an arbitrary scene from a satellite image and identify contrails if they are there, even though most of the time there won't be any. You're welcome to try out different training approaches that might involve upweighting the samples that do have contrails or subsampling the ones that don't.
\n It's also expected that the combined mask can be all 0's in cases where individual masks are not all 0's. This is because the combined mask is a majority vote per-pixel of all the individual masks.

\n
\n
\n\n

Answer (By Patchef):

\n
\n

As mentioned in the data section:
\n \"human_pixel_masks.npy: array with size of H x W x 1 x R. Each example is labeled by R individual human labelers. R is not the same for all samples. The labeled masks have value either 0 or 1 and correspond to the (n_times_before+1)-th image in band_{08-16}.npy. They are available only in the training set.\"
\n It corresponds to the time stamp (n_times_before+1)

\n
\n

This is an interesting discussion overall, I suggest reading it.

\n
\n\n

Answer (from the host):

\n
\n

If you look at the metric definition at https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/overview/evaluation, you see that the numerator is the size of the intersection of X and Y, where X is the set of predicted contrail pixels and Y is the set of groundtruth contrail pixels. In your case X is the empty set, so the size of the intersection will be 0.

\n
\n
\n

Means and Standard Deviation for Normalization By Soumyadeep Khandual

\n

Sharing with us normaliztion values: since we are working on 9 infrared channel at different wavelengths which are converted to brightness temperatures, we cannot simply divide each pixel intensity by 255.

\n

These are the mean and std of train dataset for the 9 channels which can be used to normalize images:

\n
Mean = [, , , , , , , , ]\nStd = [ .,  ., ., ., ., ., ., ., .]\n
\n

And the 9 channels correspond to band_08 to band_16. The code i used is given below:

\n
 = torch.zeros()\n = torch.zeros()\n\n data, _ in tqdm(train_loader):\n     += torch.sum(data, axis=[,,,]) \n     += torch.sum(data**, axis=[,,,])\n\n = len(train_loader)*train_loader.batch_size***\n = total_sum / total_count\n = total_sum_of_sq/total_count\n = torch.sqrt(total_mean_of_sq - total_mean**)\n
\n
\n\n

\"\"
\n\"\"

\n
\n

Preprocessed Data By Kenni

\n

For people struggling to preprocess their data, Kenni have enclosed a dataset using false image generation here.

\n
\n\n

In this post, it is reported that by reducing the model's confidence in contrail predictions, it is highly likely that the Dice coeff will improve.

\n
k = - \n = F.softmax(, =)\n[:, , :, :] += k  # reducing confidence  contrail predictions\n = F.softmax(, =)\n = torch.argmax(, =)\n
\n

The model in this thread was trained with WCE loss function and its performance is suboptimal (~0.309), But the author suspect that this could be helpful for a much better model or a model trained using DICE loss.

\n

\"\"

\n
\n\n

Since there are many failed submission, on this post there is a ref notebook for helping others getting started with this notebook.

\n
\n\n

In this post, LUPIN11 reported experimental results from two different loss functions: WCE: Weighted Cross Entropy and Dice Loss.

\n

In this experiment, WCE performed better.

\n
def ce:\n    weight = torch..('cuda')\n    criterion = nn.\n    loss = criterion(y_p, y_t)\n    return loss\n
\n
def dice_loss(y_p, y_t, =1e-6):\n    y_p = y_p.(-1)\n    y_t = y_t.(-1)\n    i = (y_p * y_t).()\n     1 - (2. * i + ) / (y_p.() + y_t.() + )\n
\n

This post also go in depth into the reasons this might be the case. Interesting!

\n
\n\n

In this post, sergey shared with us a simple trick to overcome memory issues in this competition (since the data is so large). The trick is to send everything to CUDA.

\n
 = torch.cat(get_predictions(ckpt_exp6_path, test_loader6)).cuda()\n = torch.cat(get_predictions(ckpt_exp8_path, test_loader8)).cuda()\n = test_preds6* + test_preds8*\n
\n
\n\n

Sharing with us a Unet Pytorch Baseline:

\n

Training

\n

Submission

\n
\n\n

Library: Smp
\nData: Ash Color images (With only the labeled frames and human_pixel_masks)
\nBackbone: EfficientNet-B0
\nPostprocessing: Finding the best threshold

\n
\n\n

Plotted all ASHRGB images and their corresponding mask in both the train and validation set.

\n

here

\n

\"\"

\n
\n\n

In this competition, finding the optimal threshold is a crucial step in improving the score.

\n

So in this post, a dual threshold is proposed and improved the authors model's score from 0.55 to 0.552.

\n
\n\n

Q1. What is the correct value for \"n_times_before\"?

\n\n

Q2. What is the correct false-color formula?

\n\n
\n

r = normalize_range(band15 - band14, _TDIFF_BOUNDS)
\n g = normalize_range(band14 - band11, _CLOUD_TOP_TDIFF_BOUNDS)
\n b = normalize_range(band14, _T11_BOUNDS)

\n
\n

In other words:

\n\n

Answers (from the host):

\n
\n

For Q2. I believe that the description in the data section of this competition is correct and the preprint is incorrect. We'll fix this in the next draft of the preprint.
\n For Q1, I'll have to confirm with coauthors, but it is possible that both are correct. Since the labelers only labeled a single frame, it's possible that we showed them 5 frames before and 2 after, but for this competition we provide 4 frames before and 3 after. I don't know what would have driven them to be different, but they could both be correct.

\n
\n
\n\n

Proposed a method to load numpy datasets much faster:

\n
class fastnumpyio:\n    def ():\n        =(,)\n        header = .()\n        descr = str(header[:], ).(,).(,)\n        shape = tuple(int()    str(header[:], ).(, ).(, ).(, ).())\n        datasize = np.lib..descr_to_dtype(descr).itemsize\n         dimension  shape:\n            datasize *= dimension\n         np.ndarray(shape, dtype=descr, buffer=.(datasize))\n
\n
\n\n
\n

Spoiler: It doesn't use BatchNorm

\n
\n

In this post, J€ANMPIA proposes to use batch norm to improve unet models.

\n

Code example:

\n
def double_conv(in_channels, out_channels):\n    return nn.Sequential(\n        nn.Conv2d(in_channels, out_channels, 3, =1, =), \n        nn.BatchNorm2d(out_channels),\n        nn.ReLU(=),\n        nn.Conv2d(out_channels, out_channels, 3, =1, =), \n        nn.BatchNorm2d(out_channels),\n        nn.ReLU(=)\n    )   \n
\n
\n
\n

This is true. It usually always help (but you still have to test it out to make sure!)

\n
\n\n

Egor Trushin shared with us a powerful pytorch lightning pipeline:

\n\n
\n\n

Stressing that we have to pay attention to post-processing and it's impact on the score because improving average dice score per image is not the same as improving global dice score.

\n

Some Ideas from this post:

\n\n

Some comments also tried the following:

\n\n
\n\n

In many CV tasks, bigger image size often gives us higher performance, But for the author, on this competition: bigger image size got higher score on CV but lower score on LB.

\n
\n

Serveral comments reported the same.

\n
\n
\n\n
\n\n

Pointing out that there are duplicates on the dataset.

\n

Might be the same place but different day/time.

\n
\n\n

Starting a discussion about 3D models and asking the other competitors if they had any luck with such models.

\n\n
\n

Suggested read!

\n
\n
\n\n

Reporting that so far, Randomresizedcrop alone gives improvement.

\n
\n

Martin Kovacevic Buvinic Tried flips and rotation augmentations with no luck.

\n
\n
\n\n

Highly Suggested!!

\n

A very good breakdown of many attempts and experimental results!

\n

A must read for everyone on this competition.

\n
\n\n

Again: Highly Suggested!!

\n

Again: A very good breakdown of the whole competition. A must read in my opinion.

\n
\n

This is it for now.
\nI will keep updating this post when new information is discovered.

\n

Good luck on the last month!

\n
\n
\n

This is not ChatGPT. I actually go by hand and read everything. So do call me out if anything I wrote here is inaccurate because I want to know for myself 🙃

\n
", "messages": [ { "id": 2325989, "postDate": "2023-07-01T18:42:56.237Z", "content": "

One month to go: Summary of everything that happened

\n

Happy last month to everyone!
\nA good time to take a look back and summarize everything we know so far.

\n

Starting from the basics: In this competition we are tasked with detecting contrails in satelite images. (segmentation)

\n

Evaluation

\n

As for the metric, we are being evaluated using Dice coefficient.

\n
\n

Dice Coefficient

\n

$$\\frac{2 * |X \\cap Y|}{|X| + |Y|}$$

\n

Torch

\n
def dice_coefficient(y_true, y_pred,  = 1e-6):\n     (2. * (y_true.(-1) * y_pred.(-1)).() + ) / ((y_true.(-1).() + y_pred.(-1).()) + )\n
\n

Tensorflow

\n
def dice:\n    y_true_f = flatten(y_true)\n    y_pred_f = flatten(y_pred)\n    intersection = tf.reduce\n    return (intersection + smooth)(tf.reduce + tf.reduce + smooth)\n
\n

Numpy

\n
def dice_coefficient(y_true, y_pred, smooth=):\n   y_true_f = y_true.()\n   y_pred_f = y_pred.()\n    = .(y_true_f * y_pred_f)\n    (. *  + smooth) / (.(y_true_f) + .(y_pred_f) + smooth)\n
\n
\n
\n

Large Dataset

\n\n
\n

Previous Competitions Winning Solutions By Dewei Chen

\n\n
\n\n
\n\n
\n

Questions

\n
\n\n

Answer (By Ericka42)

\n
\n

In the preprint, the Advected Flight Density is shown to labelers as a heat map with a color scale ranging from blue to red. The blue color represents low flight density, while the red color represents high flight density.
\n To replicate the reddish false color scheme shown in Fig 3, you could try adjusting the color scale in your heatmap visualization to highlight the higher density regions with a reddish hue. You could also experiment with different color palettes and custom color schemes to find what works best for your specific use case.

\n
\n
\n\n

Pointing out that out of the first 120 entries (ordered by recordId), there are 49 with empty masks.

\n

Answer (From the host):

\n
\n

What you're seeing is expected. Most scenes do not contain any contrails, so most masks are all 0's. The distribution of empty masks should roughly reflect what is seen in the real world for the space-time region that the dataset covers. The goal is to train a model that can eventually be run on an arbitrary scene from a satellite image and identify contrails if they are there, even though most of the time there won't be any. You're welcome to try out different training approaches that might involve upweighting the samples that do have contrails or subsampling the ones that don't.
\n It's also expected that the combined mask can be all 0's in cases where individual masks are not all 0's. This is because the combined mask is a majority vote per-pixel of all the individual masks.

\n
\n
\n\n

Answer (By Patchef):

\n
\n

As mentioned in the data section:
\n \"human_pixel_masks.npy: array with size of H x W x 1 x R. Each example is labeled by R individual human labelers. R is not the same for all samples. The labeled masks have value either 0 or 1 and correspond to the (n_times_before+1)-th image in band_{08-16}.npy. They are available only in the training set.\"
\n It corresponds to the time stamp (n_times_before+1)

\n
\n

This is an interesting discussion overall, I suggest reading it.

\n
\n\n

Answer (from the host):

\n
\n

If you look at the metric definition at https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/overview/evaluation, you see that the numerator is the size of the intersection of X and Y, where X is the set of predicted contrail pixels and Y is the set of groundtruth contrail pixels. In your case X is the empty set, so the size of the intersection will be 0.

\n
\n
\n

Means and Standard Deviation for Normalization By Soumyadeep Khandual

\n

Sharing with us normaliztion values: since we are working on 9 infrared channel at different wavelengths which are converted to brightness temperatures, we cannot simply divide each pixel intensity by 255.

\n

These are the mean and std of train dataset for the 9 channels which can be used to normalize images:

\n
Mean = [, , , , , , , , ]\nStd = [ .,  ., ., ., ., ., ., ., .]\n
\n

And the 9 channels correspond to band_08 to band_16. The code i used is given below:

\n
 = torch.zeros()\n = torch.zeros()\n\n data, _ in tqdm(train_loader):\n     += torch.sum(data, axis=[,,,]) \n     += torch.sum(data**, axis=[,,,])\n\n = len(train_loader)*train_loader.batch_size***\n = total_sum / total_count\n = total_sum_of_sq/total_count\n = torch.sqrt(total_mean_of_sq - total_mean**)\n
\n
\n\n

\"\"
\n\"\"

\n
\n

Preprocessed Data By Kenni

\n

For people struggling to preprocess their data, Kenni have enclosed a dataset using false image generation here.

\n
\n\n

In this post, it is reported that by reducing the model's confidence in contrail predictions, it is highly likely that the Dice coeff will improve.

\n
k = - \n = F.softmax(, =)\n[:, , :, :] += k  # reducing confidence  contrail predictions\n = F.softmax(, =)\n = torch.argmax(, =)\n
\n

The model in this thread was trained with WCE loss function and its performance is suboptimal (~0.309), But the author suspect that this could be helpful for a much better model or a model trained using DICE loss.

\n

\"\"

\n
\n\n

Since there are many failed submission, on this post there is a ref notebook for helping others getting started with this notebook.

\n
\n\n

In this post, LUPIN11 reported experimental results from two different loss functions: WCE: Weighted Cross Entropy and Dice Loss.

\n

In this experiment, WCE performed better.

\n
def ce:\n    weight = torch..('cuda')\n    criterion = nn.\n    loss = criterion(y_p, y_t)\n    return loss\n
\n
def dice_loss(y_p, y_t, =1e-6):\n    y_p = y_p.(-1)\n    y_t = y_t.(-1)\n    i = (y_p * y_t).()\n     1 - (2. * i + ) / (y_p.() + y_t.() + )\n
\n

This post also go in depth into the reasons this might be the case. Interesting!

\n
\n\n

In this post, sergey shared with us a simple trick to overcome memory issues in this competition (since the data is so large). The trick is to send everything to CUDA.

\n
 = torch.cat(get_predictions(ckpt_exp6_path, test_loader6)).cuda()\n = torch.cat(get_predictions(ckpt_exp8_path, test_loader8)).cuda()\n = test_preds6* + test_preds8*\n
\n
\n\n

Sharing with us a Unet Pytorch Baseline:

\n

Training

\n

Submission

\n
\n\n

Library: Smp
\nData: Ash Color images (With only the labeled frames and human_pixel_masks)
\nBackbone: EfficientNet-B0
\nPostprocessing: Finding the best threshold

\n
\n\n

Plotted all ASHRGB images and their corresponding mask in both the train and validation set.

\n

here

\n

\"\"

\n
\n\n

In this competition, finding the optimal threshold is a crucial step in improving the score.

\n

So in this post, a dual threshold is proposed and improved the authors model's score from 0.55 to 0.552.

\n
\n\n

Q1. What is the correct value for \"n_times_before\"?

\n\n

Q2. What is the correct false-color formula?

\n\n
\n

r = normalize_range(band15 - band14, _TDIFF_BOUNDS)
\n g = normalize_range(band14 - band11, _CLOUD_TOP_TDIFF_BOUNDS)
\n b = normalize_range(band14, _T11_BOUNDS)

\n
\n

In other words:

\n\n

Answers (from the host):

\n
\n

For Q2. I believe that the description in the data section of this competition is correct and the preprint is incorrect. We'll fix this in the next draft of the preprint.
\n For Q1, I'll have to confirm with coauthors, but it is possible that both are correct. Since the labelers only labeled a single frame, it's possible that we showed them 5 frames before and 2 after, but for this competition we provide 4 frames before and 3 after. I don't know what would have driven them to be different, but they could both be correct.

\n
\n
\n\n

Proposed a method to load numpy datasets much faster:

\n
class fastnumpyio:\n    def ():\n        =(,)\n        header = .()\n        descr = str(header[:], ).(,).(,)\n        shape = tuple(int()    str(header[:], ).(, ).(, ).(, ).())\n        datasize = np.lib..descr_to_dtype(descr).itemsize\n         dimension  shape:\n            datasize *= dimension\n         np.ndarray(shape, dtype=descr, buffer=.(datasize))\n
\n
\n\n
\n

Spoiler: It doesn't use BatchNorm

\n
\n

In this post, J€ANMPIA proposes to use batch norm to improve unet models.

\n

Code example:

\n
def double_conv(in_channels, out_channels):\n    return nn.Sequential(\n        nn.Conv2d(in_channels, out_channels, 3, =1, =), \n        nn.BatchNorm2d(out_channels),\n        nn.ReLU(=),\n        nn.Conv2d(out_channels, out_channels, 3, =1, =), \n        nn.BatchNorm2d(out_channels),\n        nn.ReLU(=)\n    )   \n
\n
\n
\n

This is true. It usually always help (but you still have to test it out to make sure!)

\n
\n\n

Egor Trushin shared with us a powerful pytorch lightning pipeline:

\n\n
\n\n

Stressing that we have to pay attention to post-processing and it's impact on the score because improving average dice score per image is not the same as improving global dice score.

\n

Some Ideas from this post:

\n\n

Some comments also tried the following:

\n\n
\n\n

In many CV tasks, bigger image size often gives us higher performance, But for the author, on this competition: bigger image size got higher score on CV but lower score on LB.

\n
\n

Serveral comments reported the same.

\n
\n
\n\n
\n\n

Pointing out that there are duplicates on the dataset.

\n

Might be the same place but different day/time.

\n
\n\n

Starting a discussion about 3D models and asking the other competitors if they had any luck with such models.

\n\n
\n

Suggested read!

\n
\n
\n\n

Reporting that so far, Randomresizedcrop alone gives improvement.

\n
\n

Martin Kovacevic Buvinic Tried flips and rotation augmentations with no luck.

\n
\n
\n\n

Highly Suggested!!

\n

A very good breakdown of many attempts and experimental results!

\n

A must read for everyone on this competition.

\n
\n\n

Again: Highly Suggested!!

\n

Again: A very good breakdown of the whole competition. A must read in my opinion.

\n
\n

This is it for now.
\nI will keep updating this post when new information is discovered.

\n

Good luck on the last month!

\n
\n
\n

This is not ChatGPT. I actually go by hand and read everything. So do call me out if anything I wrote here is inaccurate because I want to know for myself 🙃

\n
", "rawMarkdown": "## One month to go: Summary of everything that happened\n\n\nHappy last month to everyone! \nA good time to take a look back and summarize everything we know so far.\n\nStarting from the basics: In this competition we are tasked with detecting contrails in satelite images. (segmentation)\n\n## Evaluation\n\nAs for the metric, we are being evaluated using [Dice coefficient](https://en.wikipedia.org/wiki/S%C3%B8rensen%E2%80%93Dice_coefficient).\n\n> ### Dice Coefficient\n>$$\\frac{2 * |X \\cap Y|}{|X| + |Y|}$$\n>\n>**Torch**\n> ```\n> def dice_coefficient(y_true, y_pred, smooth = 1e-6):\n> return (2. * (y_true.view(-1) * y_pred.view(-1)).sum() + smooth) / ((y_true.view(-1).sum() + y_pred.view(-1).sum()) + smooth)\n>```\n>**Tensorflow**\n>```\n> def dice_coefficient(y_true, y_pred, smooth=1e-6):\n> y_true_f = flatten(y_true)\n> y_pred_f = flatten(y_pred)\n> intersection = tf.reduce_sum(y_true_f * y_pred_f)\n> return (2. * intersection + smooth) / (tf.reduce_sum(y_true_f) + tf.reduce_sum(y_pred_f) + smooth)\n>```\n>\n>**Numpy**\n>```\n> def dice_coefficient(y_true, y_pred, smooth=1e-6):\n> y_true_f = y_true.flatten()\n> y_pred_f = y_pred.flatten()\n> intersection = np.sum(y_true_f * y_pred_f)\n> return (2. * intersection + smooth) / (np.sum(y_true_f) + np.sum(y_pred_f) + smooth)\n>```\n\n-----\n\n**Large Dataset**\n\n- One of the main challenges of this competition is [handleing](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/409439) [big datasets](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/409332). Since the dataset for this competition is very large (450GB) - it can not be loaded into memory and one should use a dataloader to load it in chunks.\n\n- On the same week, [some new information on how to handle big datasets using h5py](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/409401) was shared by [Jacob Sushenok](https://www.kaggle.com/yakovsushenok): Information can be found [here](https://towardsdatascience.com/hdf5-datasets-for-pytorch-631ff1d750f5) and [here](https://github.com/h5py/h5py)\n\n\n-----\n\n**[Previous Competitions Winning Solutions](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/409434)** By [Dewei Chen](https://www.kaggle.com/dwchen)\n\n\n- [1st place solution](https://www.kaggle.com/competitions/nfl-player-contact-detection/discussion/391635)\n- [2nd place solution](https://www.kaggle.com/competitions/nfl-player-contact-detection/discussion/391740)\n- [3rd place solution](https://www.kaggle.com/competitions/nfl-player-contact-detection/discussion/392182)\n- [4th place solution](https://www.kaggle.com/competitions/nfl-player-contact-detection/discussion/391761)\n- [5th place solution](https://www.kaggle.com/competitions/nfl-player-contact-detection/discussion/392290)\n\n\n-----\n\n\n- Shortly after the competition started, we got a [nice compilation](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/409455) of previous notebooks using this metric from [Ravi Ramakrishnan](https://www.kaggle.com/ravi20076).\n\n\n - https://www.kaggle.com/code/dschettler8845/uwm-gi-tract-image-segmentation-eda\n - https://www.kaggle.com/code/yerramvarun/understanding-dice-coefficient\n - https://www.kaggle.com/code/bigironsphere/loss-function-library-keras-pytorch\n - https://www.kaggle.com/code/iafoss/unet34-dice-0-87\n - https://www.kaggle.com/code/dschettler8845/eda-hubmap-hpa-organ-segmentation\n - https://www.kaggle.com/code/ekhtiar/resunet-a-baseline-on-tensorflow\n - https://www.kaggle.com/code/vineeth1999/hubmap-eda-pytorch-efficientunet-offline-training\n \n-----\n \n- We then got a [super fast data loading but with some tradeoff](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/410316) from [Soumyadeep Khandual](https://www.kaggle.com/soumyadeepkhandual) - A more efficient version of the competition dataset by reducing IO and utilizing float16. Despite a minute information loss (less than 0.019%) due to the float16 usage, this version boosts data loading speed and lessens CPU usage.\n\n - [part1](https://www.kaggle.com/datasets/soumyadeepkhandual/google-contrails-normalized-float16-part-1)\n - [part2](https://www.kaggle.com/datasets/soumyadeepkhandual/google-contrails-normalized-float16-part-2)\n - [part3](https://www.kaggle.com/datasets/soumyadeepkhandual/google-contrails-normalized-float16-part-3)\n - [part4](https://www.kaggle.com/datasets/soumyadeepkhandual/google-contrails-normalized-float16-part-4)\n - [part5](https://www.kaggle.com/datasets/soumyadeepkhandual/google-contrails-normalized-float16-part-5)\n\n - [Example Notebook](https://www.kaggle.com/code/soumyadeepkhandual/superfast-dataloading)\n\n-----\n\n### Questions\n\n-----\n\n- Question: [Can we somehow get the reddish false color scheme as shown in Fig 3](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/409913) By [Aryan Garg](https://www.kaggle.com/aryangarg01)\n\n**Answer (By [Ericka42](https://www.kaggle.com/ericka42))**\n\n> In the preprint, the Advected Flight Density is shown to labelers as a heat map with a color scale ranging from blue to red. The blue color represents low flight density, while the red color represents high flight density.\n> To replicate the reddish false color scheme shown in Fig 3, you could try adjusting the color scale in your heatmap visualization to highlight the higher density regions with a reddish hue. You could also experiment with different color palettes and custom color schemes to find what works best for your specific use case.\n\n\n-----\n\n- Question: [About missing masks](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/410727) By [Tord Malmgren](https://www.kaggle.com/glimmung)\n\nPointing out that out of the first 120 entries (ordered by recordId), there are 49 with empty masks.\n\n**Answer (From the host):**\n\n> What you're seeing is expected. Most scenes do not contain any contrails, so most masks are all 0's. The distribution of empty masks should roughly reflect what is seen in the real world for the space-time region that the dataset covers. The goal is to train a model that can eventually be run on an arbitrary scene from a satellite image and identify contrails if they are there, even though most of the time there won't be any. You're welcome to try out different training approaches that might involve upweighting the samples that do have contrails or subsampling the ones that don't.\n> It's also expected that the combined mask can be all 0's in cases where individual masks are not all 0's. This is because the combined mask is a majority vote per-pixel of all the individual masks.\n\n-----\n\n- Question [About false color image](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/410540) By [Kenni](https://www.kaggle.com/thejavanka), Intrigued by false color image generated using bands 11 -14. But keep coming back to bands from 8-16. Is building a model using false color image viable or do we need to use all frames?\n\n\n**Answer (By [Patchef](https://www.kaggle.com/patchef)):**\n\n> As mentioned in the data section:\n> \"human_pixel_masks.npy: array with size of H x W x 1 x R. Each example is labeled by R individual human labelers. R is not the same for all samples. The labeled masks have value either 0 or 1 and correspond to the (n_times_before+1)-th image in band_{08-16}.npy. They are available only in the training set.\"\n> It corresponds to the time stamp (n_times_before+1)\n\nThis is an interesting discussion overall, I suggest reading it.\n\n\n-----\n\n- Question: [What happens when the predicted mask and the ground truth is empty?](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/412673) By [MD Mushfirat Mohaimin](https://www.kaggle.com/mushfirat)\n\n**Answer (from the host):**\n\n> If you look at the metric definition at https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/overview/evaluation, you see that the numerator is the size of the intersection of X and Y, where X is the set of predicted contrail pixels and Y is the set of groundtruth contrail pixels. In your case X is the empty set, so the size of the intersection will be 0.\n\n-----\n\n**Means and Standard Deviation for Normalization** By [Soumyadeep Khandual](https://www.kaggle.com/soumyadeepkhandual)\n\nSharing with us normaliztion values: since we are working on 9 infrared channel at different wavelengths which are converted to brightness temperatures, we cannot simply divide each pixel intensity by 255.\n\nThese are the mean and std of train dataset for the 9 channels which can be used to normalize images:\n\n```\nMean = [233.6771, 242.2548, 250.7509, 274.4108, 255.5268, 276.6016, 275.3604, 272.5643, 260.4260]\nStd = [ 7.0181, 9.1566, 11.3484, 19.6334, 13.1177, 20.7182, 21.0882, 20.5616, 15.8269]\n```\n\nAnd the 9 channels correspond to band_08 to band_16. The code i used is given below:\n\n```\ntotal_sum = torch.zeros(9)\ntotal_sum_of_sq = torch.zeros(9)\n\nfor data, _ in tqdm(train_loader):\n total_sum += torch.sum(data, axis=[0,1,3,4]) \n total_sum_of_sq += torch.sum(data**2, axis=[0,1,3,4])\n\ntotal_count = len(train_loader)*train_loader.batch_size*8*256*256\ntotal_mean = total_sum / total_count\ntotal_mean_of_sq = total_sum_of_sq/total_count\ntotal_std = torch.sqrt(total_mean_of_sq - total_mean**2)\n```\n\n-----\n\n- During the same time [Sergey Saharovskiy](https://www.kaggle.com/sergiosaharovskiy) [explored the file size patterns of the data](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/409624) and found an interesting pattern in the data. In total, He identified 57 records of `human_individual_masks.npy` files that had the exact same size as `band_{}.npy` files. Most of those masks are blank.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6259210%2Fe736d4a036fe8ac76485380c7bd71f3e%2Frec_3528549774485167627.png?generation=1683848300543927&alt=media)\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6259210%2F330e944cd197fb6a4a920b065dcdec08%2Fhm_3528549774485167627.png?generation=1683848319676525&alt=media)\n\n\n-----\n\n[Preprocessed Data](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/411713) By [Kenni](https://www.kaggle.com/thejavanka)\n\nFor people struggling to preprocess their data, Kenni have enclosed a dataset using false image generation [here](https://www.kaggle.com/datasets/thejavanka/google-research-identify-contrails-preprocessing).\n\n\n-----\n\n- [A 0.05+ lift by reducing confidence](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/412951) By [LUPIN11](https://www.kaggle.com/lupin11)\n\nIn this post, it is reported that by reducing the model's confidence in contrail predictions, it is highly likely that the Dice coeff will improve.\n\n```\nk = -0.5 \npred = F.softmax(pred, dim=1)\npred[:, 1, :, :] += k # reducing confidence in contrail predictions\npred = F.softmax(pred, dim=1)\npred = torch.argmax(pred, dim=1)\n```\n\nThe model in this thread was trained with WCE loss function and its performance is suboptimal (~0.309), But the author suspect that this could be helpful for a much better model or a model trained using DICE loss.\n\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F9893459%2Fc022874a97d47b3e642d3c213a8a260c%2Fp.png?generation=1685068320179092&alt=media)\n\n\n-----\n\n- [Updated reference code with real submission](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/412528) By [Doomsday](https://www.kaggle.com/phoenix9032)\n\nSince there are many failed submission, on this post there is a [ref notebook](https://www.kaggle.com/code/phoenix9032/inference-unet-effnetb0-on-ash-v2-baseline) for helping others getting started with this notebook.\n\n\n-----\n\n- [Loss Fuction: Dice Loss or WCE?](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/412554) By [LUPIN11](https://www.kaggle.com/lupin11)\n\nIn this post, [LUPIN11](https://www.kaggle.com/lupin11) reported experimental results from two different loss functions: WCE: Weighted Cross Entropy and Dice Loss.\n\nIn this experiment, WCE performed better.\n\n```\ndef ce_loss(y_p, y_t):\n weight = torch.Tensor([0.57, 4.17]).to('cuda')\n criterion = nn.CrossEntropyLoss(weight)\n loss = criterion(y_p, y_t)\n return loss\n```\n\n```\ndef dice_loss(y_p, y_t, smooth=1e-6):\n y_p = y_p.reshape(-1)\n y_t = y_t.reshape(-1)\n i = (y_p * y_t).sum()\n return 1 - (2. * i + smooth) / (y_p.sum() + y_t.sum() + smooth)\n```\n\nThis post also go in depth into the reasons this might be the case. Interesting!\n\n-----\n\n- [For Those Who Consider Big Ensembles (Not)](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/414700) By [Sergey Saharovskiy](https://www.kaggle.com/sergiosaharovskiy)\n\nIn this post, sergey shared with us a simple trick to overcome memory issues in this competition (since the data is so large). The trick is to send everything to CUDA.\n\n```\ntest_preds6 = torch.cat(get_predictions(ckpt_exp6_path, test_loader6)).cuda()\ntest_preds8 = torch.cat(get_predictions(ckpt_exp8_path, test_loader8)).cuda()\ntest_preds8 = test_preds6*0.7 + test_preds8*0.3\n```\n\n-----\n\n- [Unet Pytorch Baseline (LB 0.608)](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/415327) By [Jan H](https://www.kaggle.com/janhuebi)\n\nSharing with us a Unet Pytorch Baseline:\n\n**[Training](https://www.kaggle.com/code/janhuebi/unet-pytorch-baseline-lb-0-608-training)**\n\n**[Submission](https://www.kaggle.com/code/janhuebi/unet-pytorch-baseline-lb-0-608-submission)**\n\n\n-----\n\n- [Unet Baseline using PyTorch - [LB - 0.580]](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/415176) By [Shashwat Raman](https://www.kaggle.com/shashwatraman)\n\n- [Training Notebook](https://www.kaggle.com/code/shashwatraman/simple-unet-baseline-train-lb-0-580)\n- [Inference Notebook](https://www.kaggle.com/code/shashwatraman/simple-unet-baseline-infer-lb-0-580)\n- [Dataset Notebook](https://www.kaggle.com/code/shashwatraman/contrails-dataset-ash-color)\n\n**Library:** Smp\n**Data:** Ash Color images (With only the labeled frames and human_pixel_masks)\n**Backbone:** EfficientNet-B0\n**Postprocessing:** Finding the best threshold\n\n-----\n\n- [J€ANMPIA](https://www.kaggle.com/janmpia) [Visualized every training image so we don't have to](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/415603)\n\nPlotted all ASHRGB images and their corresponding mask in both the train and validation set.\n\n[here](https://www.kaggle.com/code/janmpia/visualise-all-images-targets)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12870466%2F225bac2a82c64ffe87f90d3c09a68e4c%2FScreenshot_100.jpg?generation=1686096240704345&alt=media)\n\n\n-----\n\n- [Dual Thresholds are Slightly Better than Single Threshold](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/416630) By [LUPIN11](https://www.kaggle.com/lupin11)\n\n\nIn this competition, finding the optimal threshold is a crucial step in improving the score.\n\nSo in this post, a [dual threshold](https://www.kaggle.com/code/lupin11/doubleshreshold/notebook) is proposed and improved the authors model's score from 0.55 to 0.552.\n\n-----\n\n- [Data description vs Preprint](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/417497) By [ticadumi](https://www.kaggle.com/constantindumitrascu)\n\n**Q1. What is the correct value for \"n_times_before\"?**\n\n- Preprint reads: \"we show labelers 5 images (50 minutes) before and 2 images (20 minutes) after the image being labeled\"\n- Kaggle data section reads: \"all examples have n_times_before=4 and n_times_after=3\"\n\n**Q2. What is the correct false-color formula?**\n\n- Preprint reads: \" The red, blue and green channels are represented by the 12µm, difference between 12µm and 11µm, and difference between 11µm and 8µm respectively.\"\n- Kaggle data section notebook seems to have red as difference between 12µm and 11µm, green as difference between 11µm and 8µm, and blue as 11µm:\n\n> r = normalize_range(band15 - band14, _TDIFF_BOUNDS)\n> g = normalize_range(band14 - band11, _CLOUD_TOP_TDIFF_BOUNDS)\n> b = normalize_range(band14, _T11_BOUNDS)\n\nIn other words:\n\n- kaggle red matches preprint blue\n- kaggle green is preprint green (ok)\n- kaggle blue matches preprint red as self-contained value, except that value is 11µm in kaggle vs 12µm in preprint\n\n**Answers (from the host):**\n\n> For Q2. I believe that the description in the data section of this competition is correct and the preprint is incorrect. We'll fix this in the next draft of the preprint.\n> For Q1, I'll have to confirm with coauthors, but it is possible that both are correct. Since the labelers only labeled a single frame, it's possible that we showed them 5 frames before and 2 after, but for this competition we provide 4 frames before and 3 after. I don't know what would have driven them to be different, but they could both be correct.\n\n-----\n\n- [Load numpy arrays 1.7x faster](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/414549) By [J€ANMPIA](https://www.kaggle.com/janmpia)\n\n\nProposed a method to load numpy datasets much faster:\n\n```\nclass fastnumpyio:\n def load(file):\n file=open(file,\"rb\")\n header = file.read(128)\n descr = str(header[19:25], 'utf-8').replace(\"'\",\"\").replace(\" \",\"\")\n shape = tuple(int(num) for num in str(header[60:120], 'utf-8').replace(', }', '').replace('(', '').replace(')', '').split(','))\n datasize = np.lib.format.descr_to_dtype(descr).itemsize\n for dimension in shape:\n datasize *= dimension\n return np.ndarray(shape, dtype=descr, buffer=file.read(datasize))\n```\n\n-----\n\n- [U-Net is missing something](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/413767) By [J€ANMPIA](https://www.kaggle.com/janmpia)\n\n> Spoiler: It doesn't use BatchNorm\n\nIn this post, [J€ANMPIA](https://www.kaggle.com/janmpia) proposes to use batch norm to improve unet models.\n\nCode example:\n\n```\ndef double_conv(in_channels, out_channels):\n return nn.Sequential(\n nn.Conv2d(in_channels, out_channels, 3, padding=1, bias=False), \n nn.BatchNorm2d(out_channels),\n nn.ReLU(inplace=True),\n nn.Conv2d(out_channels, out_channels, 3, padding=1, bias=False), \n nn.BatchNorm2d(out_channels),\n nn.ReLU(inplace=True)\n ) \n``` \n\nThis is true. It usually always help (but you still have to test it out to make sure!)\n\n-----\n\n- [Pytorch Lightning baseline](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/416746) By [Egor Trushin](https://www.kaggle.com/egortrushin)\n\n\n[Egor Trushin](https://www.kaggle.com/egortrushin) shared with us a powerful pytorch lightning pipeline:\n\n- [Notebook](https://www.kaggle.com/code/egortrushin/gr-icrgw-pytorch-lightning-baseline-unet-resnest)\n- [[GR-ICRGW] PL Pipeline Improved](https://www.kaggle.com/code/egortrushin/gr-icrgw-pl-pipeline-improved)\n- [[GR-ICRGW] Training with 4 folds](https://www.kaggle.com/code/egortrushin/gr-icrgw-training-with-4-folds?scriptVersionId=134148499)\n\n\n-----\n\n- [Post-Processing](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/416436) By [MPWARE](https://www.kaggle.com/mpware)\n\nStressing that we have to pay attention to post-processing and it's impact on the score because improving average dice score per image is not the same as improving global dice score.\n\n**Some Ideas from this post:**\n\n- **Dropping small items:** Tried to remove small predicted items below a certain pixel threshold, but found that this improved average dice score per image, but not the global dice score.\n- **Removing masks with small sums:** Attempted to remove any masks where the sum of the mask was less than a certain threshold. This didn't improve the validation score.\n- **Using Morphological Operations:** Experimented with was morphological operations like dilation, opening, closing, and tophat. These are operations that can potentially help to remove noise and small anomalies in the masks, but in this case, they didn't lead to improvement in validation score.\n- **Line Segment Detection:** Mentioned a paper that used OpenCV's LineSegmentDetector for post-processing. This technique might be useful in certain contexts where the objects of interest are line-like or have linear features.\n\nSome comments also tried the following:\n\n- Implementing a drop mask with np.sum(mask) < N, particularly with a threshold of fewer than 11 pixels, which was noted to bring minor improvement.\n- Using morphology operators like dilatation, opening, closing, and tophat. Although different kernel sizes and shapes were experimented with, these methods seemed to negatively impact the score for the most part, with only some bins showing a minimal increase.\n\n-----\n\n- [Increasing image size doesn't work for me on LB](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/420079) By [Tawara](https://www.kaggle.com/ttahara)\n\nIn many CV tasks, bigger image size often gives us higher performance, But for the author, on this competition: bigger image size got higher score on CV but lower score on LB.\n\n> Serveral comments reported the same.\n\n-----\n\n\n- [Trust ur CV even though LB drops](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/420135) By [LUPIN11](https://www.kaggle.com/lupin11)\n\n- On this thread, the author points to significant improvements on a larger dataset may translate into minor improvements or even drops on a smaller subset due to sample size limitations. Therefore, trust your Cross-Validation (CV) even when Leaderboard (LB) drops.\n- The optimal threshold may vary on different datasets or subsets. A model that falls behind on a smaller subset due to fluctuating optimal thresholds may actually have potential to surpass another model by just adjusting the threshold.\n- Experiments on subsets of validation data can explain seemingly trivial LB improvements in relation to more significant CV improvements. This is due to the smaller representation of data on the public LB which may not accurately reflect model performance improvements.\n\n\n-----\n\n\n- [Data Leakage: Duplicate Images and Masks in Training and Validation Sets](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/419994) By [Sergey Saharovskiy](https://www.kaggle.com/sergiosaharovskiy)\n\nPointing out that there are duplicates on the dataset.\n\n\nMight be the same place but different day/time.\n\n-----\n\n\n- [Are 3D models worth?](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/419816) By [MPWARE](https://www.kaggle.com/mpware)\n\nStarting a discussion about 3D models and asking the other competitors if they had any luck with such models. \n- Most competitors reported disappointing results using 3D backbones for image segmentation models, noting that basic 2D backbones often outperformed them.\n- Some reported that adding temporal context in the form of extra channels in a 2D model (2.5D approach) also failed to provide significant improvements.\n- It appears that the choice of encoder has a more significant impact on the results than the decoder, however, more experimentation is needed to fully understand this outcome.\n\n> **Suggested read!**\n\n\n-----\n\n- [Augmentation Analysis](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/420180) By [Balaji Selvaraj](https://www.kaggle.com/dhakshiin1601)\n\nReporting that so far, Randomresizedcrop alone gives improvement.\n \n> [Martin Kovacevic Buvinic](https://www.kaggle.com/ragnar123) Tried flips and rotation augmentations with no luck.\n\n-----\n\n\n- [Some Successful/Fail Experiment](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/414344) By [lyu](https://www.kaggle.com/zhuwanglju)\n\n**Highly Suggested!!**\n\nA very good breakdown of many attempts and experimental results!\n\nA must read for everyone on this competition.\n\n\n-----\n\n\n- [Main Findings after the first two Competition Weeks](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/413068) By [Jan H](https://www.kaggle.com/janhuebi)\n\n**Again: Highly Suggested!!**\n\nAgain: A very good breakdown of the whole competition. A must read in my opinion.\n\n-----\n\n\nThis is it for now.\nI will keep updating this post when new information is discovered.\n\nGood luck on the last month!\n\n-----\n\n> This is not ChatGPT. I **actually** go by hand and read everything. So do call me out if anything I wrote here is inaccurate because I want to know for myself 🙃\n\n\n\n\n\n\n\n\n\n\n", "votes": 110 }, { "id": 2329096, "postDate": "2023-07-04T05:19:37.393Z", "content": "

I came into this competition late and this was a great recap. Thanks.

", "rawMarkdown": "I came into this competition late and this was a great recap. Thanks. ", "votes": 2 }, { "id": 2326144, "postDate": "2023-07-01T23:58:15.043Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 2329096, "author_name": "Grant Bowling", "author_url": "", "post_date": "2023-07-04T05:19:37.393000", "content": "

I came into this competition late and this was a great recap. Thanks.

", "votes": 2, "replies": [] }, { "id": 2326144, "author_name": "", "author_url": "", "post_date": "2023-07-01T23:58:15.043000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2325989": "## One month to go: Summary of everything that happened\n\n\nHappy last month to everyone! \nA good time to take a look back and summarize everything we know so far.\n\nStarting from the basics: In this competition we are tasked with detecting contrails in satelite images. (segmentation)\n\n## Evaluation\n\nAs for the metric, we are being evaluated using [Dice coefficient](https://en.wikipedia.org/wiki/S%C3%B8rensen%E2%80%93Dice_coefficient).\n\n> ### Dice Coefficient\n>$$\\frac{2 * |X \\cap Y|}{|X| + |Y|}$$\n>\n>**Torch**\n> ```\n> def dice_coefficient(y_true, y_pred, smooth = 1e-6):\n> return (2. * (y_true.view(-1) * y_pred.view(-1)).sum() + smooth) / ((y_true.view(-1).sum() + y_pred.view(-1).sum()) + smooth)\n>```\n>**Tensorflow**\n>```\n> def dice_coefficient(y_true, y_pred, smooth=1e-6):\n> y_true_f = flatten(y_true)\n> y_pred_f = flatten(y_pred)\n> intersection = tf.reduce_sum(y_true_f * y_pred_f)\n> return (2. * intersection + smooth) / (tf.reduce_sum(y_true_f) + tf.reduce_sum(y_pred_f) + smooth)\n>```\n>\n>**Numpy**\n>```\n> def dice_coefficient(y_true, y_pred, smooth=1e-6):\n> y_true_f = y_true.flatten()\n> y_pred_f = y_pred.flatten()\n> intersection = np.sum(y_true_f * y_pred_f)\n> return (2. * intersection + smooth) / (np.sum(y_true_f) + np.sum(y_pred_f) + smooth)\n>```\n\n-----\n\n**Large Dataset**\n\n- One of the main challenges of this competition is [handleing](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/409439) [big datasets](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/409332). Since the dataset for this competition is very large (450GB) - it can not be loaded into memory and one should use a dataloader to load it in chunks.\n\n- On the same week, [some new information on how to handle big datasets using h5py](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/409401) was shared by [Jacob Sushenok](https://www.kaggle.com/yakovsushenok): Information can be found [here](https://towardsdatascience.com/hdf5-datasets-for-pytorch-631ff1d750f5) and [here](https://github.com/h5py/h5py)\n\n\n-----\n\n**[Previous Competitions Winning Solutions](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/409434)** By [Dewei Chen](https://www.kaggle.com/dwchen)\n\n\n- [1st place solution](https://www.kaggle.com/competitions/nfl-player-contact-detection/discussion/391635)\n- [2nd place solution](https://www.kaggle.com/competitions/nfl-player-contact-detection/discussion/391740)\n- [3rd place solution](https://www.kaggle.com/competitions/nfl-player-contact-detection/discussion/392182)\n- [4th place solution](https://www.kaggle.com/competitions/nfl-player-contact-detection/discussion/391761)\n- [5th place solution](https://www.kaggle.com/competitions/nfl-player-contact-detection/discussion/392290)\n\n\n-----\n\n\n- Shortly after the competition started, we got a [nice compilation](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/409455) of previous notebooks using this metric from [Ravi Ramakrishnan](https://www.kaggle.com/ravi20076).\n\n\n - https://www.kaggle.com/code/dschettler8845/uwm-gi-tract-image-segmentation-eda\n - https://www.kaggle.com/code/yerramvarun/understanding-dice-coefficient\n - https://www.kaggle.com/code/bigironsphere/loss-function-library-keras-pytorch\n - https://www.kaggle.com/code/iafoss/unet34-dice-0-87\n - https://www.kaggle.com/code/dschettler8845/eda-hubmap-hpa-organ-segmentation\n - https://www.kaggle.com/code/ekhtiar/resunet-a-baseline-on-tensorflow\n - https://www.kaggle.com/code/vineeth1999/hubmap-eda-pytorch-efficientunet-offline-training\n \n-----\n \n- We then got a [super fast data loading but with some tradeoff](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/410316) from [Soumyadeep Khandual](https://www.kaggle.com/soumyadeepkhandual) - A more efficient version of the competition dataset by reducing IO and utilizing float16. Despite a minute information loss (less than 0.019%) due to the float16 usage, this version boosts data loading speed and lessens CPU usage.\n\n - [part1](https://www.kaggle.com/datasets/soumyadeepkhandual/google-contrails-normalized-float16-part-1)\n - [part2](https://www.kaggle.com/datasets/soumyadeepkhandual/google-contrails-normalized-float16-part-2)\n - [part3](https://www.kaggle.com/datasets/soumyadeepkhandual/google-contrails-normalized-float16-part-3)\n - [part4](https://www.kaggle.com/datasets/soumyadeepkhandual/google-contrails-normalized-float16-part-4)\n - [part5](https://www.kaggle.com/datasets/soumyadeepkhandual/google-contrails-normalized-float16-part-5)\n\n - [Example Notebook](https://www.kaggle.com/code/soumyadeepkhandual/superfast-dataloading)\n\n-----\n\n### Questions\n\n-----\n\n- Question: [Can we somehow get the reddish false color scheme as shown in Fig 3](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/409913) By [Aryan Garg](https://www.kaggle.com/aryangarg01)\n\n**Answer (By [Ericka42](https://www.kaggle.com/ericka42))**\n\n> In the preprint, the Advected Flight Density is shown to labelers as a heat map with a color scale ranging from blue to red. The blue color represents low flight density, while the red color represents high flight density.\n> To replicate the reddish false color scheme shown in Fig 3, you could try adjusting the color scale in your heatmap visualization to highlight the higher density regions with a reddish hue. You could also experiment with different color palettes and custom color schemes to find what works best for your specific use case.\n\n\n-----\n\n- Question: [About missing masks](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/410727) By [Tord Malmgren](https://www.kaggle.com/glimmung)\n\nPointing out that out of the first 120 entries (ordered by recordId), there are 49 with empty masks.\n\n**Answer (From the host):**\n\n> What you're seeing is expected. Most scenes do not contain any contrails, so most masks are all 0's. The distribution of empty masks should roughly reflect what is seen in the real world for the space-time region that the dataset covers. The goal is to train a model that can eventually be run on an arbitrary scene from a satellite image and identify contrails if they are there, even though most of the time there won't be any. You're welcome to try out different training approaches that might involve upweighting the samples that do have contrails or subsampling the ones that don't.\n> It's also expected that the combined mask can be all 0's in cases where individual masks are not all 0's. This is because the combined mask is a majority vote per-pixel of all the individual masks.\n\n-----\n\n- Question [About false color image](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/410540) By [Kenni](https://www.kaggle.com/thejavanka), Intrigued by false color image generated using bands 11 -14. But keep coming back to bands from 8-16. Is building a model using false color image viable or do we need to use all frames?\n\n\n**Answer (By [Patchef](https://www.kaggle.com/patchef)):**\n\n> As mentioned in the data section:\n> \"human_pixel_masks.npy: array with size of H x W x 1 x R. Each example is labeled by R individual human labelers. R is not the same for all samples. The labeled masks have value either 0 or 1 and correspond to the (n_times_before+1)-th image in band_{08-16}.npy. They are available only in the training set.\"\n> It corresponds to the time stamp (n_times_before+1)\n\nThis is an interesting discussion overall, I suggest reading it.\n\n\n-----\n\n- Question: [What happens when the predicted mask and the ground truth is empty?](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/412673) By [MD Mushfirat Mohaimin](https://www.kaggle.com/mushfirat)\n\n**Answer (from the host):**\n\n> If you look at the metric definition at https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/overview/evaluation, you see that the numerator is the size of the intersection of X and Y, where X is the set of predicted contrail pixels and Y is the set of groundtruth contrail pixels. In your case X is the empty set, so the size of the intersection will be 0.\n\n-----\n\n**Means and Standard Deviation for Normalization** By [Soumyadeep Khandual](https://www.kaggle.com/soumyadeepkhandual)\n\nSharing with us normaliztion values: since we are working on 9 infrared channel at different wavelengths which are converted to brightness temperatures, we cannot simply divide each pixel intensity by 255.\n\nThese are the mean and std of train dataset for the 9 channels which can be used to normalize images:\n\n```\nMean = [233.6771, 242.2548, 250.7509, 274.4108, 255.5268, 276.6016, 275.3604, 272.5643, 260.4260]\nStd = [ 7.0181, 9.1566, 11.3484, 19.6334, 13.1177, 20.7182, 21.0882, 20.5616, 15.8269]\n```\n\nAnd the 9 channels correspond to band_08 to band_16. The code i used is given below:\n\n```\ntotal_sum = torch.zeros(9)\ntotal_sum_of_sq = torch.zeros(9)\n\nfor data, _ in tqdm(train_loader):\n total_sum += torch.sum(data, axis=[0,1,3,4]) \n total_sum_of_sq += torch.sum(data**2, axis=[0,1,3,4])\n\ntotal_count = len(train_loader)*train_loader.batch_size*8*256*256\ntotal_mean = total_sum / total_count\ntotal_mean_of_sq = total_sum_of_sq/total_count\ntotal_std = torch.sqrt(total_mean_of_sq - total_mean**2)\n```\n\n-----\n\n- During the same time [Sergey Saharovskiy](https://www.kaggle.com/sergiosaharovskiy) [explored the file size patterns of the data](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/409624) and found an interesting pattern in the data. In total, He identified 57 records of `human_individual_masks.npy` files that had the exact same size as `band_{}.npy` files. Most of those masks are blank.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6259210%2Fe736d4a036fe8ac76485380c7bd71f3e%2Frec_3528549774485167627.png?generation=1683848300543927&alt=media)\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6259210%2F330e944cd197fb6a4a920b065dcdec08%2Fhm_3528549774485167627.png?generation=1683848319676525&alt=media)\n\n\n-----\n\n[Preprocessed Data](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/411713) By [Kenni](https://www.kaggle.com/thejavanka)\n\nFor people struggling to preprocess their data, Kenni have enclosed a dataset using false image generation [here](https://www.kaggle.com/datasets/thejavanka/google-research-identify-contrails-preprocessing).\n\n\n-----\n\n- [A 0.05+ lift by reducing confidence](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/412951) By [LUPIN11](https://www.kaggle.com/lupin11)\n\nIn this post, it is reported that by reducing the model's confidence in contrail predictions, it is highly likely that the Dice coeff will improve.\n\n```\nk = -0.5 \npred = F.softmax(pred, dim=1)\npred[:, 1, :, :] += k # reducing confidence in contrail predictions\npred = F.softmax(pred, dim=1)\npred = torch.argmax(pred, dim=1)\n```\n\nThe model in this thread was trained with WCE loss function and its performance is suboptimal (~0.309), But the author suspect that this could be helpful for a much better model or a model trained using DICE loss.\n\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F9893459%2Fc022874a97d47b3e642d3c213a8a260c%2Fp.png?generation=1685068320179092&alt=media)\n\n\n-----\n\n- [Updated reference code with real submission](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/412528) By [Doomsday](https://www.kaggle.com/phoenix9032)\n\nSince there are many failed submission, on this post there is a [ref notebook](https://www.kaggle.com/code/phoenix9032/inference-unet-effnetb0-on-ash-v2-baseline) for helping others getting started with this notebook.\n\n\n-----\n\n- [Loss Fuction: Dice Loss or WCE?](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/412554) By [LUPIN11](https://www.kaggle.com/lupin11)\n\nIn this post, [LUPIN11](https://www.kaggle.com/lupin11) reported experimental results from two different loss functions: WCE: Weighted Cross Entropy and Dice Loss.\n\nIn this experiment, WCE performed better.\n\n```\ndef ce_loss(y_p, y_t):\n weight = torch.Tensor([0.57, 4.17]).to('cuda')\n criterion = nn.CrossEntropyLoss(weight)\n loss = criterion(y_p, y_t)\n return loss\n```\n\n```\ndef dice_loss(y_p, y_t, smooth=1e-6):\n y_p = y_p.reshape(-1)\n y_t = y_t.reshape(-1)\n i = (y_p * y_t).sum()\n return 1 - (2. * i + smooth) / (y_p.sum() + y_t.sum() + smooth)\n```\n\nThis post also go in depth into the reasons this might be the case. Interesting!\n\n-----\n\n- [For Those Who Consider Big Ensembles (Not)](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/414700) By [Sergey Saharovskiy](https://www.kaggle.com/sergiosaharovskiy)\n\nIn this post, sergey shared with us a simple trick to overcome memory issues in this competition (since the data is so large). The trick is to send everything to CUDA.\n\n```\ntest_preds6 = torch.cat(get_predictions(ckpt_exp6_path, test_loader6)).cuda()\ntest_preds8 = torch.cat(get_predictions(ckpt_exp8_path, test_loader8)).cuda()\ntest_preds8 = test_preds6*0.7 + test_preds8*0.3\n```\n\n-----\n\n- [Unet Pytorch Baseline (LB 0.608)](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/415327) By [Jan H](https://www.kaggle.com/janhuebi)\n\nSharing with us a Unet Pytorch Baseline:\n\n**[Training](https://www.kaggle.com/code/janhuebi/unet-pytorch-baseline-lb-0-608-training)**\n\n**[Submission](https://www.kaggle.com/code/janhuebi/unet-pytorch-baseline-lb-0-608-submission)**\n\n\n-----\n\n- [Unet Baseline using PyTorch - [LB - 0.580]](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/415176) By [Shashwat Raman](https://www.kaggle.com/shashwatraman)\n\n- [Training Notebook](https://www.kaggle.com/code/shashwatraman/simple-unet-baseline-train-lb-0-580)\n- [Inference Notebook](https://www.kaggle.com/code/shashwatraman/simple-unet-baseline-infer-lb-0-580)\n- [Dataset Notebook](https://www.kaggle.com/code/shashwatraman/contrails-dataset-ash-color)\n\n**Library:** Smp\n**Data:** Ash Color images (With only the labeled frames and human_pixel_masks)\n**Backbone:** EfficientNet-B0\n**Postprocessing:** Finding the best threshold\n\n-----\n\n- [J€ANMPIA](https://www.kaggle.com/janmpia) [Visualized every training image so we don't have to](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/415603)\n\nPlotted all ASHRGB images and their corresponding mask in both the train and validation set.\n\n[here](https://www.kaggle.com/code/janmpia/visualise-all-images-targets)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12870466%2F225bac2a82c64ffe87f90d3c09a68e4c%2FScreenshot_100.jpg?generation=1686096240704345&alt=media)\n\n\n-----\n\n- [Dual Thresholds are Slightly Better than Single Threshold](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/416630) By [LUPIN11](https://www.kaggle.com/lupin11)\n\n\nIn this competition, finding the optimal threshold is a crucial step in improving the score.\n\nSo in this post, a [dual threshold](https://www.kaggle.com/code/lupin11/doubleshreshold/notebook) is proposed and improved the authors model's score from 0.55 to 0.552.\n\n-----\n\n- [Data description vs Preprint](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/417497) By [ticadumi](https://www.kaggle.com/constantindumitrascu)\n\n**Q1. What is the correct value for \"n_times_before\"?**\n\n- Preprint reads: \"we show labelers 5 images (50 minutes) before and 2 images (20 minutes) after the image being labeled\"\n- Kaggle data section reads: \"all examples have n_times_before=4 and n_times_after=3\"\n\n**Q2. What is the correct false-color formula?**\n\n- Preprint reads: \" The red, blue and green channels are represented by the 12µm, difference between 12µm and 11µm, and difference between 11µm and 8µm respectively.\"\n- Kaggle data section notebook seems to have red as difference between 12µm and 11µm, green as difference between 11µm and 8µm, and blue as 11µm:\n\n> r = normalize_range(band15 - band14, _TDIFF_BOUNDS)\n> g = normalize_range(band14 - band11, _CLOUD_TOP_TDIFF_BOUNDS)\n> b = normalize_range(band14, _T11_BOUNDS)\n\nIn other words:\n\n- kaggle red matches preprint blue\n- kaggle green is preprint green (ok)\n- kaggle blue matches preprint red as self-contained value, except that value is 11µm in kaggle vs 12µm in preprint\n\n**Answers (from the host):**\n\n> For Q2. I believe that the description in the data section of this competition is correct and the preprint is incorrect. We'll fix this in the next draft of the preprint.\n> For Q1, I'll have to confirm with coauthors, but it is possible that both are correct. Since the labelers only labeled a single frame, it's possible that we showed them 5 frames before and 2 after, but for this competition we provide 4 frames before and 3 after. I don't know what would have driven them to be different, but they could both be correct.\n\n-----\n\n- [Load numpy arrays 1.7x faster](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/414549) By [J€ANMPIA](https://www.kaggle.com/janmpia)\n\n\nProposed a method to load numpy datasets much faster:\n\n```\nclass fastnumpyio:\n def load(file):\n file=open(file,\"rb\")\n header = file.read(128)\n descr = str(header[19:25], 'utf-8').replace(\"'\",\"\").replace(\" \",\"\")\n shape = tuple(int(num) for num in str(header[60:120], 'utf-8').replace(', }', '').replace('(', '').replace(')', '').split(','))\n datasize = np.lib.format.descr_to_dtype(descr).itemsize\n for dimension in shape:\n datasize *= dimension\n return np.ndarray(shape, dtype=descr, buffer=file.read(datasize))\n```\n\n-----\n\n- [U-Net is missing something](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/413767) By [J€ANMPIA](https://www.kaggle.com/janmpia)\n\n> Spoiler: It doesn't use BatchNorm\n\nIn this post, [J€ANMPIA](https://www.kaggle.com/janmpia) proposes to use batch norm to improve unet models.\n\nCode example:\n\n```\ndef double_conv(in_channels, out_channels):\n return nn.Sequential(\n nn.Conv2d(in_channels, out_channels, 3, padding=1, bias=False), \n nn.BatchNorm2d(out_channels),\n nn.ReLU(inplace=True),\n nn.Conv2d(out_channels, out_channels, 3, padding=1, bias=False), \n nn.BatchNorm2d(out_channels),\n nn.ReLU(inplace=True)\n ) \n``` \n\nThis is true. It usually always help (but you still have to test it out to make sure!)\n\n-----\n\n- [Pytorch Lightning baseline](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/416746) By [Egor Trushin](https://www.kaggle.com/egortrushin)\n\n\n[Egor Trushin](https://www.kaggle.com/egortrushin) shared with us a powerful pytorch lightning pipeline:\n\n- [Notebook](https://www.kaggle.com/code/egortrushin/gr-icrgw-pytorch-lightning-baseline-unet-resnest)\n- [[GR-ICRGW] PL Pipeline Improved](https://www.kaggle.com/code/egortrushin/gr-icrgw-pl-pipeline-improved)\n- [[GR-ICRGW] Training with 4 folds](https://www.kaggle.com/code/egortrushin/gr-icrgw-training-with-4-folds?scriptVersionId=134148499)\n\n\n-----\n\n- [Post-Processing](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/416436) By [MPWARE](https://www.kaggle.com/mpware)\n\nStressing that we have to pay attention to post-processing and it's impact on the score because improving average dice score per image is not the same as improving global dice score.\n\n**Some Ideas from this post:**\n\n- **Dropping small items:** Tried to remove small predicted items below a certain pixel threshold, but found that this improved average dice score per image, but not the global dice score.\n- **Removing masks with small sums:** Attempted to remove any masks where the sum of the mask was less than a certain threshold. This didn't improve the validation score.\n- **Using Morphological Operations:** Experimented with was morphological operations like dilation, opening, closing, and tophat. These are operations that can potentially help to remove noise and small anomalies in the masks, but in this case, they didn't lead to improvement in validation score.\n- **Line Segment Detection:** Mentioned a paper that used OpenCV's LineSegmentDetector for post-processing. This technique might be useful in certain contexts where the objects of interest are line-like or have linear features.\n\nSome comments also tried the following:\n\n- Implementing a drop mask with np.sum(mask) < N, particularly with a threshold of fewer than 11 pixels, which was noted to bring minor improvement.\n- Using morphology operators like dilatation, opening, closing, and tophat. Although different kernel sizes and shapes were experimented with, these methods seemed to negatively impact the score for the most part, with only some bins showing a minimal increase.\n\n-----\n\n- [Increasing image size doesn't work for me on LB](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/420079) By [Tawara](https://www.kaggle.com/ttahara)\n\nIn many CV tasks, bigger image size often gives us higher performance, But for the author, on this competition: bigger image size got higher score on CV but lower score on LB.\n\n> Serveral comments reported the same.\n\n-----\n\n\n- [Trust ur CV even though LB drops](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/420135) By [LUPIN11](https://www.kaggle.com/lupin11)\n\n- On this thread, the author points to significant improvements on a larger dataset may translate into minor improvements or even drops on a smaller subset due to sample size limitations. Therefore, trust your Cross-Validation (CV) even when Leaderboard (LB) drops.\n- The optimal threshold may vary on different datasets or subsets. A model that falls behind on a smaller subset due to fluctuating optimal thresholds may actually have potential to surpass another model by just adjusting the threshold.\n- Experiments on subsets of validation data can explain seemingly trivial LB improvements in relation to more significant CV improvements. This is due to the smaller representation of data on the public LB which may not accurately reflect model performance improvements.\n\n\n-----\n\n\n- [Data Leakage: Duplicate Images and Masks in Training and Validation Sets](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/419994) By [Sergey Saharovskiy](https://www.kaggle.com/sergiosaharovskiy)\n\nPointing out that there are duplicates on the dataset.\n\n\nMight be the same place but different day/time.\n\n-----\n\n\n- [Are 3D models worth?](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/419816) By [MPWARE](https://www.kaggle.com/mpware)\n\nStarting a discussion about 3D models and asking the other competitors if they had any luck with such models. \n- Most competitors reported disappointing results using 3D backbones for image segmentation models, noting that basic 2D backbones often outperformed them.\n- Some reported that adding temporal context in the form of extra channels in a 2D model (2.5D approach) also failed to provide significant improvements.\n- It appears that the choice of encoder has a more significant impact on the results than the decoder, however, more experimentation is needed to fully understand this outcome.\n\n> **Suggested read!**\n\n\n-----\n\n- [Augmentation Analysis](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/420180) By [Balaji Selvaraj](https://www.kaggle.com/dhakshiin1601)\n\nReporting that so far, Randomresizedcrop alone gives improvement.\n \n> [Martin Kovacevic Buvinic](https://www.kaggle.com/ragnar123) Tried flips and rotation augmentations with no luck.\n\n-----\n\n\n- [Some Successful/Fail Experiment](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/414344) By [lyu](https://www.kaggle.com/zhuwanglju)\n\n**Highly Suggested!!**\n\nA very good breakdown of many attempts and experimental results!\n\nA must read for everyone on this competition.\n\n\n-----\n\n\n- [Main Findings after the first two Competition Weeks](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/413068) By [Jan H](https://www.kaggle.com/janhuebi)\n\n**Again: Highly Suggested!!**\n\nAgain: A very good breakdown of the whole competition. A must read in my opinion.\n\n-----\n\n\nThis is it for now.\nI will keep updating this post when new information is discovered.\n\nGood luck on the last month!\n\n-----\n\n> This is not ChatGPT. I **actually** go by hand and read everything. So do call me out if anything I wrote here is inaccurate because I want to know for myself 🙃\n\n\n\n\n\n\n\n\n\n\n", "2329096": "I came into this competition late and this was a great recap. Thanks. ", "2326144": "" } }