{ "id": 361693, "title": "Target Variable ~ [0, 1] ?", "url": "/competitions/g2net-detecting-continuous-gravitational-waves/discussion/361693", "author_name": "goto_conversion", "post_date": "2022-10-23T07:06:45.453000", "votes": 3, "comment_count": 10, "views": 0, "content": "

Hello, this is my first day working on this competition for me and I am still trying to properly understand the problem.

\n

Some notebooks' submission files have negative values on the target variable column, but I thought we were predicting probabilities so our target variable column's values will be bounded between 0 and 1?

\n

Update:
\n@junkoda has clarified this issue. Thanks!
\nhttps://www.kaggle.com/competitions/g2net-detecting-continuous-gravitational-waves/discussion/361693#2001301

", "messages": [ { "id": 2000249, "postDate": "2022-10-23T07:06:45.453Z", "content": "

Hello, this is my first day working on this competition for me and I am still trying to properly understand the problem.

\n

Some notebooks' submission files have negative values on the target variable column, but I thought we were predicting probabilities so our target variable column's values will be bounded between 0 and 1?

\n

Update:
\n@junkoda has clarified this issue. Thanks!
\nhttps://www.kaggle.com/competitions/g2net-detecting-continuous-gravitational-waves/discussion/361693#2001301

", "rawMarkdown": "Hello, this is my first day working on this competition for me and I am still trying to properly understand the problem.\n\nSome notebooks' submission files have negative values on the target variable column, but I thought we were predicting probabilities so our target variable column's values will be bounded between 0 and 1?\n\nUpdate:\n@junkoda has clarified this issue. Thanks!\nhttps://www.kaggle.com/competitions/g2net-detecting-continuous-gravitational-waves/discussion/361693#2001301", "votes": 3 }, { "id": 2001730, "postDate": "2022-10-24T09:14:10.920Z", "content": "

I think the evaluation indicator is auc, which is related to the relative size of the value. For example, subtracting a number at the same time makes the predicted value negative, but auc remains unchanged.
\n\"\"

", "rawMarkdown": "I think the evaluation indicator is auc, which is related to the relative size of the value. For example, subtracting a number at the same time makes the predicted value negative, but auc remains unchanged.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6803056%2F9d884e4865a84ab29422b33003167a17%2F2022-10-24%2017.13.24.png?generation=1666602837657735&alt=media)", "votes": 1 }, { "id": 2000886, "postDate": "2022-10-23T15:40:25.330Z", "content": "

I have the same confusion. Thanks to the answerer.👍

", "rawMarkdown": "I have the same confusion. Thanks to the answerer.👍", "votes": 1 }, { "id": 2001301, "postDate": "2022-10-23T23:05:03.530Z", "content": "

I forgot to apply sigmoid function in my public notebook, in which the model outputs logits. PyTorch tensor can be mapped to [0, 1] with

\n
y_pred.sigmoid()\n
\n

The score, roc_auc, only depends on the order of the predictions if you think how the ROC curve are drawn. Applying monotonically increasing function should not change the score at all.

\n

Still, I agree that the target values should be in [0, 1].

", "rawMarkdown": "I forgot to apply sigmoid function in my public notebook, in which the model outputs logits. PyTorch tensor can be mapped to [0, 1] with\n\n```\ny_pred.sigmoid()\n```\n\nThe score, roc_auc, only depends on the *order* of the predictions if you think how the ROC curve are drawn. Applying monotonically increasing function should not change the score at all.\n\nStill, I agree that the target values should be in [0, 1].\n", "votes": 2 }, { "id": 2000821, "postDate": "2022-10-23T14:53:13.293Z", "content": "

Always helps to fully read the information provided on the data page.

\n
\n

train_labels.csv - a file containing the target labels; 1 if the data contains the presence of a gravitational wave, 0 otherwise. (Please note the presence of a small number of files labeled -1. Physicists are currently unable to determine the status of these files.)>

\n
", "rawMarkdown": "Always helps to fully read the information provided on the data page.\n\n> train_labels.csv - a file containing the target labels; 1 if the data contains the presence of a gravitational wave, 0 otherwise. (Please note the presence of a small number of files labeled -1. Physicists are currently unable to determine the status of these files.)> ", "replies": [ { "id": 2001743, "postDate": "2022-10-24T09:25:54.717Z", "content": "

When I say \"submission files\" I'm talking about submission.csv not train_labels.csv

", "rawMarkdown": "When I say \"submission files\" I'm talking about submission.csv not train_labels.csv" }, { "id": 2002369, "postDate": "2022-10-24T18:30:05.960Z", "content": "

The notebooks that you have seen could be trained on all the labels and also include the -1 into the prediction which should not be the case.

", "rawMarkdown": "The notebooks that you have seen could be trained on all the labels and also include the -1 into the prediction which should not be the case. " } ] }, { "id": 2000283, "postDate": "2022-10-23T07:23:55.677Z", "content": "

Yes, your prediction should be in the real range [0.0, 1.0] as it represents a probability, and it will be evaluated with the AUC of the ROC, against the ground truth which is a binary label [0, 1]

", "rawMarkdown": "Yes, your prediction should be in the real range `[0.0, 1.0]` as it represents a probability, and it will be evaluated with the AUC of the ROC, against the ground truth which is a binary label [0, 1]", "replies": [ { "id": 2000300, "postDate": "2022-10-23T07:29:13.220Z", "content": "

Thanks, that's what I initially thought but then why does the notebook below have negative values on the 'target' coulumn?
\nhttps://www.kaggle.com/code/junkoda/basic-spectrogram-image-classification/data?select=submission.csv

", "rawMarkdown": "Thanks, that's what I initially thought but then why does the notebook below have negative values on the 'target' coulumn?\nhttps://www.kaggle.com/code/junkoda/basic-spectrogram-image-classification/data?select=submission.csv" }, { "id": 2000332, "postDate": "2022-10-23T07:51:18.877Z", "content": "

My guess is that those are raw logits (-inf, inf) which have not been normalized to [0, 1] and maybe the kaggle scoring function is smart enough to do the conversion…
\nIf you were curious enough, you can download the submission.csv, normalize it to [0, 1] and submit it yourself to see if you get the same score.
\nThis is not a code competition, which means you can just upload a submission.csv file directly, no need for an actual notebook that generates it.

", "rawMarkdown": "My guess is that those are raw logits (-inf, inf) which have not been normalized to [0, 1] and maybe the kaggle scoring function is smart enough to do the conversion...\nIf you were curious enough, you can download the submission.csv, normalize it to [0, 1] and submit it yourself to see if you get the same score.\nThis is not a code competition, which means you can just upload a submission.csv file directly, no need for an actual notebook that generates it." }, { "id": 2000335, "postDate": "2022-10-23T07:54:51.453Z", "content": "

Thanks @darkbitur! Yes I think that will be the best way to validate the story behind the negative values

", "rawMarkdown": "Thanks @darkbitur! Yes I think that will be the best way to validate the story behind the negative values", "votes": 1 } ] } ], "comments": [ { "id": 2001730, "author_name": "zh", "author_url": "", "post_date": "2022-10-24T09:14:10.920000", "content": "

I think the evaluation indicator is auc, which is related to the relative size of the value. For example, subtracting a number at the same time makes the predicted value negative, but auc remains unchanged.
\n\"\"

", "votes": 1, "replies": [] }, { "id": 2000886, "author_name": "BooHuu", "author_url": "", "post_date": "2022-10-23T15:40:25.330000", "content": "

I have the same confusion. Thanks to the answerer.👍

", "votes": 1, "replies": [] }, { "id": 2001301, "author_name": "🐢 Jun Koda", "author_url": "", "post_date": "2022-10-23T23:05:03.530000", "content": "

I forgot to apply sigmoid function in my public notebook, in which the model outputs logits. PyTorch tensor can be mapped to [0, 1] with

\n
y_pred.sigmoid()\n
\n

The score, roc_auc, only depends on the order of the predictions if you think how the ROC curve are drawn. Applying monotonically increasing function should not change the score at all.

\n

Still, I agree that the target values should be in [0, 1].

", "votes": 2, "replies": [] }, { "id": 2000821, "author_name": "PC Jimmmy", "author_url": "", "post_date": "2022-10-23T14:53:13.293000", "content": "

Always helps to fully read the information provided on the data page.

\n
\n

train_labels.csv - a file containing the target labels; 1 if the data contains the presence of a gravitational wave, 0 otherwise. (Please note the presence of a small number of files labeled -1. Physicists are currently unable to determine the status of these files.)>

\n
", "votes": 0, "replies": [ { "id": 2001743, "author_name": "goto_conversion", "author_url": "", "post_date": "2022-10-24T09:25:54.717000", "content": "

When I say \"submission files\" I'm talking about submission.csv not train_labels.csv

", "votes": 0, "replies": [] }, { "id": 2002369, "author_name": "Ali Abdin", "author_url": "", "post_date": "2022-10-24T18:30:05.960000", "content": "

The notebooks that you have seen could be trained on all the labels and also include the -1 into the prediction which should not be the case.

", "votes": 0, "replies": [] } ] }, { "id": 2000283, "author_name": "Victor Gonzalez", "author_url": "", "post_date": "2022-10-23T07:23:55.677000", "content": "

Yes, your prediction should be in the real range [0.0, 1.0] as it represents a probability, and it will be evaluated with the AUC of the ROC, against the ground truth which is a binary label [0, 1]

", "votes": 0, "replies": [ { "id": 2000300, "author_name": "goto_conversion", "author_url": "", "post_date": "2022-10-23T07:29:13.220000", "content": "

Thanks, that's what I initially thought but then why does the notebook below have negative values on the 'target' coulumn?
\nhttps://www.kaggle.com/code/junkoda/basic-spectrogram-image-classification/data?select=submission.csv

", "votes": 0, "replies": [] }, { "id": 2000332, "author_name": "Victor Gonzalez", "author_url": "", "post_date": "2022-10-23T07:51:18.877000", "content": "

My guess is that those are raw logits (-inf, inf) which have not been normalized to [0, 1] and maybe the kaggle scoring function is smart enough to do the conversion…
\nIf you were curious enough, you can download the submission.csv, normalize it to [0, 1] and submit it yourself to see if you get the same score.
\nThis is not a code competition, which means you can just upload a submission.csv file directly, no need for an actual notebook that generates it.

", "votes": 0, "replies": [] }, { "id": 2000335, "author_name": "goto_conversion", "author_url": "", "post_date": "2022-10-23T07:54:51.453000", "content": "

Thanks @darkbitur! Yes I think that will be the best way to validate the story behind the negative values

", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "2000249": "Hello, this is my first day working on this competition for me and I am still trying to properly understand the problem.\n\nSome notebooks' submission files have negative values on the target variable column, but I thought we were predicting probabilities so our target variable column's values will be bounded between 0 and 1?\n\nUpdate:\n@junkoda has clarified this issue. Thanks!\nhttps://www.kaggle.com/competitions/g2net-detecting-continuous-gravitational-waves/discussion/361693#2001301", "2001730": "I think the evaluation indicator is auc, which is related to the relative size of the value. For example, subtracting a number at the same time makes the predicted value negative, but auc remains unchanged.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6803056%2F9d884e4865a84ab29422b33003167a17%2F2022-10-24%2017.13.24.png?generation=1666602837657735&alt=media)", "2000886": "I have the same confusion. Thanks to the answerer.👍", "2001301": "I forgot to apply sigmoid function in my public notebook, in which the model outputs logits. PyTorch tensor can be mapped to [0, 1] with\n\n```\ny_pred.sigmoid()\n```\n\nThe score, roc_auc, only depends on the *order* of the predictions if you think how the ROC curve are drawn. Applying monotonically increasing function should not change the score at all.\n\nStill, I agree that the target values should be in [0, 1].\n", "2000821": "Always helps to fully read the information provided on the data page.\n\n> train_labels.csv - a file containing the target labels; 1 if the data contains the presence of a gravitational wave, 0 otherwise. (Please note the presence of a small number of files labeled -1. Physicists are currently unable to determine the status of these files.)> ", "2000283": "Yes, your prediction should be in the real range `[0.0, 1.0]` as it represents a probability, and it will be evaluated with the AUC of the ROC, against the ground truth which is a binary label [0, 1]" } }