dears ,
\n i have an issue with submit result due to Notebook Exceeded Allowed Compute.
\n so how can i solve this issue
\nnote book link is :
\nhttps://www.kaggle.com/code/ahmedgeka/stroke-blood-clot-origin
\nthanks for helping
I have the same problem, when I try to process all test dataset images. This challenge went from being a machine learning problem to being an error management problem in the pre-processing of test data. This seems to be more important to sponsors than an effective rating model.
\nThe error you mentioned seems to be related to the lack of memory to load some test set images, which seem to be too large. The alternatives are to ignore these images or try to read these images in parts. I'm trying to work on this, but I consider a loss of focus generated by the organizers of this challenge.
same happens to me twice. only 5-6 hours.
", "rawMarkdown": "same happens to me twice. only 5-6 hours.", "votes": 1 }, { "id": 1878765, "postDate": "2022-07-31T15:56:21.267Z", "content": "dears ,
\n i have an issue with submit result due to Notebook Exceeded Allowed Compute.
\n so how can i solve this issue
\nnote book link is :
\nhttps://www.kaggle.com/code/ahmedgeka/stroke-blood-clot-origin
\nthanks for helping
@icemantd @ahmedgeka
\nSame issue for us with same happens always. only 7 hours for Train data to complete .
\nAnd there is no issue with the imageProcessing as our algorithm for TILES works without a issue.
\nBut at the submission time only we get the NOTEBOOK-EXCEPTION with no-log /No clear stack trace.
\nIsn't there a mechanism to get the SUBMISSION exception for the submit time notebook.
Thanks.
", "rawMarkdown": "@icemantd @ahmedgeka \nSame issue for us with same happens always. only 7 hours for Train data to complete .\nAnd there is no issue with the imageProcessing as our algorithm for TILES works without a issue. \nBut at the submission time only we get the NOTEBOOK-EXCEPTION with no-log /No clear stack trace.\nIsn't there a mechanism to get the SUBMISSION exception for the submit time notebook.\n\nThanks.", "replies": [ { "id": 1903848, "postDate": "2022-08-17T17:36:00.857Z", "content": "@thushthusinthaka in my experience the error 'Notebook Exceeded Allowed Compute' for the submission case is related to OOM issue. I got it a few times where I also successfully created tiles from training data, but it also depends on your code using memory at inference time for scoring submission. I found that reading an image file, creating tiles, using those tiles for inference, all of this used much more memory than just tile creation.
\nMy solution was to limit file size that I read based on this notebook code block 4. I suggest that you limit the file size to the biggest image size that you are successfully able to run from a sample training dataset. The test dataset max image size is just ~1.3 Gb which may be far lower than the public scoring test dataset. If it works on an image size upto X Gb, it will work for prediction as well, but you have to make sure that your code limits file reading and processing till that point - and that depends on your inference notebook and how it uses RAM.
\nAs for the 'Notebook Threw Exception' error, I had problems with the way I was compiling inference results and writing them to the submission.csv. There I was skipping a few images but not adding their score and probably when I assign prediction scores against patient_id there was an error. When I fixed that, this error went away. So this error is most likely an error in execution of your code at some point - most likely the dataframe/csv part if it happens towards the end/after a long time in your scoring process. I would suggest to look the link in the paragraph above to see how to avoid that, and also check other shared kernels on the LB.
\nThis is my current understanding and hope it helps you to move to the actual fun part :)
", "rawMarkdown": "@thushthusinthaka in my experience the error 'Notebook Exceeded Allowed Compute' for the submission case is related to OOM issue. I got it a few times where I also successfully created tiles from training data, but it also depends on your code using memory at inference time for scoring submission. I found that reading an image file, creating tiles, using those tiles for inference, all of this used much more memory than just tile creation.\n\nMy solution was to limit file size that I read based on [this](https://www.kaggle.com/code/realneuralnetwork/cnn-strip-ai-inference/notebook?scriptVersionId=100707854) notebook code block 4. I suggest that you limit the file size to the biggest image size that you are successfully able to run from a sample training dataset. The test dataset max image size is just ~1.3 Gb which may be far lower than the public scoring test dataset. If it works on an image size upto X Gb, it will work for prediction as well, but you have to make sure that your code limits file reading and processing till that point - and that depends on your inference notebook and how it uses RAM.\n\nAs for the 'Notebook Threw Exception' error, I had problems with the way I was compiling inference results and writing them to the submission.csv. There I was skipping a few images but not adding their score and probably when I assign prediction scores against patient_id there was an error. When I fixed that, this error went away. So this error is most likely an error in execution of your code at some point - most likely the dataframe/csv part if it happens towards the end/after a long time in your scoring process. I would suggest to look the link in the paragraph above to see how to avoid that, and also check other shared kernels on the LB.\n\nThis is my current understanding and hope it helps you to move to the actual fun part :)", "votes": 1 } ] }, { "id": 1881022, "postDate": "2022-08-02T07:35:59.823Z", "content": "@paulorzp @dragonzhang so what we can do
", "rawMarkdown": "@paulorzp @dragonzhang so what we can do", "replies": [ { "id": 1898484, "postDate": "2022-08-14T15:51:12.960Z", "content": "@ahmedgeka check code block 4 in this notebook for a method to avoid this error initially until you can figure out what file size your code can handle depending on its RAM usage… hope this helps you get started
", "rawMarkdown": "@ahmedgeka check code block 4 in [this](https://www.kaggle.com/code/realneuralnetwork/cnn-strip-ai-inference/notebook?scriptVersionId=100707854) notebook for a method to avoid this error initially until you can figure out what file size your code can handle depending on its RAM usage... hope this helps you get started", "votes": 1 } ] } ], "comments": [ { "id": 1880706, "author_name": "Paulo Pinto", "author_url": "", "post_date": "2022-08-02T01:24:54.030000", "content": "I have the same problem, when I try to process all test dataset images. This challenge went from being a machine learning problem to being an error management problem in the pre-processing of test data. This seems to be more important to sponsors than an effective rating model.
\nThe error you mentioned seems to be related to the lack of memory to load some test set images, which seem to be too large. The alternatives are to ignore these images or try to read these images in parts. I'm trying to work on this, but I consider a loss of focus generated by the organizers of this challenge.
same happens to me twice. only 5-6 hours.
", "votes": 1, "replies": [] }, { "id": 1903001, "author_name": "Thush_Thusinthaka", "author_url": "", "post_date": "2022-08-17T03:43:48.097000", "content": "@icemantd @ahmedgeka
\nSame issue for us with same happens always. only 7 hours for Train data to complete .
\nAnd there is no issue with the imageProcessing as our algorithm for TILES works without a issue.
\nBut at the submission time only we get the NOTEBOOK-EXCEPTION with no-log /No clear stack trace.
\nIsn't there a mechanism to get the SUBMISSION exception for the submit time notebook.
Thanks.
", "votes": 0, "replies": [ { "id": 1903848, "author_name": "tdiceman", "author_url": "", "post_date": "2022-08-17T17:36:00.857000", "content": "@thushthusinthaka in my experience the error 'Notebook Exceeded Allowed Compute' for the submission case is related to OOM issue. I got it a few times where I also successfully created tiles from training data, but it also depends on your code using memory at inference time for scoring submission. I found that reading an image file, creating tiles, using those tiles for inference, all of this used much more memory than just tile creation.
\nMy solution was to limit file size that I read based on this notebook code block 4. I suggest that you limit the file size to the biggest image size that you are successfully able to run from a sample training dataset. The test dataset max image size is just ~1.3 Gb which may be far lower than the public scoring test dataset. If it works on an image size upto X Gb, it will work for prediction as well, but you have to make sure that your code limits file reading and processing till that point - and that depends on your inference notebook and how it uses RAM.
\nAs for the 'Notebook Threw Exception' error, I had problems with the way I was compiling inference results and writing them to the submission.csv. There I was skipping a few images but not adding their score and probably when I assign prediction scores against patient_id there was an error. When I fixed that, this error went away. So this error is most likely an error in execution of your code at some point - most likely the dataframe/csv part if it happens towards the end/after a long time in your scoring process. I would suggest to look the link in the paragraph above to see how to avoid that, and also check other shared kernels on the LB.
\nThis is my current understanding and hope it helps you to move to the actual fun part :)
", "votes": 1, "replies": [] } ] }, { "id": 1881022, "author_name": "Ahmed___Gaber____", "author_url": "", "post_date": "2022-08-02T07:35:59.823000", "content": "@paulorzp @dragonzhang so what we can do
", "votes": 0, "replies": [ { "id": 1898484, "author_name": "tdiceman", "author_url": "", "post_date": "2022-08-14T15:51:12.960000", "content": "@ahmedgeka check code block 4 in this notebook for a method to avoid this error initially until you can figure out what file size your code can handle depending on its RAM usage… hope this helps you get started
", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "1880706": "I have the same problem, when I try to process all test dataset images. This challenge went from being a machine learning problem to being an error management problem in the pre-processing of test data. This seems to be more important to sponsors than an effective rating model.\nThe error you mentioned seems to be related to the lack of memory to load some test set images, which seem to be too large. The alternatives are to ignore these images or try to read these images in parts. I'm trying to work on this, but I consider a loss of focus generated by the organizers of this challenge.", "1879275": "same happens to me twice. only 5-6 hours.", "1878765": "dears ,\n i have an issue with submit result due to Notebook Exceeded Allowed Compute.\n so how can i solve this issue \nnote book link is :\n[https://www.kaggle.com/code/ahmedgeka/stroke-blood-clot-origin](url)\nthanks for helping", "1903001": "@icemantd @ahmedgeka \nSame issue for us with same happens always. only 7 hours for Train data to complete .\nAnd there is no issue with the imageProcessing as our algorithm for TILES works without a issue. \nBut at the submission time only we get the NOTEBOOK-EXCEPTION with no-log /No clear stack trace.\nIsn't there a mechanism to get the SUBMISSION exception for the submit time notebook.\n\nThanks.", "1881022": "@paulorzp @dragonzhang so what we can do" } }