{
  "id": 353217,
  "title": "[Important Resource for everyone] Tiled and cleaned dataset for training",
  "url": "/competitions/mayo-clinic-strip-ai/discussion/353217",
  "author_name": "Mrinal Tyagi",
  "post_date": "2022-09-17T11:32:22.619000",
  "votes": 21,
  "comment_count": 17,
  "views": 0,
  "content": "<p>Hello everyone, do check out my new dataset regarding MAYO Clinic - STRIP AI competition. <br>\nWould love to encourage everyone to use the dataset for solving the problem which we couldn't due to resource constraints. </p>\n<p>Dataset Link -&gt; <a href=\"https://www.kaggle.com/datasets/tr1gg3rtrash/mayo-clinic\" target=\"_blank\">Link</a></p>\n<p>Do upvote if the dataset helped you in any form. </p>",
  "messages": [
    {
      "id": 1943219,
      "postDate": "2022-09-17T11:32:22.620Z",
      "content": "<p>Hello everyone, do check out my new dataset regarding MAYO Clinic - STRIP AI competition. <br>\nWould love to encourage everyone to use the dataset for solving the problem which we couldn't due to resource constraints. </p>\n<p>Dataset Link -&gt; <a href=\"https://www.kaggle.com/datasets/tr1gg3rtrash/mayo-clinic\" target=\"_blank\">Link</a></p>\n<p>Do upvote if the dataset helped you in any form. </p>",
      "rawMarkdown": "Hello everyone, do check out my new dataset regarding MAYO Clinic - STRIP AI competition. \nWould love to encourage everyone to use the dataset for solving the problem which we couldn't due to resource constraints. \n\nDataset Link -> [Link](https://www.kaggle.com/datasets/tr1gg3rtrash/mayo-clinic)\n\nDo upvote if the dataset helped you in any form. ",
      "votes": 21
    },
    {
      "id": 1956096,
      "postDate": "2022-09-26T09:22:50.717Z",
      "content": "<p>Great work <a href=\"https://www.kaggle.com/tr1gg3rtrash\" target=\"_blank\">@tr1gg3rtrash</a></p>",
      "rawMarkdown": "Great work @tr1gg3rtrash",
      "votes": 1
    },
    {
      "id": 1955607,
      "postDate": "2022-09-26T04:26:59.677Z",
      "content": "<p>Great work <a href=\"https://www.kaggle.com/tr1gg3rtrash\" target=\"_blank\">@tr1gg3rtrash</a> </p>",
      "rawMarkdown": "Great work @tr1gg3rtrash ",
      "votes": 1
    },
    {
      "id": 1953428,
      "postDate": "2022-09-24T13:28:44.437Z",
      "content": "<p>That's really good work.</p>",
      "rawMarkdown": "That's really good work.",
      "votes": 1
    },
    {
      "id": 1950741,
      "postDate": "2022-09-22T14:07:09.170Z",
      "content": "<p>nice work <a href=\"https://www.kaggle.com/tr1gg3rtrash\" target=\"_blank\">@tr1gg3rtrash</a> </p>",
      "rawMarkdown": "nice work @tr1gg3rtrash ",
      "votes": 1
    },
    {
      "id": 1943255,
      "postDate": "2022-09-17T12:07:39.227Z",
      "content": "<p>Great work <a href=\"https://www.kaggle.com/tr1gg3rtrash\" target=\"_blank\">@tr1gg3rtrash</a> </p>",
      "rawMarkdown": "Great work @tr1gg3rtrash ",
      "votes": 1,
      "replies": [
        {
          "id": 1943267,
          "postDate": "2022-09-17T12:15:06.803Z",
          "content": "<p>Thank you for the positive feedback. Really appreciated. </p>",
          "rawMarkdown": "Thank you for the positive feedback. Really appreciated. "
        }
      ]
    },
    {
      "id": 1959926,
      "postDate": "2022-09-28T11:49:04.627Z",
      "content": "<p>Does training on this dataset and submission give error(notebook threw exception)? anybody faced that?</p>",
      "rawMarkdown": "Does training on this dataset and submission give error(notebook threw exception)? anybody faced that?\n",
      "votes": 2,
      "replies": [
        {
          "id": 1959967,
          "postDate": "2022-09-28T12:24:18.200Z",
          "content": "<p>Try working in chunks of data rather than complete data at once. As the images produced are a lot, hence the handling for that has to be done in order to prevent ram exceeding issues. Do checkout this notebook. <a href=\"https://www.kaggle.com/code/tr1gg3rtrash/mayo-clinic-preprocessing-notebook\" target=\"_blank\">https://www.kaggle.com/code/tr1gg3rtrash/mayo-clinic-preprocessing-notebook</a> . You might need to structure your submission with respect to it. </p>",
          "rawMarkdown": "Try working in chunks of data rather than complete data at once. As the images produced are a lot, hence the handling for that has to be done in order to prevent ram exceeding issues. Do checkout this notebook. https://www.kaggle.com/code/tr1gg3rtrash/mayo-clinic-preprocessing-notebook . You might need to structure your submission with respect to it. "
        }
      ]
    },
    {
      "id": 1943478,
      "postDate": "2022-09-17T14:41:05.850Z",
      "content": "<p>Can you show us how to do your preprocessing, because I guess we need to do exactly the same thing for the test set during inference. <br>\nThanks in advance for sharing </p>",
      "rawMarkdown": "Can you show us how to do your preprocessing, because I guess we need to do exactly the same thing for the test set during inference. \nThanks in advance for sharing ",
      "votes": 2,
      "replies": [
        {
          "id": 1943649,
          "postDate": "2022-09-17T17:03:44.843Z",
          "content": "<p>I am thinking of uploading tile dataset for the test set as well. Do let me know if that solves the issue</p>",
          "rawMarkdown": "I am thinking of uploading tile dataset for the test set as well. Do let me know if that solves the issue"
        },
        {
          "id": 1944257,
          "postDate": "2022-09-18T07:10:34.383Z",
          "content": "<p>It doesn't, because the real test set is hidden. We would need your pre-processing code and run it on the hidden test set before doing the inference (within 9 hr timeline) . </p>",
          "rawMarkdown": "It doesn't, because the real test set is hidden. We would need your pre-processing code and run it on the hidden test set before doing the inference (within 9 hr timeline) . ",
          "votes": 2
        },
        {
          "id": 1944259,
          "postDate": "2022-09-18T07:12:08.647Z",
          "content": "<p>Sure will upload the notebook for the same. </p>",
          "rawMarkdown": "Sure will upload the notebook for the same. ",
          "votes": 1
        },
        {
          "id": 1944723,
          "postDate": "2022-09-18T14:28:12.623Z",
          "content": "<p><a href=\"https://www.kaggle.com/nyleve\" target=\"_blank\">@nyleve</a> Please do check the notebook for this preprocessing. </p>\n<p><a href=\"https://www.kaggle.com/code/tr1gg3rtrash/mayo-clinic-preprocessing-notebook\" target=\"_blank\">https://www.kaggle.com/code/tr1gg3rtrash/mayo-clinic-preprocessing-notebook</a></p>",
          "rawMarkdown": "@nyleve Please do check the notebook for this preprocessing. \n\nhttps://www.kaggle.com/code/tr1gg3rtrash/mayo-clinic-preprocessing-notebook",
          "votes": 1
        }
      ]
    },
    {
      "id": 1967392,
      "postDate": "2022-10-02T13:30:47.327Z",
      "content": "<p>Please refer to the following inferencing pipeline notebook if you are using this dataset: <a href=\"https://www.kaggle.com/tr1gg3rtrash/mayo-clinic-tiled-inference\" target=\"_blank\">https://www.kaggle.com/tr1gg3rtrash/mayo-clinic-tiled-inference</a></p>",
      "rawMarkdown": "Please refer to the following inferencing pipeline notebook if you are using this dataset: https://www.kaggle.com/tr1gg3rtrash/mayo-clinic-tiled-inference"
    },
    {
      "id": 1961380,
      "postDate": "2022-09-29T06:12:18.103Z",
      "content": "<p>Please check out this dataset as well. <a href=\"https://www.kaggle.com/datasets/tr1gg3rtrash/mayo-clinic-strip-ai-normalized-dataset\" target=\"_blank\">https://www.kaggle.com/datasets/tr1gg3rtrash/mayo-clinic-strip-ai-normalized-dataset</a> </p>\n<p>It is a normalized version of the same dataset. </p>",
      "rawMarkdown": "Please check out this dataset as well. https://www.kaggle.com/datasets/tr1gg3rtrash/mayo-clinic-strip-ai-normalized-dataset \n\nIt is a normalized version of the same dataset. "
    },
    {
      "id": 1948210,
      "postDate": "2022-09-20T23:37:04.430Z",
      "content": "<p>Excellent contribution!!! but I don't know how to use such an amount of images .</p>\n<p>Thanks in advance</p>",
      "rawMarkdown": "Excellent contribution!!! but I don't know how to use such an amount of images .\n\nThanks in advance",
      "replies": [
        {
          "id": 1949227,
          "postDate": "2022-09-21T15:23:37.213Z",
          "content": "<p>If storage space is your issue - try Google Colab. They have loads of storage space :)<br>\nYou can download this dataset with a simple CLI command and import it there.</p>",
          "rawMarkdown": "If storage space is your issue - try Google Colab. They have loads of storage space :)\nYou can download this dataset with a simple CLI command and import it there."
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 1956096,
      "author_name": "tang xia",
      "author_url": "",
      "post_date": "2022-09-26T09:22:50.717000",
      "content": "<p>Great work <a href=\"https://www.kaggle.com/tr1gg3rtrash\" target=\"_blank\">@tr1gg3rtrash</a></p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1955607,
      "author_name": "Konika Rani",
      "author_url": "",
      "post_date": "2022-09-26T04:26:59.677000",
      "content": "<p>Great work <a href=\"https://www.kaggle.com/tr1gg3rtrash\" target=\"_blank\">@tr1gg3rtrash</a> </p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1953428,
      "author_name": "Syed Md Danish E Azam",
      "author_url": "",
      "post_date": "2022-09-24T13:28:44.437000",
      "content": "<p>That's really good work.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1950741,
      "author_name": "Serkan Polat",
      "author_url": "",
      "post_date": "2022-09-22T14:07:09.170000",
      "content": "<p>nice work <a href=\"https://www.kaggle.com/tr1gg3rtrash\" target=\"_blank\">@tr1gg3rtrash</a> </p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1943255,
      "author_name": "Sugata Ghosh, PhD",
      "author_url": "",
      "post_date": "2022-09-17T12:07:39.227000",
      "content": "<p>Great work <a href=\"https://www.kaggle.com/tr1gg3rtrash\" target=\"_blank\">@tr1gg3rtrash</a> </p>",
      "votes": 1,
      "replies": [
        {
          "id": 1943267,
          "author_name": "Mrinal Tyagi",
          "author_url": "",
          "post_date": "2022-09-17T12:15:06.803000",
          "content": "<p>Thank you for the positive feedback. Really appreciated. </p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1959926,
      "author_name": "Abu Bakar",
      "author_url": "",
      "post_date": "2022-09-28T11:49:04.627000",
      "content": "<p>Does training on this dataset and submission give error(notebook threw exception)? anybody faced that?</p>",
      "votes": 2,
      "replies": [
        {
          "id": 1959967,
          "author_name": "Mrinal Tyagi",
          "author_url": "",
          "post_date": "2022-09-28T12:24:18.200000",
          "content": "<p>Try working in chunks of data rather than complete data at once. As the images produced are a lot, hence the handling for that has to be done in order to prevent ram exceeding issues. Do checkout this notebook. <a href=\"https://www.kaggle.com/code/tr1gg3rtrash/mayo-clinic-preprocessing-notebook\" target=\"_blank\">https://www.kaggle.com/code/tr1gg3rtrash/mayo-clinic-preprocessing-notebook</a> . You might need to structure your submission with respect to it. </p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1943478,
      "author_name": "yukiya",
      "author_url": "",
      "post_date": "2022-09-17T14:41:05.850000",
      "content": "<p>Can you show us how to do your preprocessing, because I guess we need to do exactly the same thing for the test set during inference. <br>\nThanks in advance for sharing </p>",
      "votes": 2,
      "replies": [
        {
          "id": 1943649,
          "author_name": "Mrinal Tyagi",
          "author_url": "",
          "post_date": "2022-09-17T17:03:44.843000",
          "content": "<p>I am thinking of uploading tile dataset for the test set as well. Do let me know if that solves the issue</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1944257,
          "author_name": "yukiya",
          "author_url": "",
          "post_date": "2022-09-18T07:10:34.383000",
          "content": "<p>It doesn't, because the real test set is hidden. We would need your pre-processing code and run it on the hidden test set before doing the inference (within 9 hr timeline) . </p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1944259,
          "author_name": "Mrinal Tyagi",
          "author_url": "",
          "post_date": "2022-09-18T07:12:08.647000",
          "content": "<p>Sure will upload the notebook for the same. </p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1944723,
          "author_name": "Mrinal Tyagi",
          "author_url": "",
          "post_date": "2022-09-18T14:28:12.623000",
          "content": "<p><a href=\"https://www.kaggle.com/nyleve\" target=\"_blank\">@nyleve</a> Please do check the notebook for this preprocessing. </p>\n<p><a href=\"https://www.kaggle.com/code/tr1gg3rtrash/mayo-clinic-preprocessing-notebook\" target=\"_blank\">https://www.kaggle.com/code/tr1gg3rtrash/mayo-clinic-preprocessing-notebook</a></p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1967392,
      "author_name": "Mrinal Tyagi",
      "author_url": "",
      "post_date": "2022-10-02T13:30:47.327000",
      "content": "<p>Please refer to the following inferencing pipeline notebook if you are using this dataset: <a href=\"https://www.kaggle.com/tr1gg3rtrash/mayo-clinic-tiled-inference\" target=\"_blank\">https://www.kaggle.com/tr1gg3rtrash/mayo-clinic-tiled-inference</a></p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1961380,
      "author_name": "Mrinal Tyagi",
      "author_url": "",
      "post_date": "2022-09-29T06:12:18.103000",
      "content": "<p>Please check out this dataset as well. <a href=\"https://www.kaggle.com/datasets/tr1gg3rtrash/mayo-clinic-strip-ai-normalized-dataset\" target=\"_blank\">https://www.kaggle.com/datasets/tr1gg3rtrash/mayo-clinic-strip-ai-normalized-dataset</a> </p>\n<p>It is a normalized version of the same dataset. </p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1948210,
      "author_name": "Pablo Larrosa",
      "author_url": "",
      "post_date": "2022-09-20T23:37:04.430000",
      "content": "<p>Excellent contribution!!! but I don't know how to use such an amount of images .</p>\n<p>Thanks in advance</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1949227,
          "author_name": "David Landup",
          "author_url": "",
          "post_date": "2022-09-21T15:23:37.213000",
          "content": "<p>If storage space is your issue - try Google Colab. They have loads of storage space :)<br>\nYou can download this dataset with a simple CLI command and import it there.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1943219": "Hello everyone, do check out my new dataset regarding MAYO Clinic - STRIP AI competition. \nWould love to encourage everyone to use the dataset for solving the problem which we couldn't due to resource constraints. \n\nDataset Link -> [Link](https://www.kaggle.com/datasets/tr1gg3rtrash/mayo-clinic)\n\nDo upvote if the dataset helped you in any form. ",
    "1956096": "Great work @tr1gg3rtrash",
    "1955607": "Great work @tr1gg3rtrash ",
    "1953428": "That's really good work.",
    "1950741": "nice work @tr1gg3rtrash ",
    "1943255": "Great work @tr1gg3rtrash ",
    "1959926": "Does training on this dataset and submission give error(notebook threw exception)? anybody faced that?\n",
    "1943478": "Can you show us how to do your preprocessing, because I guess we need to do exactly the same thing for the test set during inference. \nThanks in advance for sharing ",
    "1967392": "Please refer to the following inferencing pipeline notebook if you are using this dataset: https://www.kaggle.com/tr1gg3rtrash/mayo-clinic-tiled-inference",
    "1961380": "Please check out this dataset as well. https://www.kaggle.com/datasets/tr1gg3rtrash/mayo-clinic-strip-ai-normalized-dataset \n\nIt is a normalized version of the same dataset. ",
    "1948210": "Excellent contribution!!! but I don't know how to use such an amount of images .\n\nThanks in advance"
  }
}