{
  "id": 355735,
  "title": "Almost all images in the private set are larger than 1.3 gb",
  "url": "/competitions/mayo-clinic-strip-ai/discussion/355735",
  "author_name": "Pierre Tisseur",
  "post_date": "2022-09-28T00:01:14.120000",
  "votes": 6,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I have tried something like : <br>\ntry:<br>\n        size = os.path.getsize(img_path)<br>\n    except :<br>\n        size = 2.8<em>1e9\n    print('Size = ',size)\n    if size&gt; 1.32</em>1e9:<br>\n        pred = 0.5<br>\n    else:<br>\n        pred=process(img_path)<br>\nThe submission took 2 minutes and succeed and gave me an LB = 0.6 which is the LB for equal probability 0.5.<br>\nMaybe this is known?</p>",
  "messages": [
    {
      "id": 1959104,
      "postDate": "2022-09-28T00:01:14.120Z",
      "content": "<p>I have tried something like : <br>\ntry:<br>\n        size = os.path.getsize(img_path)<br>\n    except :<br>\n        size = 2.8<em>1e9\n    print('Size = ',size)\n    if size&gt; 1.32</em>1e9:<br>\n        pred = 0.5<br>\n    else:<br>\n        pred=process(img_path)<br>\nThe submission took 2 minutes and succeed and gave me an LB = 0.6 which is the LB for equal probability 0.5.<br>\nMaybe this is known?</p>",
      "rawMarkdown": "I have tried something like : \ntry:\n        size = os.path.getsize(img_path)\n    except :\n        size = 2.8*1e9\n    print('Size = ',size)\n    if size> 1.32*1e9:\n        pred = 0.5\n    else:\n        pred=process(img_path)\nThe submission took 2 minutes and succeed and gave me an LB = 0.6 which is the LB for equal probability 0.5.\nMaybe this is known?",
      "votes": 6
    },
    {
      "id": 1963221,
      "postDate": "2022-09-30T06:55:55.473Z",
      "content": "<p>Thank you for this information.</p>\n<p>I have tried<br>\nif size&lt;1.3e9:<br>\n    then do things appropriately<br>\nelse:<br>\n    pred = 0.5</p>\n<p>but when submitting notebook threw exception. Do you have any idea how to approach it with getting the error.</p>",
      "rawMarkdown": "Thank you for this information.\n\nI have tried\nif size<1.3e9:\n    then do things appropriately\nelse:\n    pred = 0.5\n\nbut when submitting notebook threw exception. Do you have any idea how to approach it with getting the error.\n",
      "votes": 1
    },
    {
      "id": 1960157,
      "postDate": "2022-09-28T14:05:42.340Z",
      "content": "<p>Thanks for running this test. </p>\n<p>It would have been nice if the competition sponsors included the largest image in the 'public' version of the test set. This way we would know if the notebook instance will run out of memory. Probably would have saved a lot of headaches.</p>",
      "rawMarkdown": "Thanks for running this test. \n\nIt would have been nice if the competition sponsors included the largest image in the 'public' version of the test set. This way we would know if the notebook instance will run out of memory. Probably would have saved a lot of headaches.",
      "votes": 2
    },
    {
      "id": 1961602,
      "postDate": "2022-09-29T08:59:42.080Z",
      "content": "<p>I have doubts if this is true. <br>\nI have submitted a notebook with 512 Mb threshold and the total time was about 4. Next, I have submitted the same solution, but with 1 Gb threshold and the total time was about 6h. <br>\nTotal time of the same solution on the test directory (4 tifs) is about 20 minutes. <br>\nIn my opinion it clearly shows, that there are lot of images under 1.2 Gb. I would suspect, that the distribution of the hidden test set should be similar to the train set one (i. e. 95% of images are of lower than 512 Mb). </p>",
      "rawMarkdown": "I have doubts if this is true. \nI have submitted a notebook with 512 Mb threshold and the total time was about 4. Next, I have submitted the same solution, but with 1 Gb threshold and the total time was about 6h. \nTotal time of the same solution on the test directory (4 tifs) is about 20 minutes. \nIn my opinion it clearly shows, that there are lot of images under 1.2 Gb. I would suspect, that the distribution of the hidden test set should be similar to the train set one (i. e. 95% of images are of lower than 512 Mb). "
    }
  ],
  "comments": [
    {
      "id": 1963221,
      "author_name": "Abu Bakar",
      "author_url": "",
      "post_date": "2022-09-30T06:55:55.473000",
      "content": "<p>Thank you for this information.</p>\n<p>I have tried<br>\nif size&lt;1.3e9:<br>\n    then do things appropriately<br>\nelse:<br>\n    pred = 0.5</p>\n<p>but when submitting notebook threw exception. Do you have any idea how to approach it with getting the error.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1960157,
      "author_name": "Joe Marturano",
      "author_url": "",
      "post_date": "2022-09-28T14:05:42.340000",
      "content": "<p>Thanks for running this test. </p>\n<p>It would have been nice if the competition sponsors included the largest image in the 'public' version of the test set. This way we would know if the notebook instance will run out of memory. Probably would have saved a lot of headaches.</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 1961602,
      "author_name": "JanGlinko2",
      "author_url": "",
      "post_date": "2022-09-29T08:59:42.080000",
      "content": "<p>I have doubts if this is true. <br>\nI have submitted a notebook with 512 Mb threshold and the total time was about 4. Next, I have submitted the same solution, but with 1 Gb threshold and the total time was about 6h. <br>\nTotal time of the same solution on the test directory (4 tifs) is about 20 minutes. <br>\nIn my opinion it clearly shows, that there are lot of images under 1.2 Gb. I would suspect, that the distribution of the hidden test set should be similar to the train set one (i. e. 95% of images are of lower than 512 Mb). </p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1959104": "I have tried something like : \ntry:\n        size = os.path.getsize(img_path)\n    except :\n        size = 2.8*1e9\n    print('Size = ',size)\n    if size> 1.32*1e9:\n        pred = 0.5\n    else:\n        pred=process(img_path)\nThe submission took 2 minutes and succeed and gave me an LB = 0.6 which is the LB for equal probability 0.5.\nMaybe this is known?",
    "1963221": "Thank you for this information.\n\nI have tried\nif size<1.3e9:\n    then do things appropriately\nelse:\n    pred = 0.5\n\nbut when submitting notebook threw exception. Do you have any idea how to approach it with getting the error.\n",
    "1960157": "Thanks for running this test. \n\nIt would have been nice if the competition sponsors included the largest image in the 'public' version of the test set. This way we would know if the notebook instance will run out of memory. Probably would have saved a lot of headaches.",
    "1961602": "I have doubts if this is true. \nI have submitted a notebook with 512 Mb threshold and the total time was about 4. Next, I have submitted the same solution, but with 1 Gb threshold and the total time was about 6h. \nTotal time of the same solution on the test directory (4 tifs) is about 20 minutes. \nIn my opinion it clearly shows, that there are lot of images under 1.2 Gb. I would suspect, that the distribution of the hidden test set should be similar to the train set one (i. e. 95% of images are of lower than 512 Mb). "
  }
}