{
  "id": 544225,
  "title": "Borken PNG file?",
  "url": "/competitions/bhf-data-science-centre-ecg-challenge/discussion/544225",
  "author_name": "Rodrigo M Carrillo Larco",
  "post_date": "2024-11-04T00:06:42.636000",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Has anyone found this error when training a CNN? Any help will be appreciated! :) </p>\n<p>Below just a line of the error:<br>\n\"SyntaxError: broken PNG file (chunk b'\\x00\\x00\\x00\\x00')\"</p>\n<p>This has happened to me twice. The first time I went back and looked for the \"broken PNG file\" like this (code below), which identified that this file \"/train_1/train_encoded_relabelled/train_000142.png\" was apparently broken.  I dropped the file and got the same error. But I could not find the apparently second image broken.</p>\n<h1>from PIL import Image</h1>\n<h1>import os</h1>\n<h1>def find_broken_images(root_folder):</h1>\n<h1>broken_images = []</h1>\n<h1># Loop through each subfolder and check all .png files</h1>\n<h1>for subdir, _, files in os.walk(root_folder):</h1>\n<h1>for file in files:</h1>\n<h1>if file.endswith('.png'):</h1>\n<h1>file_path = os.path.join(subdir, file)</h1>\n<h1>try:</h1>\n<h1># Attempt to open the image</h1>\n<h1>with Image.open(file_path) as img:</h1>\n<h1>img.verify()  # Check if it's broken</h1>\n<h1>except (IOError, SyntaxError) as e:</h1>\n<h1># If there's an error, the file might be broken</h1>\n<h1>print(f\"Broken image detected: {file_path}\")</h1>\n<h1>broken_images.append(file_path)</h1>\n<h1>return broken_images</h1>\n<h1># Usage example:</h1>\n<h1>root_folder = \"/kaggle/input/bhf-data-science-centre-ecg-challenge\"  # Replace with your folder path</h1>\n<h1>broken_images = find_broken_images(root_folder)</h1>\n<h1>if broken_images:</h1>\n<h1>print(f\"Found {len(broken_images)} broken images.\")</h1>\n<h1>else:</h1>\n<h1>print(\"No broken images found.\")</h1>\n<p>Thanks!</p>",
  "messages": [
    {
      "id": 3038958,
      "postDate": "2024-11-07T14:32:34.533Z",
      "content": "<p>Yes, our team also discovered a few broken files, including the one that you mentioned. We excluded such files during training, using the following code. Fortunately, there were no broken files in the test set.</p>\n<pre><code> filename  filename_list:\n  :\n    image = cv2.imread(filename)\n  :\n    \n</code></pre>",
      "rawMarkdown": "Yes, our team also discovered a few broken files, including the one that you mentioned. We excluded such files during training, using the following code. Fortunately, there were no broken files in the test set.\n\n```python\nfor filename in filename_list:\n  try:\n    image = cv2.imread(filename)\n  except:\n    continue\n```",
      "votes": 1,
      "replies": [
        {
          "id": 3039396,
          "postDate": "2024-11-08T02:15:07.120Z",
          "content": "<p>Thank you so much for taking the time to answer. Indeed. Second time I ran my code it also discovered another file in train (train_008401.png).</p>",
          "rawMarkdown": "Thank you so much for taking the time to answer. Indeed. Second time I ran my code it also discovered another file in train (train_008401.png)."
        }
      ]
    },
    {
      "id": 3035869,
      "postDate": "2024-11-04T00:06:42.637Z",
      "content": "<p>Has anyone found this error when training a CNN? Any help will be appreciated! :) </p>\n<p>Below just a line of the error:<br>\n\"SyntaxError: broken PNG file (chunk b'\\x00\\x00\\x00\\x00')\"</p>\n<p>This has happened to me twice. The first time I went back and looked for the \"broken PNG file\" like this (code below), which identified that this file \"/train_1/train_encoded_relabelled/train_000142.png\" was apparently broken.  I dropped the file and got the same error. But I could not find the apparently second image broken.</p>\n<h1>from PIL import Image</h1>\n<h1>import os</h1>\n<h1>def find_broken_images(root_folder):</h1>\n<h1>broken_images = []</h1>\n<h1># Loop through each subfolder and check all .png files</h1>\n<h1>for subdir, _, files in os.walk(root_folder):</h1>\n<h1>for file in files:</h1>\n<h1>if file.endswith('.png'):</h1>\n<h1>file_path = os.path.join(subdir, file)</h1>\n<h1>try:</h1>\n<h1># Attempt to open the image</h1>\n<h1>with Image.open(file_path) as img:</h1>\n<h1>img.verify()  # Check if it's broken</h1>\n<h1>except (IOError, SyntaxError) as e:</h1>\n<h1># If there's an error, the file might be broken</h1>\n<h1>print(f\"Broken image detected: {file_path}\")</h1>\n<h1>broken_images.append(file_path)</h1>\n<h1>return broken_images</h1>\n<h1># Usage example:</h1>\n<h1>root_folder = \"/kaggle/input/bhf-data-science-centre-ecg-challenge\"  # Replace with your folder path</h1>\n<h1>broken_images = find_broken_images(root_folder)</h1>\n<h1>if broken_images:</h1>\n<h1>print(f\"Found {len(broken_images)} broken images.\")</h1>\n<h1>else:</h1>\n<h1>print(\"No broken images found.\")</h1>\n<p>Thanks!</p>",
      "rawMarkdown": "Has anyone found this error when training a CNN? Any help will be appreciated! :) \n\nBelow just a line of the error:\n\"SyntaxError: broken PNG file (chunk b'\\x00\\x00\\x00\\x00')\"\n\nThis has happened to me twice. The first time I went back and looked for the \"broken PNG file\" like this (code below), which identified that this file \"/train_1/train_encoded_relabelled/train_000142.png\" was apparently broken.  I dropped the file and got the same error. But I could not find the apparently second image broken.\n\n# from PIL import Image\n# import os\n\n# def find_broken_images(root_folder):\n#     broken_images = []\n\n#     # Loop through each subfolder and check all .png files\n#     for subdir, _, files in os.walk(root_folder):\n#         for file in files:\n#             if file.endswith('.png'):\n#                 file_path = os.path.join(subdir, file)\n#                 try:\n#                     # Attempt to open the image\n#                     with Image.open(file_path) as img:\n#                         img.verify()  # Check if it's broken\n#                 except (IOError, SyntaxError) as e:\n#                     # If there's an error, the file might be broken\n#                     print(f\"Broken image detected: {file_path}\")\n#                     broken_images.append(file_path)\n\n#     return broken_images\n\n# # Usage example:\n# root_folder = \"/kaggle/input/bhf-data-science-centre-ecg-challenge\"  # Replace with your folder path\n# broken_images = find_broken_images(root_folder)\n\n# if broken_images:\n#     print(f\"Found {len(broken_images)} broken images.\")\n# else:\n#     print(\"No broken images found.\")\n\n\nThanks!",
      "votes": 1
    }
  ],
  "comments": [
    {
      "id": 3038958,
      "author_name": "Wounsuk Rhee, M.D.",
      "author_url": "",
      "post_date": "2024-11-07T14:32:34.533000",
      "content": "<p>Yes, our team also discovered a few broken files, including the one that you mentioned. We excluded such files during training, using the following code. Fortunately, there were no broken files in the test set.</p>\n<pre><code> filename  filename_list:\n  :\n    image = cv2.imread(filename)\n  :\n    \n</code></pre>",
      "votes": 1,
      "replies": [
        {
          "id": 3039396,
          "author_name": "Rodrigo M Carrillo Larco",
          "author_url": "",
          "post_date": "2024-11-08T02:15:07.120000",
          "content": "<p>Thank you so much for taking the time to answer. Indeed. Second time I ran my code it also discovered another file in train (train_008401.png).</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3038958": "Yes, our team also discovered a few broken files, including the one that you mentioned. We excluded such files during training, using the following code. Fortunately, there were no broken files in the test set.\n\n```python\nfor filename in filename_list:\n  try:\n    image = cv2.imread(filename)\n  except:\n    continue\n```",
    "3035869": "Has anyone found this error when training a CNN? Any help will be appreciated! :) \n\nBelow just a line of the error:\n\"SyntaxError: broken PNG file (chunk b'\\x00\\x00\\x00\\x00')\"\n\nThis has happened to me twice. The first time I went back and looked for the \"broken PNG file\" like this (code below), which identified that this file \"/train_1/train_encoded_relabelled/train_000142.png\" was apparently broken.  I dropped the file and got the same error. But I could not find the apparently second image broken.\n\n# from PIL import Image\n# import os\n\n# def find_broken_images(root_folder):\n#     broken_images = []\n\n#     # Loop through each subfolder and check all .png files\n#     for subdir, _, files in os.walk(root_folder):\n#         for file in files:\n#             if file.endswith('.png'):\n#                 file_path = os.path.join(subdir, file)\n#                 try:\n#                     # Attempt to open the image\n#                     with Image.open(file_path) as img:\n#                         img.verify()  # Check if it's broken\n#                 except (IOError, SyntaxError) as e:\n#                     # If there's an error, the file might be broken\n#                     print(f\"Broken image detected: {file_path}\")\n#                     broken_images.append(file_path)\n\n#     return broken_images\n\n# # Usage example:\n# root_folder = \"/kaggle/input/bhf-data-science-centre-ecg-challenge\"  # Replace with your folder path\n# broken_images = find_broken_images(root_folder)\n\n# if broken_images:\n#     print(f\"Found {len(broken_images)} broken images.\")\n# else:\n#     print(\"No broken images found.\")\n\n\nThanks!"
  }
}