{
  "id": 361312,
  "title": "Minimum float32 is 1e-38 and data**2 is 1e-44",
  "url": "/competitions/g2net-detecting-continuous-gravitational-waves/discussion/361312",
  "author_name": "🐢 Jun Koda",
  "post_date": "2022-10-20T23:28:09.541000",
  "votes": 38,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Recall that the smallest positive number float32 can express is ~ 1e-38 and data is ~1e-22.<br>\nOnce we square the data, they go beyond this limit.</p>\n<pre><code>np.float32(1.23456e-44) =&gt; 1.3e-44\n</code></pre>\n<p>float32 has about 8 significant digits and 1.23456e-44 will be expressed as 0.0000012e-38 with float32,<br>\nonly 2 digits remaining. (I computed mean amplitude squared and got surprisingly identical values due to this lack of digits)</p>\n<p>I suggest that we multiply the data by 1e21 or 1e22 as soon as we load them.</p>",
  "messages": [
    {
      "id": 1997471,
      "postDate": "2022-10-20T23:28:09.540Z",
      "content": "<p>Recall that the smallest positive number float32 can express is ~ 1e-38 and data is ~1e-22.<br>\nOnce we square the data, they go beyond this limit.</p>\n<pre><code>np.float32(1.23456e-44) =&gt; 1.3e-44\n</code></pre>\n<p>float32 has about 8 significant digits and 1.23456e-44 will be expressed as 0.0000012e-38 with float32,<br>\nonly 2 digits remaining. (I computed mean amplitude squared and got surprisingly identical values due to this lack of digits)</p>\n<p>I suggest that we multiply the data by 1e21 or 1e22 as soon as we load them.</p>",
      "rawMarkdown": "Recall that the smallest positive number float32 can express is ~ 1e-38 and data is ~1e-22.\nOnce we square the data, they go beyond this limit.\n\n```python\nnp.float32(1.23456e-44) => 1.3e-44\n```\n\nfloat32 has about 8 significant digits and 1.23456e-44 will be expressed as 0.0000012e-38 with float32,\nonly 2 digits remaining. (I computed mean amplitude squared and got surprisingly identical values due to this lack of digits)\n\nI suggest that we multiply the data by 1e21 or 1e22 as soon as we load them.",
      "votes": 37
    },
    {
      "id": 1997831,
      "postDate": "2022-10-21T07:29:41.313Z",
      "content": "<p>You can use mean/std normalization</p>",
      "rawMarkdown": "You can use mean/std normalization",
      "votes": 2
    },
    {
      "id": 2006551,
      "postDate": "2022-10-27T16:34:13.687Z",
      "content": "<p>I agree, it seems that perhaps it is good just to multiply it by 1e22 in order to compute the energy and then normalize.</p>",
      "rawMarkdown": "I agree, it seems that perhaps it is good just to multiply it by 1e22 in order to compute the energy and then normalize."
    },
    {
      "id": 1999829,
      "postDate": "2022-10-22T18:44:18.737Z",
      "content": "<p>why not use float64? </p>",
      "rawMarkdown": "why not use float64? ",
      "replies": [
        {
          "id": 1999960,
          "postDate": "2022-10-22T22:15:18.437Z",
          "content": "<p>Yes, that’s another good solution.<br>\nastype(np.complex128)</p>",
          "rawMarkdown": "Yes, that’s another good solution.\nastype(np.complex128)"
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 1997831,
      "author_name": "DennisSakva",
      "author_url": "",
      "post_date": "2022-10-21T07:29:41.313000",
      "content": "<p>You can use mean/std normalization</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 2006551,
      "author_name": "Aaditya Agnihotri",
      "author_url": "",
      "post_date": "2022-10-27T16:34:13.687000",
      "content": "<p>I agree, it seems that perhaps it is good just to multiply it by 1e22 in order to compute the energy and then normalize.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1999829,
      "author_name": "Salman Ahmad",
      "author_url": "",
      "post_date": "2022-10-22T18:44:18.737000",
      "content": "<p>why not use float64? </p>",
      "votes": 0,
      "replies": [
        {
          "id": 1999960,
          "author_name": "🐢 Jun Koda",
          "author_url": "",
          "post_date": "2022-10-22T22:15:18.437000",
          "content": "<p>Yes, that’s another good solution.<br>\nastype(np.complex128)</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1997471": "Recall that the smallest positive number float32 can express is ~ 1e-38 and data is ~1e-22.\nOnce we square the data, they go beyond this limit.\n\n```python\nnp.float32(1.23456e-44) => 1.3e-44\n```\n\nfloat32 has about 8 significant digits and 1.23456e-44 will be expressed as 0.0000012e-38 with float32,\nonly 2 digits remaining. (I computed mean amplitude squared and got surprisingly identical values due to this lack of digits)\n\nI suggest that we multiply the data by 1e21 or 1e22 as soon as we load them.",
    "1997831": "You can use mean/std normalization",
    "2006551": "I agree, it seems that perhaps it is good just to multiply it by 1e22 in order to compute the energy and then normalize.",
    "1999829": "why not use float64? "
  }
}