{
  "id": 366572,
  "title": "Non-stationary noise issue in the tutorial",
  "url": "/competitions/g2net-detecting-continuous-gravitational-waves/discussion/366572",
  "author_name": "Alex Fernandez",
  "post_date": "2022-11-16T18:33:37.009000",
  "votes": 2,
  "comment_count": 4,
  "views": 0,
  "content": "<p>In the <a href=\"https://github.com/PyFstat/PyFstat/blob/5ded4a672c3ab7309b64ce2531bcf704d4ed2f4e/examples/tutorials/0_generating_noise.ipynb\" target=\"_blank\">Tutorial 0: Noise generation</a> i think there is a issue that create gaps in the data. Where there is: </p>\n<blockquote>\n  <p><code>if segment &gt; 0:</code><br>\n  <code>writer_kwargs[\"tstart\"] += writer_kwargs[\"Tsft\"] + segment_lengths[segment - 1]</code></p>\n</blockquote>\n<p>it must be:</p>\n<blockquote>\n  <p><code>if segment &gt; 0:</code><br>\n  <code>writer_kwargs[\"tstart\"] += segment_lengths[segment - 1]</code></p>\n</blockquote>\n<p>Otherwise, it creates gaps between data of <code>Tsft</code> length first, two times <code>Tsft</code> after, three times <code>Tsft</code>, and so on… </p>\n<p>Supouse that <code>writer_kwargs[\"tstart\"]</code> starts at 0 and supouse that <code>segment_lengths[0]</code> and <code>Tsft</code>are equal 1. For the first <code>fourier_data</code> segment we'll have only one column and we want that the following segment starts at <code>writer_kwargs[\"tstart\"]</code> equal 1, not 2, as it works in the tutorial. <br>\nIf we have an other segment following the last one, assuming that the length is equal 1 for this, we want <code>writer_kwargs[\"tstart\"]</code> to be 2, not 4, as it working in the tutorial. </p>",
  "messages": [
    {
      "id": 2032664,
      "postDate": "2022-11-16T18:33:37.010Z",
      "content": "<p>In the <a href=\"https://github.com/PyFstat/PyFstat/blob/5ded4a672c3ab7309b64ce2531bcf704d4ed2f4e/examples/tutorials/0_generating_noise.ipynb\" target=\"_blank\">Tutorial 0: Noise generation</a> i think there is a issue that create gaps in the data. Where there is: </p>\n<blockquote>\n  <p><code>if segment &gt; 0:</code><br>\n  <code>writer_kwargs[\"tstart\"] += writer_kwargs[\"Tsft\"] + segment_lengths[segment - 1]</code></p>\n</blockquote>\n<p>it must be:</p>\n<blockquote>\n  <p><code>if segment &gt; 0:</code><br>\n  <code>writer_kwargs[\"tstart\"] += segment_lengths[segment - 1]</code></p>\n</blockquote>\n<p>Otherwise, it creates gaps between data of <code>Tsft</code> length first, two times <code>Tsft</code> after, three times <code>Tsft</code>, and so on… </p>\n<p>Supouse that <code>writer_kwargs[\"tstart\"]</code> starts at 0 and supouse that <code>segment_lengths[0]</code> and <code>Tsft</code>are equal 1. For the first <code>fourier_data</code> segment we'll have only one column and we want that the following segment starts at <code>writer_kwargs[\"tstart\"]</code> equal 1, not 2, as it works in the tutorial. <br>\nIf we have an other segment following the last one, assuming that the length is equal 1 for this, we want <code>writer_kwargs[\"tstart\"]</code> to be 2, not 4, as it working in the tutorial. </p>",
      "rawMarkdown": "In the [Tutorial 0: Noise generation](https://github.com/PyFstat/PyFstat/blob/5ded4a672c3ab7309b64ce2531bcf704d4ed2f4e/examples/tutorials/0_generating_noise.ipynb) i think there is a issue that create gaps in the data. Where there is: \n\n>`if segment > 0:`\n>`        writer_kwargs[\"tstart\"] += writer_kwargs[\"Tsft\"] + segment_lengths[segment - 1]`\n\nit must be:\n\n>`if segment > 0:`\n>`        writer_kwargs[\"tstart\"] += segment_lengths[segment - 1]`\n\nOtherwise, it creates gaps between data of `Tsft` length first, two times `Tsft` after, three times `Tsft`, and so on... \n\nSupouse that `writer_kwargs[\"tstart\"]` starts at 0 and supouse that `segment_lengths[0]` and `Tsft`are equal 1. For the first `fourier_data` segment we'll have only one column and we want that the following segment starts at `writer_kwargs[\"tstart\"]` equal 1, not 2, as it works in the tutorial. \nIf we have an other segment following the last one, assuming that the length is equal 1 for this, we want `writer_kwargs[\"tstart\"]` to be 2, not 4, as it working in the tutorial. ",
      "votes": 2
    },
    {
      "id": 2034703,
      "postDate": "2022-11-18T11:06:27.567Z",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/alexfernandezcz\" target=\"_blank\">@alexfernandezcz</a> ,</p>\n<p>It's not necessarily incorrect, I just chose that convention back when I did the tutorials.<br>\nSince the section where this pops up is though to illustrate non-stationary noise rather than gaps<br>\nI don't think it's too much of an issue, but of course any suggestions are welcome as PRs.</p>",
      "rawMarkdown": "Hi @alexfernandezcz ,\n\nIt's not necessarily incorrect, I just chose that convention back when I did the tutorials.\nSince the section where this pops up is though to illustrate non-stationary noise rather than gaps\nI don't think it's too much of an issue, but of course any suggestions are welcome as PRs.\n\n",
      "replies": [
        {
          "id": 2035448,
          "postDate": "2022-11-18T21:58:01.837Z",
          "content": "<p>Hi Rodrigo. In the tutorial it shows how to generate non-stationary noise, however it doesn't show how to incorporate signal into that non-stationary noise. Is there a tutorial somewhere I can find to do that?</p>",
          "rawMarkdown": "Hi Rodrigo. In the tutorial it shows how to generate non-stationary noise, however it doesn't show how to incorporate signal into that non-stationary noise. Is there a tutorial somewhere I can find to do that?",
          "votes": 1,
          "isDeleted": true
        },
        {
          "id": 2038100,
          "postDate": "2022-11-21T07:40:27.230Z",
          "content": "<p>Generate noise, save it as SFTs, provide the link to the saved file with noiseSFTs option of the writer.<br>\n<a href=\"https://pyfstat.readthedocs.io/en/latest/pyfstat.html#module-pyfstat.make_sfts\" target=\"_blank\">https://pyfstat.readthedocs.io/en/latest/pyfstat.html#module-pyfstat.make_sfts</a></p>",
          "rawMarkdown": "Generate noise, save it as SFTs, provide the link to the saved file with noiseSFTs option of the writer.\nhttps://pyfstat.readthedocs.io/en/latest/pyfstat.html#module-pyfstat.make_sfts",
          "votes": 2
        },
        {
          "id": 2039882,
          "postDate": "2022-11-22T15:51:10.587Z",
          "content": "<p>thank you))</p>",
          "rawMarkdown": "thank you))",
          "isDeleted": true
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 2034703,
      "author_name": "Rodrigo Tenorio",
      "author_url": "",
      "post_date": "2022-11-18T11:06:27.567000",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/alexfernandezcz\" target=\"_blank\">@alexfernandezcz</a> ,</p>\n<p>It's not necessarily incorrect, I just chose that convention back when I did the tutorials.<br>\nSince the section where this pops up is though to illustrate non-stationary noise rather than gaps<br>\nI don't think it's too much of an issue, but of course any suggestions are welcome as PRs.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 2035448,
          "author_name": "",
          "author_url": "",
          "post_date": "2022-11-18T21:58:01.837000",
          "content": "<p>Hi Rodrigo. In the tutorial it shows how to generate non-stationary noise, however it doesn't show how to incorporate signal into that non-stationary noise. Is there a tutorial somewhere I can find to do that?</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 2038100,
          "author_name": "DennisSakva",
          "author_url": "",
          "post_date": "2022-11-21T07:40:27.230000",
          "content": "<p>Generate noise, save it as SFTs, provide the link to the saved file with noiseSFTs option of the writer.<br>\n<a href=\"https://pyfstat.readthedocs.io/en/latest/pyfstat.html#module-pyfstat.make_sfts\" target=\"_blank\">https://pyfstat.readthedocs.io/en/latest/pyfstat.html#module-pyfstat.make_sfts</a></p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 2039882,
          "author_name": "",
          "author_url": "",
          "post_date": "2022-11-22T15:51:10.587000",
          "content": "<p>thank you))</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2032664": "In the [Tutorial 0: Noise generation](https://github.com/PyFstat/PyFstat/blob/5ded4a672c3ab7309b64ce2531bcf704d4ed2f4e/examples/tutorials/0_generating_noise.ipynb) i think there is a issue that create gaps in the data. Where there is: \n\n>`if segment > 0:`\n>`        writer_kwargs[\"tstart\"] += writer_kwargs[\"Tsft\"] + segment_lengths[segment - 1]`\n\nit must be:\n\n>`if segment > 0:`\n>`        writer_kwargs[\"tstart\"] += segment_lengths[segment - 1]`\n\nOtherwise, it creates gaps between data of `Tsft` length first, two times `Tsft` after, three times `Tsft`, and so on... \n\nSupouse that `writer_kwargs[\"tstart\"]` starts at 0 and supouse that `segment_lengths[0]` and `Tsft`are equal 1. For the first `fourier_data` segment we'll have only one column and we want that the following segment starts at `writer_kwargs[\"tstart\"]` equal 1, not 2, as it works in the tutorial. \nIf we have an other segment following the last one, assuming that the length is equal 1 for this, we want `writer_kwargs[\"tstart\"]` to be 2, not 4, as it working in the tutorial. ",
    "2034703": "Hi @alexfernandezcz ,\n\nIt's not necessarily incorrect, I just chose that convention back when I did the tutorials.\nSince the section where this pops up is though to illustrate non-stationary noise rather than gaps\nI don't think it's too much of an issue, but of course any suggestions are welcome as PRs.\n\n"
  }
}