{
  "id": 358832,
  "title": "Timestamps and Frequencies Insights",
  "url": "/competitions/g2net-detecting-continuous-gravitational-waves/discussion/358832",
  "author_name": "Mark Wijkhuizen",
  "post_date": "2022-10-09T17:55:12.204000",
  "votes": 11,
  "comment_count": 3,
  "views": 0,
  "content": "<p>The data provided consists of hort-time Fourier Transforms (SFTs), but there seems to be 2 differences with conventional SFT's.</p>\n<p>1) The time delta between each timestamp is inconsistent, meaning the time between measurement in the SFT's can vary greatly. Besides the unknown unit of the timestamps, the time delta in several samples examined varies between 33.0 and 48648 seconds, with ~91% of time deltas being 1800 (30 minutes). How could we handle these inconsistent time deltas?</p>\n<p>2) The frequency range is consistently ~0.2Hz, however the min/max frequency ranges from around ~50to ~498Hz. As the frequency range is consistent between SFT's I expect the varying min/max frequency can be ignored, how do you feel about this?</p>\n<p>3) When taking a look at the timestamps they start at 2009-03-27/28 and end on 2009-07-25/26/27. The recordings are thus gathered during 4 months with often 30 minutes between the recordings. We should thus not interpret the spectrograms as continuous sounds, which is conventionally the case.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4433335%2F6c98524b69a6de2b6334d63f56cd2810%2FScreenshot%202022-10-12%20171247.png?generation=1665587669575759&amp;alt=media\" alt=\"\"></p>",
  "messages": [
    {
      "id": 1979796,
      "postDate": "2022-10-09T17:55:12.203Z",
      "content": "<p>The data provided consists of hort-time Fourier Transforms (SFTs), but there seems to be 2 differences with conventional SFT's.</p>\n<p>1) The time delta between each timestamp is inconsistent, meaning the time between measurement in the SFT's can vary greatly. Besides the unknown unit of the timestamps, the time delta in several samples examined varies between 33.0 and 48648 seconds, with ~91% of time deltas being 1800 (30 minutes). How could we handle these inconsistent time deltas?</p>\n<p>2) The frequency range is consistently ~0.2Hz, however the min/max frequency ranges from around ~50to ~498Hz. As the frequency range is consistent between SFT's I expect the varying min/max frequency can be ignored, how do you feel about this?</p>\n<p>3) When taking a look at the timestamps they start at 2009-03-27/28 and end on 2009-07-25/26/27. The recordings are thus gathered during 4 months with often 30 minutes between the recordings. We should thus not interpret the spectrograms as continuous sounds, which is conventionally the case.</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4433335%2F6c98524b69a6de2b6334d63f56cd2810%2FScreenshot%202022-10-12%20171247.png?generation=1665587669575759&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "The data provided consists of hort-time Fourier Transforms (SFTs), but there seems to be 2 differences with conventional SFT's.\n\n1) The time delta between each timestamp is inconsistent, meaning the time between measurement in the SFT's can vary greatly. Besides the unknown unit of the timestamps, the time delta in several samples examined varies between 33.0 and 48648 seconds, with ~91% of time deltas being 1800 (30 minutes). How could we handle these inconsistent time deltas?\n\n2) The frequency range is consistently ~0.2Hz, however the min/max frequency ranges from around ~50to ~498Hz. As the frequency range is consistent between SFT's I expect the varying min/max frequency can be ignored, how do you feel about this?\n\n3) When taking a look at the timestamps they start at 2009-03-27/28 and end on 2009-07-25/26/27. The recordings are thus gathered during 4 months with often 30 minutes between the recordings. We should thus not interpret the spectrograms as continuous sounds, which is conventionally the case.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4433335%2F6c98524b69a6de2b6334d63f56cd2810%2FScreenshot%202022-10-12%20171247.png?generation=1665587669575759&alt=media)",
      "votes": 10
    },
    {
      "id": 1985196,
      "postDate": "2022-10-13T07:16:42.967Z",
      "content": "<p>Hi Mark,</p>\n<blockquote>\n  <p>Besides the unknown unit of the timestamps</p>\n</blockquote>\n<p>You are right, thanks for the catch: we didn't specify timestamps units in the Data page (although we did in the tutorial).<br>\nTimestamps are GPS time, which started ~10 years after UNIX time (i.e, data starts at 2019 rather than at 2009).</p>\n<p>It should be clear now from the Data page, but do let me know if you think there's a better way of clarifying this unit.</p>\n<p>Cheers,</p>",
      "rawMarkdown": "Hi Mark,\n\n> Besides the unknown unit of the timestamps\n\nYou are right, thanks for the catch: we didn't specify timestamps units in the Data page (although we did in the tutorial).\nTimestamps are GPS time, which started ~10 years after UNIX time (i.e, data starts at 2019 rather than at 2009).\n\nIt should be clear now from the Data page, but do let me know if you think there's a better way of clarifying this unit.\n\nCheers,",
      "votes": 4
    },
    {
      "id": 2013067,
      "postDate": "2022-11-01T15:48:33.440Z",
      "content": "<p>Hi all! Thanks Mark for the insightful analysis and thanks Rodrigo for your answer. I have a folllow-up question: when the time-delta between two time-stamps is smaller than 1800s, does it mean that there is an overlap in the Fourier windows used in the SFTs corresponding to these time stamps? In that case, the noise component between this two SFTs is somehow correlated?</p>",
      "rawMarkdown": "Hi all! Thanks Mark for the insightful analysis and thanks Rodrigo for your answer. I have a folllow-up question: when the time-delta between two time-stamps is smaller than 1800s, does it mean that there is an overlap in the Fourier windows used in the SFTs corresponding to these time stamps? In that case, the noise component between this two SFTs is somehow correlated?",
      "isDeleted": true,
      "replies": [
        {
          "id": 2014358,
          "postDate": "2022-11-02T14:17:39.347Z",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/tantto\" target=\"_blank\">@tantto</a> ,</p>\n<blockquote>\n  <p>when the time-delta between two time-stamps is smaller than 1800s, does it mean that there is an overlap in the Fourier windows used in the SFTs corresponding to these time stamps?</p>\n</blockquote>\n<p>If those SFTs were taken by the same detector, then yes, there is an overlap in the Fourier windows.<br>\nThere is no restriction about how much overlap you may impose whenever creating SFTs.</p>",
          "rawMarkdown": "Hi @tantto ,\n\n> when the time-delta between two time-stamps is smaller than 1800s, does it mean that there is an overlap in the Fourier windows used in the SFTs corresponding to these time stamps?\n\nIf those SFTs were taken by the same detector, then yes, there is an overlap in the Fourier windows.\nThere is no restriction about how much overlap you may impose whenever creating SFTs.",
          "votes": 1
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 1985196,
      "author_name": "Rodrigo Tenorio",
      "author_url": "",
      "post_date": "2022-10-13T07:16:42.967000",
      "content": "<p>Hi Mark,</p>\n<blockquote>\n  <p>Besides the unknown unit of the timestamps</p>\n</blockquote>\n<p>You are right, thanks for the catch: we didn't specify timestamps units in the Data page (although we did in the tutorial).<br>\nTimestamps are GPS time, which started ~10 years after UNIX time (i.e, data starts at 2019 rather than at 2009).</p>\n<p>It should be clear now from the Data page, but do let me know if you think there's a better way of clarifying this unit.</p>\n<p>Cheers,</p>",
      "votes": 4,
      "replies": []
    },
    {
      "id": 2013067,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-11-01T15:48:33.440000",
      "content": "<p>Hi all! Thanks Mark for the insightful analysis and thanks Rodrigo for your answer. I have a folllow-up question: when the time-delta between two time-stamps is smaller than 1800s, does it mean that there is an overlap in the Fourier windows used in the SFTs corresponding to these time stamps? In that case, the noise component between this two SFTs is somehow correlated?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 2014358,
          "author_name": "Rodrigo Tenorio",
          "author_url": "",
          "post_date": "2022-11-02T14:17:39.347000",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/tantto\" target=\"_blank\">@tantto</a> ,</p>\n<blockquote>\n  <p>when the time-delta between two time-stamps is smaller than 1800s, does it mean that there is an overlap in the Fourier windows used in the SFTs corresponding to these time stamps?</p>\n</blockquote>\n<p>If those SFTs were taken by the same detector, then yes, there is an overlap in the Fourier windows.<br>\nThere is no restriction about how much overlap you may impose whenever creating SFTs.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1979796": "The data provided consists of hort-time Fourier Transforms (SFTs), but there seems to be 2 differences with conventional SFT's.\n\n1) The time delta between each timestamp is inconsistent, meaning the time between measurement in the SFT's can vary greatly. Besides the unknown unit of the timestamps, the time delta in several samples examined varies between 33.0 and 48648 seconds, with ~91% of time deltas being 1800 (30 minutes). How could we handle these inconsistent time deltas?\n\n2) The frequency range is consistently ~0.2Hz, however the min/max frequency ranges from around ~50to ~498Hz. As the frequency range is consistent between SFT's I expect the varying min/max frequency can be ignored, how do you feel about this?\n\n3) When taking a look at the timestamps they start at 2009-03-27/28 and end on 2009-07-25/26/27. The recordings are thus gathered during 4 months with often 30 minutes between the recordings. We should thus not interpret the spectrograms as continuous sounds, which is conventionally the case.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4433335%2F6c98524b69a6de2b6334d63f56cd2810%2FScreenshot%202022-10-12%20171247.png?generation=1665587669575759&alt=media)",
    "1985196": "Hi Mark,\n\n> Besides the unknown unit of the timestamps\n\nYou are right, thanks for the catch: we didn't specify timestamps units in the Data page (although we did in the tutorial).\nTimestamps are GPS time, which started ~10 years after UNIX time (i.e, data starts at 2019 rather than at 2009).\n\nIt should be clear now from the Data page, but do let me know if you think there's a better way of clarifying this unit.\n\nCheers,",
    "2013067": "Hi all! Thanks Mark for the insightful analysis and thanks Rodrigo for your answer. I have a folllow-up question: when the time-delta between two time-stamps is smaller than 1800s, does it mean that there is an overlap in the Fourier windows used in the SFTs corresponding to these time stamps? In that case, the noise component between this two SFTs is somehow correlated?"
  }
}