{
  "id": 369591,
  "title": "Merging generated signal into noise",
  "url": "/competitions/g2net-detecting-continuous-gravitational-waves/discussion/369591",
  "author_name": "Vladimir Slaykovskiy",
  "post_date": "2022-11-30T15:48:52.246000",
  "votes": 7,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Suppose I've generated noise and signal separately using pyfstat and now want to combine them in a single spectrogram.<br>\nCan it be done without fancy libraries?</p>\n<p>For example is it correct to just weight-average them?</p>\n<p><code>sample=0.9 * noise + 0.1 + signal</code></p>",
  "messages": [
    {
      "id": 2050238,
      "postDate": "2022-11-30T15:48:52.247Z",
      "content": "<p>Suppose I've generated noise and signal separately using pyfstat and now want to combine them in a single spectrogram.<br>\nCan it be done without fancy libraries?</p>\n<p>For example is it correct to just weight-average them?</p>\n<p><code>sample=0.9 * noise + 0.1 + signal</code></p>",
      "rawMarkdown": "Suppose I've generated noise and signal separately using pyfstat and now want to combine them in a single spectrogram.\nCan it be done without fancy libraries?\n\nFor example is it correct to just weight-average them?\n\n`sample=0.9 * noise + 0.1 + signal`",
      "votes": 7
    },
    {
      "id": 2050263,
      "postDate": "2022-11-30T16:06:05.017Z",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/vslaykovsky\" target=\"_blank\">@vslaykovsky</a>,</p>\n<p>If <code>noise</code> and <code>signal</code> are Fourier <em>amplitudes</em> (i.e. the <em>complex</em> numbers you get from <code>get_stft_as_arrays</code>) and the generated SFTs have the same timestamps and frequency, then <em>yes</em>.</p>\n<p>Note that you can literally do <em>whatever you want</em> to that data. For example, if you wanted to create an instrumental artifact with a circular shape, you could create such a shape in a numpy array with the appropriate array shape and directly added to your data.</p>\n<blockquote>\n  <p>For example is it correct to just weight-average them?</p>\n</blockquote>\n<p>The standard way in which we do this is by generating signals the amplitude of which is already a certain fraction of your noise's amplitude (so we can just <em>add</em> both arrays).</p>\n<p>Hope this helps.</p>",
      "rawMarkdown": "Hi @vslaykovsky,\n\nIf `noise` and `signal` are Fourier *amplitudes* (i.e. the *complex* numbers you get from `get_stft_as_arrays`) and the generated SFTs have the same timestamps and frequency, then *yes*.\n\nNote that you can literally do *whatever you want* to that data. For example, if you wanted to create an instrumental artifact with a circular shape, you could create such a shape in a numpy array with the appropriate array shape and directly added to your data.\n\n> For example is it correct to just weight-average them?\n\nThe standard way in which we do this is by generating signals the amplitude of which is already a certain fraction of your noise's amplitude (so we can just *add* both arrays).\n\nHope this helps.\n",
      "votes": 4
    },
    {
      "id": 2053374,
      "postDate": "2022-12-03T07:42:13.017Z",
      "content": "<p>How to explain 0.9 and 0.1, how to ensure that it does not overfit?</p>",
      "rawMarkdown": "How to explain 0.9 and 0.1, how to ensure that it does not overfit?",
      "votes": 1,
      "replies": [
        {
          "id": 2059292,
          "postDate": "2022-12-08T17:16:45.640Z",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/royalacecat\" target=\"_blank\">@royalacecat</a> ,</p>\n<p>A simpler way of doing that may be to use <code>sample = noise + K * signal</code> or <code>sample = K * noise + signal</code>,<br>\nwhere <code>K</code> can be tuned to generate stronger or weaker signals depending on what you try to do.</p>",
          "rawMarkdown": "Hi @royalacecat ,\n\nA simpler way of doing that may be to use `sample = noise + K * signal` or `sample = K * noise + signal`,\nwhere `K` can be tuned to generate stronger or weaker signals depending on what you try to do.\n\n\n"
        }
      ]
    },
    {
      "id": 2050473,
      "postDate": "2022-11-30T19:07:51.137Z",
      "content": "<p>i was wondering too.<br>\nthanks for asking the question. luckily got a nice answer from the host.<br>\nawesome.</p>",
      "rawMarkdown": "i was wondering too.\nthanks for asking the question. luckily got a nice answer from the host.\nawesome.",
      "votes": 1
    }
  ],
  "comments": [
    {
      "id": 2050263,
      "author_name": "Rodrigo Tenorio",
      "author_url": "",
      "post_date": "2022-11-30T16:06:05.017000",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/vslaykovsky\" target=\"_blank\">@vslaykovsky</a>,</p>\n<p>If <code>noise</code> and <code>signal</code> are Fourier <em>amplitudes</em> (i.e. the <em>complex</em> numbers you get from <code>get_stft_as_arrays</code>) and the generated SFTs have the same timestamps and frequency, then <em>yes</em>.</p>\n<p>Note that you can literally do <em>whatever you want</em> to that data. For example, if you wanted to create an instrumental artifact with a circular shape, you could create such a shape in a numpy array with the appropriate array shape and directly added to your data.</p>\n<blockquote>\n  <p>For example is it correct to just weight-average them?</p>\n</blockquote>\n<p>The standard way in which we do this is by generating signals the amplitude of which is already a certain fraction of your noise's amplitude (so we can just <em>add</em> both arrays).</p>\n<p>Hope this helps.</p>",
      "votes": 4,
      "replies": []
    },
    {
      "id": 2053374,
      "author_name": "E-Max AI",
      "author_url": "",
      "post_date": "2022-12-03T07:42:13.017000",
      "content": "<p>How to explain 0.9 and 0.1, how to ensure that it does not overfit?</p>",
      "votes": 1,
      "replies": [
        {
          "id": 2059292,
          "author_name": "Rodrigo Tenorio",
          "author_url": "",
          "post_date": "2022-12-08T17:16:45.640000",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/royalacecat\" target=\"_blank\">@royalacecat</a> ,</p>\n<p>A simpler way of doing that may be to use <code>sample = noise + K * signal</code> or <code>sample = K * noise + signal</code>,<br>\nwhere <code>K</code> can be tuned to generate stronger or weaker signals depending on what you try to do.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 2050473,
      "author_name": "ibi",
      "author_url": "",
      "post_date": "2022-11-30T19:07:51.137000",
      "content": "<p>i was wondering too.<br>\nthanks for asking the question. luckily got a nice answer from the host.<br>\nawesome.</p>",
      "votes": 1,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2050238": "Suppose I've generated noise and signal separately using pyfstat and now want to combine them in a single spectrogram.\nCan it be done without fancy libraries?\n\nFor example is it correct to just weight-average them?\n\n`sample=0.9 * noise + 0.1 + signal`",
    "2050263": "Hi @vslaykovsky,\n\nIf `noise` and `signal` are Fourier *amplitudes* (i.e. the *complex* numbers you get from `get_stft_as_arrays`) and the generated SFTs have the same timestamps and frequency, then *yes*.\n\nNote that you can literally do *whatever you want* to that data. For example, if you wanted to create an instrumental artifact with a circular shape, you could create such a shape in a numpy array with the appropriate array shape and directly added to your data.\n\n> For example is it correct to just weight-average them?\n\nThe standard way in which we do this is by generating signals the amplitude of which is already a certain fraction of your noise's amplitude (so we can just *add* both arrays).\n\nHope this helps.\n",
    "2053374": "How to explain 0.9 and 0.1, how to ensure that it does not overfit?",
    "2050473": "i was wondering too.\nthanks for asking the question. luckily got a nice answer from the host.\nawesome."
  }
}