{
  "id": 362721,
  "title": "Amplitude Spectral Density and Short - Time Fourier Transform",
  "url": "/competitions/g2net-detecting-continuous-gravitational-waves/discussion/362721",
  "author_name": "Cristo JV",
  "post_date": "2022-10-28T18:39:42.395000",
  "votes": 0,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hi!</p>\n<p>I'm having a hard time trying to understand the relationship between the amplitude spectral density (ASD) and the STFT's absolute values (real valued) of a signal.</p>\n<p><strong>Are the STFT's coefficients (absolute values) equivalent to the ASD's values?</strong></p>\n<p>The reason behind that cuestion is to obtain STFT's absolute values of the LIGO detectors' noise from the amplitude spectral density (ASD). Here there is an example:</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5184030%2Fff757420270a70bc772b3d95eecca8cb%2FG1H1L1V1-OBSERVING_HOFT_SPECTRUM-1239321618-86400.png?generation=1666982197428544&amp;alt=media\" alt=\"\"></p>\n<p>Thank you!</p>",
  "messages": [
    {
      "id": 2009683,
      "postDate": "2022-10-30T08:11:55.377Z",
      "content": "<p>STFT is just a fourier transform (FT) done in each consequential time window ([<a href=\"https://en.wikipedia.org/wiki/Short-time_Fourier_transform#Forward_STFT](see\" target=\"_blank\">https://en.wikipedia.org/wiki/Short-time_Fourier_transform#Forward_STFT](see</a> <a href=\"https://en.wikipedia.org/wiki/Short-time_Fourier_transform#Forward_STFT))\" target=\"_blank\">https://en.wikipedia.org/wiki/Short-time_Fourier_transform#Forward_STFT))</a>. <br>\nAs our initial signal are real valued, we don't need to consider negative frequencies (see <code>return_onesided</code> parameter from the docs <a href=\"https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.stft.html)\" target=\"_blank\">https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.stft.html)</a>.</p>\n<p>On the other hand, ASD is the square root of the power spectrum (not the STFT). And the power spectrum is the squared absolute value of FT, which is equivalent to the FT multiplied by its complex conjugation. Of course, it is real non-negative frequency function, and the square root leads to real non-negative ASD. Again, the power spectrum is symmetric for the real valued initial signal, and we don't need to consider negative frequencies. The lack of negative frequencies makes ASD and STFT similar. However, the first is real valued and the second is complex valued, and their physical sences are different.</p>",
      "rawMarkdown": "STFT is just a fourier transform (FT) done in each consequential time window ([https://en.wikipedia.org/wiki/Short-time_Fourier_transform#Forward_STFT](see https://en.wikipedia.org/wiki/Short-time_Fourier_transform#Forward_STFT)). \nAs our initial signal are real valued, we don't need to consider negative frequencies (see `return_onesided` parameter from the docs https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.stft.html).\n\nOn the other hand, ASD is the square root of the power spectrum (not the STFT). And the power spectrum is the squared absolute value of FT, which is equivalent to the FT multiplied by its complex conjugation. Of course, it is real non-negative frequency function, and the square root leads to real non-negative ASD. Again, the power spectrum is symmetric for the real valued initial signal, and we don't need to consider negative frequencies. The lack of negative frequencies makes ASD and STFT similar. However, the first is real valued and the second is complex valued, and their physical sences are different.",
      "votes": 2,
      "replies": [
        {
          "id": 2010884,
          "postDate": "2022-10-31T08:04:27.140Z",
          "content": "<p>Thank you for your answer Konstantin!<br>\nIndeed, I was refering to the absolute value of the STFT's coefficients (real valued). I've change the question because it was bad formulated.<br>\nSo, I can extract from your answer that as it is a real signal, the STFT's coefficients are equivalent to the ASD's coefficients.<br>\nIs that right? Is there any hidden assumption we should take for this statement? <br>\nCan you point me out how Parseval's theorem fits in this, and the welch's method used for spectral density stimation?</p>",
          "rawMarkdown": "Thank you for your answer Konstantin!\nIndeed, I was refering to the absolute value of the STFT's coefficients (real valued). I've change the question because it was bad formulated.\nSo, I can extract from your answer that as it is a real signal, the STFT's coefficients are equivalent to the ASD's coefficients.\nIs that right? Is there any hidden assumption we should take for this statement? \nCan you point me out how Parseval's theorem fits in this, and the welch's method used for spectral density stimation?"
        },
        {
          "id": 2011009,
          "postDate": "2022-10-31T10:22:31.100Z",
          "content": "<p>Maybe I don't quite understand the question, but I'll try to make a few remarks here.<br>\n1) STFT is a \"short-time\" transform. It is used to analyze the time dependence of frequency components. On the other hand, the power spectrum (and ASD) characterizes the full signal.<br>\n2) When you do the discrete FT of some N-point signal, you may or may not normalize the result by dividing it by N. To calculate the power spectrum you divide by N the squared absolute value of not-normalized FT. As ASD is square root, it is equally divided by sqrt(N). So you may notice, it is measured in [smth/sqrt(Hz)] whilst FT is measured in [smth/Hz]. <br>\n3) When applying Welch or Bartlett methods you split the signal into parts and calculate the power spectrum of each part (<a href=\"https://en.wikipedia.org/wiki/Bartlett%27s_method)\" target=\"_blank\">https://en.wikipedia.org/wiki/Bartlett%27s_method)</a>. After that you average the results. So you reduce the noise in the estimation. The price for this is that the frequency resolution is worse, since each part of the signal is shorter, than the full signal. <br>\nNote, that power spectrum (not the absolute value of FT) is averaged in these methods. So, you have to square the absolute values of STFT coefficients,  then normalize them correctly, average and take the root to get ASD.</p>",
          "rawMarkdown": "Maybe I don't quite understand the question, but I'll try to make a few remarks here.\n1) STFT is a \"short-time\" transform. It is used to analyze the time dependence of frequency components. On the other hand, the power spectrum (and ASD) characterizes the full signal.\n2) When you do the discrete FT of some N-point signal, you may or may not normalize the result by dividing it by N. To calculate the power spectrum you divide by N the squared absolute value of not-normalized FT. As ASD is square root, it is equally divided by sqrt(N). So you may notice, it is measured in [smth/sqrt(Hz)] whilst FT is measured in [smth/Hz]. \n3) When applying Welch or Bartlett methods you split the signal into parts and calculate the power spectrum of each part (https://en.wikipedia.org/wiki/Bartlett%27s_method). After that you average the results. So you reduce the noise in the estimation. The price for this is that the frequency resolution is worse, since each part of the signal is shorter, than the full signal. \nNote, that power spectrum (not the absolute value of FT) is averaged in these methods. So, you have to square the absolute values of STFT coefficients,  then normalize them correctly, average and take the root to get ASD.",
          "votes": 2
        }
      ]
    },
    {
      "id": 2008100,
      "postDate": "2022-10-28T18:39:42.397Z",
      "content": "<p>Hi!</p>\n<p>I'm having a hard time trying to understand the relationship between the amplitude spectral density (ASD) and the STFT's absolute values (real valued) of a signal.</p>\n<p><strong>Are the STFT's coefficients (absolute values) equivalent to the ASD's values?</strong></p>\n<p>The reason behind that cuestion is to obtain STFT's absolute values of the LIGO detectors' noise from the amplitude spectral density (ASD). Here there is an example:</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5184030%2Fff757420270a70bc772b3d95eecca8cb%2FG1H1L1V1-OBSERVING_HOFT_SPECTRUM-1239321618-86400.png?generation=1666982197428544&amp;alt=media\" alt=\"\"></p>\n<p>Thank you!</p>",
      "rawMarkdown": "Hi!\n\nI'm having a hard time trying to understand the relationship between the amplitude spectral density (ASD) and the STFT's absolute values (real valued) of a signal.\n\n**Are the STFT's coefficients (absolute values) equivalent to the ASD's values?**\n\nThe reason behind that cuestion is to obtain STFT's absolute values of the LIGO detectors' noise from the amplitude spectral density (ASD). Here there is an example:\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5184030%2Fff757420270a70bc772b3d95eecca8cb%2FG1H1L1V1-OBSERVING_HOFT_SPECTRUM-1239321618-86400.png?generation=1666982197428544&alt=media)\n\nThank you!\n"
    }
  ],
  "comments": [
    {
      "id": 2009683,
      "author_name": "Konstantin Dmitriev",
      "author_url": "",
      "post_date": "2022-10-30T08:11:55.377000",
      "content": "<p>STFT is just a fourier transform (FT) done in each consequential time window ([<a href=\"https://en.wikipedia.org/wiki/Short-time_Fourier_transform#Forward_STFT](see\" target=\"_blank\">https://en.wikipedia.org/wiki/Short-time_Fourier_transform#Forward_STFT](see</a> <a href=\"https://en.wikipedia.org/wiki/Short-time_Fourier_transform#Forward_STFT))\" target=\"_blank\">https://en.wikipedia.org/wiki/Short-time_Fourier_transform#Forward_STFT))</a>. <br>\nAs our initial signal are real valued, we don't need to consider negative frequencies (see <code>return_onesided</code> parameter from the docs <a href=\"https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.stft.html)\" target=\"_blank\">https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.stft.html)</a>.</p>\n<p>On the other hand, ASD is the square root of the power spectrum (not the STFT). And the power spectrum is the squared absolute value of FT, which is equivalent to the FT multiplied by its complex conjugation. Of course, it is real non-negative frequency function, and the square root leads to real non-negative ASD. Again, the power spectrum is symmetric for the real valued initial signal, and we don't need to consider negative frequencies. The lack of negative frequencies makes ASD and STFT similar. However, the first is real valued and the second is complex valued, and their physical sences are different.</p>",
      "votes": 2,
      "replies": [
        {
          "id": 2010884,
          "author_name": "Cristo JV",
          "author_url": "",
          "post_date": "2022-10-31T08:04:27.140000",
          "content": "<p>Thank you for your answer Konstantin!<br>\nIndeed, I was refering to the absolute value of the STFT's coefficients (real valued). I've change the question because it was bad formulated.<br>\nSo, I can extract from your answer that as it is a real signal, the STFT's coefficients are equivalent to the ASD's coefficients.<br>\nIs that right? Is there any hidden assumption we should take for this statement? <br>\nCan you point me out how Parseval's theorem fits in this, and the welch's method used for spectral density stimation?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 2011009,
          "author_name": "Konstantin Dmitriev",
          "author_url": "",
          "post_date": "2022-10-31T10:22:31.100000",
          "content": "<p>Maybe I don't quite understand the question, but I'll try to make a few remarks here.<br>\n1) STFT is a \"short-time\" transform. It is used to analyze the time dependence of frequency components. On the other hand, the power spectrum (and ASD) characterizes the full signal.<br>\n2) When you do the discrete FT of some N-point signal, you may or may not normalize the result by dividing it by N. To calculate the power spectrum you divide by N the squared absolute value of not-normalized FT. As ASD is square root, it is equally divided by sqrt(N). So you may notice, it is measured in [smth/sqrt(Hz)] whilst FT is measured in [smth/Hz]. <br>\n3) When applying Welch or Bartlett methods you split the signal into parts and calculate the power spectrum of each part (<a href=\"https://en.wikipedia.org/wiki/Bartlett%27s_method)\" target=\"_blank\">https://en.wikipedia.org/wiki/Bartlett%27s_method)</a>. After that you average the results. So you reduce the noise in the estimation. The price for this is that the frequency resolution is worse, since each part of the signal is shorter, than the full signal. <br>\nNote, that power spectrum (not the absolute value of FT) is averaged in these methods. So, you have to square the absolute values of STFT coefficients,  then normalize them correctly, average and take the root to get ASD.</p>",
          "votes": 2,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2009683": "STFT is just a fourier transform (FT) done in each consequential time window ([https://en.wikipedia.org/wiki/Short-time_Fourier_transform#Forward_STFT](see https://en.wikipedia.org/wiki/Short-time_Fourier_transform#Forward_STFT)). \nAs our initial signal are real valued, we don't need to consider negative frequencies (see `return_onesided` parameter from the docs https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.stft.html).\n\nOn the other hand, ASD is the square root of the power spectrum (not the STFT). And the power spectrum is the squared absolute value of FT, which is equivalent to the FT multiplied by its complex conjugation. Of course, it is real non-negative frequency function, and the square root leads to real non-negative ASD. Again, the power spectrum is symmetric for the real valued initial signal, and we don't need to consider negative frequencies. The lack of negative frequencies makes ASD and STFT similar. However, the first is real valued and the second is complex valued, and their physical sences are different.",
    "2008100": "Hi!\n\nI'm having a hard time trying to understand the relationship between the amplitude spectral density (ASD) and the STFT's absolute values (real valued) of a signal.\n\n**Are the STFT's coefficients (absolute values) equivalent to the ASD's values?**\n\nThe reason behind that cuestion is to obtain STFT's absolute values of the LIGO detectors' noise from the amplitude spectral density (ASD). Here there is an example:\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5184030%2Fff757420270a70bc772b3d95eecca8cb%2FG1H1L1V1-OBSERVING_HOFT_SPECTRUM-1239321618-86400.png?generation=1666982197428544&alt=media)\n\nThank you!\n"
  }
}