![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4308868%2Fdc5855c34dcd37e930419ec3f9039cf2%2F__results___32_1.png?generation=1668240677739409&alt=media\")
Do we know if any instrumental artifacts (persistent line-like features) are added to the noise alongside the simulated signals in testing dataset?
\nIt seems reasonable to assume they might be present, at least in the samples with real noise background. But what about the samples with simulated noise?
", "messages": [ { "id": 2022068, "postDate": "2022-11-08T17:03:33.540Z", "content": "Do we know if any instrumental artifacts (persistent line-like features) are added to the noise alongside the simulated signals in testing dataset?
\nIt seems reasonable to assume they might be present, at least in the samples with real noise background. But what about the samples with simulated noise?
", "rawMarkdown": "Do we know if any instrumental artifacts (persistent line-like features) are added to the noise alongside the simulated signals in testing dataset? \n\nIt seems reasonable to assume they might be present, at least in the samples with real noise background. But what about the samples with simulated noise?\n", "votes": 16 }, { "id": 2026621, "postDate": "2022-11-12T08:21:47.840Z", "content": "Yes, the artifacts like persistent lines do exist alongside with the CW signal and background noise. I've done some statistical testing of the whole dataset (test+train) and found them.
Here, on the L1 detector we can see frequency-modulated CGW signal, and on the H1 detector there is a constant frequency interference. When this interference is suppressed, the CGW signal becomes visible on H1 also.
\nI've shared the code, which I used to find such \"strange\" records https://www.kaggle.com/code/kdmitrie/g2net-exporing-test-train-datasets.
", "rawMarkdown": "Yes, the artifacts like persistent lines do exist alongside with the CW signal and background noise. I've done some statistical testing of the whole dataset (test+train) and found them. \n\nHere, on the L1 detector we can see frequency-modulated CGW signal, and on the H1 detector there is a constant frequency interference. When this interference is suppressed, the CGW signal becomes visible on H1 also.\n\nI've shared the code, which I used to find such \"strange\" records [https://www.kaggle.com/code/kdmitrie/g2net-exporing-test-train-datasets](https://www.kaggle.com/code/kdmitrie/g2net-exporing-test-train-datasets).", "votes": 10 }, { "id": 2022738, "postDate": "2022-11-09T08:30:10.190Z", "content": "I haven't seen any artifacts on the generated samples. On the real ones, there are all kinds of glitches, persistent lines (some narrow, some very wide), and in all of them, non-stationary noise. In my opinion all of them are real and not added to the real noise: real data is complicated enough!
\nAlso I think that the real samples have had some kind of cleaning step, maybe one that's done on the real LIGO pipeline, since if I recall correctly, I couldn't find some expected noise lines like the one at 60hz, etc. There's even a catalog for them.
", "rawMarkdown": "I haven't seen any artifacts on the generated samples. On the real ones, there are all kinds of glitches, persistent lines (some narrow, some very wide), and in all of them, non-stationary noise. In my opinion all of them are real and not added to the real noise: real data is complicated enough!\n\nAlso I think that the real samples have had some kind of cleaning step, maybe one that's done on the real LIGO pipeline, since if I recall correctly, I couldn't find some expected noise lines like the one at 60hz, etc. There's even a catalog for them.", "votes": 2 }, { "id": 2022856, "postDate": "2022-11-09T10:50:00.550Z", "content": "Agree with Victor Gonzalez - all kinds of instrumental artifacts in the testing dataset…
\nI did a spectrogram of about 1000 files and some seem badly disfigured.. (credit to Jun Koda for the spectrogram code)
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6492023%2Fc7b0b6f1658f80ab7a8814ebccf657e6%2Fcapture.jpg?generation=1667990809480582&alt=media\")
Yes, the artifacts like persistent lines do exist alongside with the CW signal and background noise. I've done some statistical testing of the whole dataset (test+train) and found them.
Here, on the L1 detector we can see frequency-modulated CGW signal, and on the H1 detector there is a constant frequency interference. When this interference is suppressed, the CGW signal becomes visible on H1 also.
\nI've shared the code, which I used to find such \"strange\" records https://www.kaggle.com/code/kdmitrie/g2net-exporing-test-train-datasets.
", "votes": 10, "replies": [] }, { "id": 2022738, "author_name": "Victor Gonzalez", "author_url": "", "post_date": "2022-11-09T08:30:10.190000", "content": "I haven't seen any artifacts on the generated samples. On the real ones, there are all kinds of glitches, persistent lines (some narrow, some very wide), and in all of them, non-stationary noise. In my opinion all of them are real and not added to the real noise: real data is complicated enough!
\nAlso I think that the real samples have had some kind of cleaning step, maybe one that's done on the real LIGO pipeline, since if I recall correctly, I couldn't find some expected noise lines like the one at 60hz, etc. There's even a catalog for them.
", "votes": 2, "replies": [] }, { "id": 2022856, "author_name": "", "author_url": "", "post_date": "2022-11-09T10:50:00.550000", "content": "Agree with Victor Gonzalez - all kinds of instrumental artifacts in the testing dataset…
\nI did a spectrogram of about 1000 files and some seem badly disfigured.. (credit to Jun Koda for the spectrogram code)
\n