When generating training data using PyFstat there are many parameters to be tuned and the data section of this competition also states there are 8 randomised parameters.
\nThe PyFstat documentation is however not very helpful at explaining the parameters, giving explanations like:
Parameters at which to compute the statistic.Additional frequency evolution and amplitude parameters for a signal.The signal generating tutorial does elaborate on some of these parameters, however it is still not fully clear what reasonable value ranges would be.
\nCould someone explain (in simple terms) how the following parameters affect the signal and what the valid/reasonable range of values would be?
\n1) phi
\n2) psi
\n3) cosi
\n4) F0
\n5) F1
\n6) F2
\n7) Band
\n8) Alpha
\n9) Delta
\n10) tp
\n11) h0
\n12) asini
\n13) period
When generating training data using PyFstat there are many parameters to be tuned and the data section of this competition also states there are 8 randomised parameters.
\nThe PyFstat documentation is however not very helpful at explaining the parameters, giving explanations like:
Parameters at which to compute the statistic.Additional frequency evolution and amplitude parameters for a signal.The signal generating tutorial does elaborate on some of these parameters, however it is still not fully clear what reasonable value ranges would be.
\nCould someone explain (in simple terms) how the following parameters affect the signal and what the valid/reasonable range of values would be?
\n1) phi
\n2) psi
\n3) cosi
\n4) F0
\n5) F1
\n6) F2
\n7) Band
\n8) Alpha
\n9) Delta
\n10) tp
\n11) h0
\n12) asini
\n13) period
As explained in the tutorials, you can use a helper function to inject some parameters based on priors:
\n{**pyfstat.injection_parameters.isotropic_amplitude_priors}
\n{'cosi': {'uniform': {'low': -1.0, 'high': 1.0}},\n 'psi': {'uniform': {'low': -0.7853981633974483, 'high': 0.7853981633974483}},\n 'phi': {'uniform': {'low': 0, 'high': 6.283185307179586}}}
This already answer part of your question. Those are the valid ranges for cosi, psi and phi, which are expected to be isotropically distributed (uniform distribution, no preference for any particular value).
0.2 hz for all files in the train and test datasets, which translates to 360 \"frequency lines\".0 for the purposes of this competition.sqrtSX. What makes the competition (and the real-world problem of finding continuous GWs) is that expected h0 will be between 1 and 2 orders of magnitude smaller than the amplitude of the noise. In my opinion, part of the challenge is to create your own signals with a distribution of h0 similar to the one used for the test set.Regarding other parameters you mentioned like yp, asini, periodI think they are used as an alternative way of specifying the pyhsical parameters of the souce and you can ignore them:
\n", "rawMarkdown": "As explained in the tutorials, you can use a helper function to inject some parameters based on priors:\n`{**pyfstat.injection_parameters.isotropic_amplitude_priors}`\n`{'cosi': {'uniform': {'low': -1.0, 'high': 1.0}},\n 'psi': {'uniform': {'low': -0.7853981633974483, 'high': 0.7853981633974483}},\n 'phi': {'uniform': {'low': 0, 'high': 6.283185307179586}}}`\n\nThis already answer part of your question. Those are the valid ranges for `cosi`, `psi` and `phi`, which are expected to be isotropically distributed (uniform distribution, no preference for any particular value). \n\n- **cosi**: Cosine of the angle between the source and us. Range: [-1, 1]\n- **psi** and **phi**: polarization angle and phase. Ranges: [-pi/2, pi/2] and [0, 2*pi]\n- **Alpha**: Right ascension of the source's position on the sky. Range: [0, 2*pi]\n- **Delta**: Declination of the source's position on the sky. Range: [-pi/2, pi/2]\n- **Band**: This is just the width of the frequency \"slice\" to be taken. It's `0.2 hz` for all files in the train and test datasets, which translates to 360 \"frequency lines\".\n- **F0**: This may be tricky. The generated noise will be centered around this frequency. If a signal is created, it will be also a signal with this frequency (at the source; when it arrives to the detector, doppler and other effects will shift the frequency), so it will appear more or less on the center of the data. Range used on the datasets: [50, 400] Hz.\n- **F1**: This is technically the first derivative of the frequency (how it changs over time). Physically, the sources (neutron stars) lose energy over time because they emit the GWs, so this should be a negative number. A reasonable range could be [-1e-9, 0] Hz but that's up to us.\n- **F2**: This is guess it's the second derivative, which maybe is `0` for the purposes of this competition.\n- **h0**: The amplitude of the signal. In the tutorials, the approach is to define it as a ratio of the `sqrtSX`. What makes the competition (and the real-world problem of finding continuous GWs) is that expected `h0` will be between 1 and 2 orders of magnitude smaller than the amplitude of the noise. In my opinion, part of the challenge is to create your own signals with a distribution of `h0` similar to the one used for the test set.\n\nRegarding other parameters you mentioned like `yp, asini, period `I think they are used as an alternative way of specifying the pyhsical parameters of the souce and you can ignore them:\n> Depending on the emission mechanism, h0 can be further described using further physical quantities such as the source's frequency, ellipticity, or distance to the detector.\n\n", "votes": 27, "replies": [ { "id": 2002131, "postDate": "2022-10-24T14:30:05.977Z", "content": "Depending on the emission mechanism, h0 can be further described using further physical quantities such as the source's frequency, ellipticity, or distance to the detector.
\n
Hi, Victor
\n\"Range used on the datasets: [50, 400] Hz.\"
\nThe range goes up to 500 Hz
You are right! I was writing that from memory.
", "rawMarkdown": "You are right! I was writing that from memory." } ] }, { "id": 2001689, "postDate": "2022-10-24T08:40:32.673Z", "content": "Hi @markwijkhuizen ,
\nLooks like @darkbitur did a good job giving an explanation, π.
\nIf I just may add the following:
\npsi is actually defined within [-pi /4 , pi / 4] (and it's periodic, so anything beyond that folds back onto this range).F2 is another term describing the intrinsic evolution of a gravitational wave. As @darkbitur correctly guesses, it's set to 0 in this competition as we only focus on frequency and spindown.tp, asini, period are parameters used to describe neutron stars in binary systems, and they affect the frequency evolution of the signal. You can safely ignore those in this competition (but do feel free to play around with them if you wish to!).\n\nDepending on the emission mechanism, h0 can be further described using further physical quantities such as the source's frequency, ellipticity, or distance to the detector.
\n
As for this, we normally don't worry about the physics at search time: We can safely encapsulate all that information on h0, as that would be what our detectors would basically measure (gory details: the signal is actually amplitude modulated depending on psi and Alpha, Delta, and then cosi acts as an overall scaling on top). Once the search is done and we get an upper bound of h0 we work out the physical consequences of that (e.g. how far away could we have seen a star with a given ellipticity, how much ellipticity can nearby NS sustainβ¦).
Hi @rodrigotenorio , Thank you for the great competition. I have questions about the F1 parameters.
\nF1: This is technically the first derivative of the frequency (how it changes over time). Physically, the sources (neutron stars) lose energy over time because they emit the GWs, so this should be a negative number. A reasonable range could be [-1e-9, 0] Hz but that's up to us.
\nAnd you also mentioned that it should be negative in your topics.
However, it will raise an error when the F1 is set to any negative value. So should we set it as positive value in this competition? Or it's something that can't tell competitors.(If so, please also tell me)
The error is
\nCalledProcessError: Command 'lalapps_Makefakedata_v5 --outSingleSFT=TRUE --outSFTdir=\".\" --outLabel=\"custom_band_signal\" --noiseSFTs=\"./H-5760_H1_1800SFT_custom_band_noise-1238166018-10368000.sft;./L-5760_L1_1800SFT_custom_band_noise-1238166018-10368000.sft\" --SFTWindowType=\"tukey\" --SFTWindowBeta=0.001 --startTime=1238166018 --duration=10368000 --Tsft=1800 --injectionSources=\"./custom_band_signal.cff\" --ephemEarth=\"earth00-40-DE405.dat.gz\" --ephemSun=\"sun00-40-DE405.dat.gz\"' returned non-zero exit status 255.
----------------------------------------Edited--------------------------------------
\nMy badπ€. It isn't about positive/negetive. It's about the abs of F1. No matter F1 positive or negative, it's abs should be no larger than 1e-8.
\nStill, I would be very grateful if you can tell me whether the F1 used in this competition is positive, or negative, or both. π
I got exactly the same error. Somewhere in examples F1 was set to 10**stats.uniform(-12, 4), so it could be significantly larger, than 1e-8, which produces the error code 255.
On the other hand, we have the records with the duration of approximately 1e7 seconds. The half band is 0.1Hz. So, to make an estimation, let's consider a linear frequency modulated signal starting with some F0 in the center of the band and having frequency derivative of F1. It can be easily seen, that the condition of |F1|<1e-8 is equivalent to that the signal is fully located inside the selected frequency band.
10**stats.uniform(-12, 4) - the API for stats is different from np.uniform. In this case it samples from -12 to -8
", "rawMarkdown": "10**stats.uniform(-12, 4) - the API for stats is different from np.uniform. In this case it samples from -12 to -8", "votes": 2 }, { "id": 2016903, "postDate": "2022-11-04T10:44:19.600Z", "content": "Hi @forcewithme , @kdmitrie ,
\nFirst, note that the error you pasted is incomplete: Your error is likely happening at LALSuite level (i.e. while Writer calls a command-line process) and hence will be printed to STDOUT. You should have gotten a few printed lines saying XLAL_ERROR or similar.
\n\nSomewhere in examples F1 was set to 10**stats.uniform(-12, 4), so it could be significantly larger, than 1e-8
\n
Note that stats.uniform(-12, 4) generates random numbers in [-12, -8] (i.e. scipy's uniform notation is different than numpy's). EDIT: Looks like I overlapped with @sakvaua
As @kdmitrie points out, if you get spindown values with too much of an absolute value your signal is probably going to drift out of the 0.2Hz frequency band.
\n\n\nStill, I would be very grateful if you can tell me whether the F1 used in this competition is positive, or negative, or both. π
\n
Both signs of F1 are relevant from the point of view of continuous wave searches :).
Thank you so much! π
", "rawMarkdown": "Thank you so much! π" }, { "id": 2016938, "postDate": "2022-11-04T11:24:35.313Z", "content": "Yeah @sakvaua ,
\nI agree10**stats.uniform(-12, 4) - the API for stats is different from np.uniform. In this case is samples from -12 to -8 .
But I guess 10**stats.uniform(-12, 4) may be the right choice. Since I think we do need to sample F1 continuously, rather than chossing from 1e-12 to 1e-8 discretely.
As explained in the tutorials, you can use a helper function to inject some parameters based on priors:
\n{**pyfstat.injection_parameters.isotropic_amplitude_priors}
\n{'cosi': {'uniform': {'low': -1.0, 'high': 1.0}},\n 'psi': {'uniform': {'low': -0.7853981633974483, 'high': 0.7853981633974483}},\n 'phi': {'uniform': {'low': 0, 'high': 6.283185307179586}}}
This already answer part of your question. Those are the valid ranges for cosi, psi and phi, which are expected to be isotropically distributed (uniform distribution, no preference for any particular value).
0.2 hz for all files in the train and test datasets, which translates to 360 \"frequency lines\".0 for the purposes of this competition.sqrtSX. What makes the competition (and the real-world problem of finding continuous GWs) is that expected h0 will be between 1 and 2 orders of magnitude smaller than the amplitude of the noise. In my opinion, part of the challenge is to create your own signals with a distribution of h0 similar to the one used for the test set.Regarding other parameters you mentioned like yp, asini, periodI think they are used as an alternative way of specifying the pyhsical parameters of the souce and you can ignore them:
\n", "votes": 27, "replies": [ { "id": 2002131, "author_name": "DennisSakva", "author_url": "", "post_date": "2022-10-24T14:30:05.977000", "content": "Depending on the emission mechanism, h0 can be further described using further physical quantities such as the source's frequency, ellipticity, or distance to the detector.
\n
Hi, Victor
\n\"Range used on the datasets: [50, 400] Hz.\"
\nThe range goes up to 500 Hz
You are right! I was writing that from memory.
", "votes": 0, "replies": [] } ] }, { "id": 2001689, "author_name": "Rodrigo Tenorio", "author_url": "", "post_date": "2022-10-24T08:40:32.673000", "content": "Hi @markwijkhuizen ,
\nLooks like @darkbitur did a good job giving an explanation, π.
\nIf I just may add the following:
\npsi is actually defined within [-pi /4 , pi / 4] (and it's periodic, so anything beyond that folds back onto this range).F2 is another term describing the intrinsic evolution of a gravitational wave. As @darkbitur correctly guesses, it's set to 0 in this competition as we only focus on frequency and spindown.tp, asini, period are parameters used to describe neutron stars in binary systems, and they affect the frequency evolution of the signal. You can safely ignore those in this competition (but do feel free to play around with them if you wish to!).\n\nDepending on the emission mechanism, h0 can be further described using further physical quantities such as the source's frequency, ellipticity, or distance to the detector.
\n
As for this, we normally don't worry about the physics at search time: We can safely encapsulate all that information on h0, as that would be what our detectors would basically measure (gory details: the signal is actually amplitude modulated depending on psi and Alpha, Delta, and then cosi acts as an overall scaling on top). Once the search is done and we get an upper bound of h0 we work out the physical consequences of that (e.g. how far away could we have seen a star with a given ellipticity, how much ellipticity can nearby NS sustainβ¦).
Hi @rodrigotenorio , Thank you for the great competition. I have questions about the F1 parameters.
\nF1: This is technically the first derivative of the frequency (how it changes over time). Physically, the sources (neutron stars) lose energy over time because they emit the GWs, so this should be a negative number. A reasonable range could be [-1e-9, 0] Hz but that's up to us.
\nAnd you also mentioned that it should be negative in your topics.
However, it will raise an error when the F1 is set to any negative value. So should we set it as positive value in this competition? Or it's something that can't tell competitors.(If so, please also tell me)
The error is
\nCalledProcessError: Command 'lalapps_Makefakedata_v5 --outSingleSFT=TRUE --outSFTdir=\".\" --outLabel=\"custom_band_signal\" --noiseSFTs=\"./H-5760_H1_1800SFT_custom_band_noise-1238166018-10368000.sft;./L-5760_L1_1800SFT_custom_band_noise-1238166018-10368000.sft\" --SFTWindowType=\"tukey\" --SFTWindowBeta=0.001 --startTime=1238166018 --duration=10368000 --Tsft=1800 --injectionSources=\"./custom_band_signal.cff\" --ephemEarth=\"earth00-40-DE405.dat.gz\" --ephemSun=\"sun00-40-DE405.dat.gz\"' returned non-zero exit status 255.
----------------------------------------Edited--------------------------------------
\nMy badπ€. It isn't about positive/negetive. It's about the abs of F1. No matter F1 positive or negative, it's abs should be no larger than 1e-8.
\nStill, I would be very grateful if you can tell me whether the F1 used in this competition is positive, or negative, or both. π
I got exactly the same error. Somewhere in examples F1 was set to 10**stats.uniform(-12, 4), so it could be significantly larger, than 1e-8, which produces the error code 255.
On the other hand, we have the records with the duration of approximately 1e7 seconds. The half band is 0.1Hz. So, to make an estimation, let's consider a linear frequency modulated signal starting with some F0 in the center of the band and having frequency derivative of F1. It can be easily seen, that the condition of |F1|<1e-8 is equivalent to that the signal is fully located inside the selected frequency band.
10**stats.uniform(-12, 4) - the API for stats is different from np.uniform. In this case it samples from -12 to -8
", "votes": 2, "replies": [] }, { "id": 2016903, "author_name": "Rodrigo Tenorio", "author_url": "", "post_date": "2022-11-04T10:44:19.600000", "content": "Hi @forcewithme , @kdmitrie ,
\nFirst, note that the error you pasted is incomplete: Your error is likely happening at LALSuite level (i.e. while Writer calls a command-line process) and hence will be printed to STDOUT. You should have gotten a few printed lines saying XLAL_ERROR or similar.
\n\nSomewhere in examples F1 was set to 10**stats.uniform(-12, 4), so it could be significantly larger, than 1e-8
\n
Note that stats.uniform(-12, 4) generates random numbers in [-12, -8] (i.e. scipy's uniform notation is different than numpy's). EDIT: Looks like I overlapped with @sakvaua
As @kdmitrie points out, if you get spindown values with too much of an absolute value your signal is probably going to drift out of the 0.2Hz frequency band.
\n\n\nStill, I would be very grateful if you can tell me whether the F1 used in this competition is positive, or negative, or both. π
\n
Both signs of F1 are relevant from the point of view of continuous wave searches :).
Thank you so much! π
", "votes": 0, "replies": [] }, { "id": 2016938, "author_name": "ForcewithMe", "author_url": "", "post_date": "2022-11-04T11:24:35.313000", "content": "Yeah @sakvaua ,
\nI agree10**stats.uniform(-12, 4) - the API for stats is different from np.uniform. In this case is samples from -12 to -8 .
But I guess 10**stats.uniform(-12, 4) may be the right choice. Since I think we do need to sample F1 continuously, rather than chossing from 1e-12 to 1e-8 discretely.