hey guys - do you know what the unit for the \"FGS1-axiso-h\" ( axis.csv) column is? I thought this is meant to be in units of hour; but 135000 observations over 7.5 hours do not equal 0.1 sec per observation (it equals 0.2 sec on average) - any thoughts would be much appreciated! thanks!
", "messages": [ { "id": 2995114, "postDate": "2024-09-21T22:35:16.907Z", "content": "Integration time and axes information are ill-explained on the data page (if explained at all), which is a shame, imho. First, what is written on the data page regarding 0.1 integration time per step is a mistake and was not corrected (why?)- as it turns out, there is an additional 0.1s per two steps (or per step, depending on how you define) see the notebook here, in particular:
\n\n\n\n\nUPDATE 29.08: We have added 0.1s to the integration time for both AIRS and FGS observations. They are important when accounting for the contributing of dark frames when calibration the image. The modification, however, should not affect too much of the calibrated product.
\n
and:
\ndt_fgs1 = np.ones(len(fgs_signal))*0.1
\ndt_fgs1[1::2] += 0.1
Even after we take it into account, we get 135000*0.1+135000*0.1/2 = 5.265. On the other hand, when we look at the last cell in 'FGS1-axis0-h', it has a value of 7.49994, matching the last not-nan value in 'AIRS-CH0-axis0-h', which is 7.49872. Now, let's take a closer look. For FGS1, if we do:
\nmes = axis_info_df['FGS1-axis0-h'].to_numpy()
\nnp.unique(mes[1:]-mes[:-1])
Then we get two values (in reality, more than two due to floating point, but it is practically only two), which are 2.77777778e-05 and 8.33333333e-05.
\nFor the first sub-exposure, we know it's 0.1s = 0.1/3600 = 0.00002777777h. So, this matches exactly the first value.
\nThe second sub-exposure is 0.2s (remember the reference above where organizers added 0.1) so it's 0.2s = 0.2/3600 = 0.00005555555h. Smaller than the second unique value we found above, which was 8.33333333e-05.
A similar analysis will yield similar results for AIRS, with the first unique value matching 0.1s and the second one a bit longer than 4.6s.
\nThe conclusion is obvious- the 'missing' time is resetting and calibration of the pixels.
", "rawMarkdown": "Integration time and axes information are ill-explained on the data page (if explained at all), which is a shame, imho. First, what is written on the data page regarding 0.1 integration time per step is a mistake and was not corrected (why?)- as it turns out, there is an additional 0.1s per two steps (or per step, depending on how you define) see the [notebook here](https://www.kaggle.com/code/gordonyip/update-calibrating-and-binning-astronomical-data), in particular:\n>>UPDATE 29.08: We have added 0.1s to the integration time for both AIRS and FGS observations. They are important when accounting for the contributing of dark frames when calibration the image. The modification, however, should not affect too much of the calibrated product. \n\nand:\ndt_fgs1 = np.ones(len(fgs_signal))*0.1\ndt_fgs1[1::2] += 0.1\n\nEven after we take it into account, we get 135000\\*0.1+135000\\*0.1/2 = 5.265. On the other hand, when we look at the last cell in 'FGS1-axis0-h', it has a value of 7.49994, matching the last not-nan value in 'AIRS-CH0-axis0-h', which is 7.49872. Now, let's take a closer look. For FGS1, if we do:\n\nmes = axis_info_df['FGS1-axis0-h'].to_numpy()\nnp.unique(mes[1:]-mes[:-1])\n\nThen we get two values (in reality, more than two due to floating point, but it is practically only two), which are 2.77777778e-05 and 8.33333333e-05.\nFor the first sub-exposure, we know it's 0.1s = 0.1/3600 = 0.00002777777h. So, this matches exactly the first value.\nThe second sub-exposure is 0.2s (remember the reference above where organizers added 0.1) so it's 0.2s = 0.2/3600 = 0.00005555555h. Smaller than the second unique value we found above, which was 8.33333333e-05. \n\nA similar analysis will yield similar results for AIRS, with the first unique value matching 0.1s and the second one a bit longer than 4.6s. \n\nThe conclusion is obvious- the 'missing' time is resetting and calibration of the pixels.", "votes": 5, "replies": [ { "id": 2995124, "postDate": "2024-09-21T23:22:34.020Z", "content": "Thanks!! Super helpful information.
", "rawMarkdown": "Thanks!! Super helpful information. " } ] }, { "id": 2994996, "postDate": "2024-09-21T18:07:41.783Z", "content": "hey guys - do you know what the unit for the \"FGS1-axiso-h\" ( axis.csv) column is? I thought this is meant to be in units of hour; but 135000 observations over 7.5 hours do not equal 0.1 sec per observation (it equals 0.2 sec on average) - any thoughts would be much appreciated! thanks!
", "rawMarkdown": "hey guys - do you know what the unit for the \"FGS1-axiso-h\" ( axis.csv) column is? I thought this is meant to be in units of hour; but 135000 observations over 7.5 hours do not equal 0.1 sec per observation (it equals 0.2 sec on average) - any thoughts would be much appreciated! thanks!", "votes": 1 } ], "comments": [ { "id": 2995114, "author_name": "greySnow", "author_url": "", "post_date": "2024-09-21T22:35:16.907000", "content": "Integration time and axes information are ill-explained on the data page (if explained at all), which is a shame, imho. First, what is written on the data page regarding 0.1 integration time per step is a mistake and was not corrected (why?)- as it turns out, there is an additional 0.1s per two steps (or per step, depending on how you define) see the notebook here, in particular:
\n\n\n\n\nUPDATE 29.08: We have added 0.1s to the integration time for both AIRS and FGS observations. They are important when accounting for the contributing of dark frames when calibration the image. The modification, however, should not affect too much of the calibrated product.
\n
and:
\ndt_fgs1 = np.ones(len(fgs_signal))*0.1
\ndt_fgs1[1::2] += 0.1
Even after we take it into account, we get 135000*0.1+135000*0.1/2 = 5.265. On the other hand, when we look at the last cell in 'FGS1-axis0-h', it has a value of 7.49994, matching the last not-nan value in 'AIRS-CH0-axis0-h', which is 7.49872. Now, let's take a closer look. For FGS1, if we do:
\nmes = axis_info_df['FGS1-axis0-h'].to_numpy()
\nnp.unique(mes[1:]-mes[:-1])
Then we get two values (in reality, more than two due to floating point, but it is practically only two), which are 2.77777778e-05 and 8.33333333e-05.
\nFor the first sub-exposure, we know it's 0.1s = 0.1/3600 = 0.00002777777h. So, this matches exactly the first value.
\nThe second sub-exposure is 0.2s (remember the reference above where organizers added 0.1) so it's 0.2s = 0.2/3600 = 0.00005555555h. Smaller than the second unique value we found above, which was 8.33333333e-05.
A similar analysis will yield similar results for AIRS, with the first unique value matching 0.1s and the second one a bit longer than 4.6s.
\nThe conclusion is obvious- the 'missing' time is resetting and calibration of the pixels.
", "votes": 5, "replies": [ { "id": 2995124, "author_name": "Heisenger", "author_url": "", "post_date": "2024-09-21T23:22:34.020000", "content": "Thanks!! Super helpful information.
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "2995114": "Integration time and axes information are ill-explained on the data page (if explained at all), which is a shame, imho. First, what is written on the data page regarding 0.1 integration time per step is a mistake and was not corrected (why?)- as it turns out, there is an additional 0.1s per two steps (or per step, depending on how you define) see the [notebook here](https://www.kaggle.com/code/gordonyip/update-calibrating-and-binning-astronomical-data), in particular:\n>>UPDATE 29.08: We have added 0.1s to the integration time for both AIRS and FGS observations. They are important when accounting for the contributing of dark frames when calibration the image. The modification, however, should not affect too much of the calibrated product. \n\nand:\ndt_fgs1 = np.ones(len(fgs_signal))*0.1\ndt_fgs1[1::2] += 0.1\n\nEven after we take it into account, we get 135000\\*0.1+135000\\*0.1/2 = 5.265. On the other hand, when we look at the last cell in 'FGS1-axis0-h', it has a value of 7.49994, matching the last not-nan value in 'AIRS-CH0-axis0-h', which is 7.49872. Now, let's take a closer look. For FGS1, if we do:\n\nmes = axis_info_df['FGS1-axis0-h'].to_numpy()\nnp.unique(mes[1:]-mes[:-1])\n\nThen we get two values (in reality, more than two due to floating point, but it is practically only two), which are 2.77777778e-05 and 8.33333333e-05.\nFor the first sub-exposure, we know it's 0.1s = 0.1/3600 = 0.00002777777h. So, this matches exactly the first value.\nThe second sub-exposure is 0.2s (remember the reference above where organizers added 0.1) so it's 0.2s = 0.2/3600 = 0.00005555555h. Smaller than the second unique value we found above, which was 8.33333333e-05. \n\nA similar analysis will yield similar results for AIRS, with the first unique value matching 0.1s and the second one a bit longer than 4.6s. \n\nThe conclusion is obvious- the 'missing' time is resetting and calibration of the pixels.", "2994996": "hey guys - do you know what the unit for the \"FGS1-axiso-h\" ( axis.csv) column is? I thought this is meant to be in units of hour; but 135000 observations over 7.5 hours do not equal 0.1 sec per observation (it equals 0.2 sec on average) - any thoughts would be much appreciated! thanks!" } }