{
  "id": 539859,
  "title": "Looking at variations in axis 1",
  "url": "/competitions/ariel-data-challenge-2024/discussion/539859",
  "author_name": "KaiH",
  "post_date": "2024-10-11T05:43:12.037000",
  "votes": 0,
  "comment_count": 1,
  "views": 0,
  "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10530406%2F7c7d477675ff87545abb30422b2af50f%2Fdownload.png?generation=1728625089040267&amp;alt=media\" alt=\"\"></p>\n<p>So I have been checking out variations in axis 1 as the satellite is jittering in space and I noticed that the place where it is strongest can deviate from the center sometimes. So I basically take the z-score and do a sort of a weighted average with all the indexes to get these charts. Also index 5 and 6 is where the light is centered to so the graphs look correct (5.5 is like the mean of those two). I'm not sure what to do with this, does anyone have any ideas?</p>\n<p>Here is the code:</p>\n<pre><code>plt.figure(figsize=(*, *))\n i  tqdm(()):\n    plt.subplot(, , i+)\n    correctedVals = outPlanet(adcDf.index[i], adcDf, trainType=)[:, :, :]\n\n    correctedVals = (correctedVals - np.nanmean(correctedVals, axis=())[, :, :]) / np.nanstd(correctedVals, axis=())[, :, :]\n\n    minI, maxI = , \n    newVals = np.exp(correctedVals[:, :, minI:maxI]) / np.nansum(np.exp(correctedVals[:, :, minI:maxI]), axis=)[:, , :]\n    newVals = np.nansum(newVals * np.arange()[, :, ], axis=) / np.nansum(newVals, axis=)\n    newVals = np.nanmean(newVals, axis=)\n\n    sns.scatterplot(moving_average(newVals, ))\n</code></pre>",
  "messages": [
    {
      "id": 3014416,
      "postDate": "2024-10-11T08:21:49.147Z",
      "content": "<p>Yes, these outliers are just strong noise that practically disappears when summed along the axis. The change associated with transit, on the contrary, is only getting stronger. It seems that this effect is related to the distribution of the amount of flux between neighboring pixels.</p>",
      "rawMarkdown": "Yes, these outliers are just strong noise that practically disappears when summed along the axis. The change associated with transit, on the contrary, is only getting stronger. It seems that this effect is related to the distribution of the amount of flux between neighboring pixels."
    },
    {
      "id": 3014285,
      "postDate": "2024-10-11T05:43:12.037Z",
      "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10530406%2F7c7d477675ff87545abb30422b2af50f%2Fdownload.png?generation=1728625089040267&amp;alt=media\" alt=\"\"></p>\n<p>So I have been checking out variations in axis 1 as the satellite is jittering in space and I noticed that the place where it is strongest can deviate from the center sometimes. So I basically take the z-score and do a sort of a weighted average with all the indexes to get these charts. Also index 5 and 6 is where the light is centered to so the graphs look correct (5.5 is like the mean of those two). I'm not sure what to do with this, does anyone have any ideas?</p>\n<p>Here is the code:</p>\n<pre><code>plt.figure(figsize=(*, *))\n i  tqdm(()):\n    plt.subplot(, , i+)\n    correctedVals = outPlanet(adcDf.index[i], adcDf, trainType=)[:, :, :]\n\n    correctedVals = (correctedVals - np.nanmean(correctedVals, axis=())[, :, :]) / np.nanstd(correctedVals, axis=())[, :, :]\n\n    minI, maxI = , \n    newVals = np.exp(correctedVals[:, :, minI:maxI]) / np.nansum(np.exp(correctedVals[:, :, minI:maxI]), axis=)[:, , :]\n    newVals = np.nansum(newVals * np.arange()[, :, ], axis=) / np.nansum(newVals, axis=)\n    newVals = np.nanmean(newVals, axis=)\n\n    sns.scatterplot(moving_average(newVals, ))\n</code></pre>",
      "rawMarkdown": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10530406%2F7c7d477675ff87545abb30422b2af50f%2Fdownload.png?generation=1728625089040267&alt=media)\n\n\n\nSo I have been checking out variations in axis 1 as the satellite is jittering in space and I noticed that the place where it is strongest can deviate from the center sometimes. So I basically take the z-score and do a sort of a weighted average with all the indexes to get these charts. Also index 5 and 6 is where the light is centered to so the graphs look correct (5.5 is like the mean of those two). I'm not sure what to do with this, does anyone have any ideas?\n\nHere is the code:\n```python\nplt.figure(figsize=(6*2, 5*6))\nfor i in tqdm(range(12)):\n    plt.subplot(6, 2, i+1)\n    correctedVals = outPlanet(adcDf.index[i], adcDf, trainType='train')[:, 10:22, 39:321]\n    \n    correctedVals = (correctedVals - np.nanmean(correctedVals, axis=(0))[None, :, :]) / np.nanstd(correctedVals, axis=(0))[None, :, :]\n    \n    minI, maxI = 0, 282\n    newVals = np.exp(correctedVals[:, :, minI:maxI]) / np.nansum(np.exp(correctedVals[:, :, minI:maxI]), axis=1)[:, None, :]\n    newVals = np.nansum(newVals * np.arange(12)[None, :, None], axis=1) / np.nansum(newVals, axis=1)\n    newVals = np.nanmean(newVals, axis=1)\n\n    sns.scatterplot(moving_average(newVals, 500))\n```"
    }
  ],
  "comments": [
    {
      "id": 3014416,
      "author_name": "Sergei Fironov",
      "author_url": "",
      "post_date": "2024-10-11T08:21:49.147000",
      "content": "<p>Yes, these outliers are just strong noise that practically disappears when summed along the axis. The change associated with transit, on the contrary, is only getting stronger. It seems that this effect is related to the distribution of the amount of flux between neighboring pixels.</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3014416": "Yes, these outliers are just strong noise that practically disappears when summed along the axis. The change associated with transit, on the contrary, is only getting stronger. It seems that this effect is related to the distribution of the amount of flux between neighboring pixels.",
    "3014285": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10530406%2F7c7d477675ff87545abb30422b2af50f%2Fdownload.png?generation=1728625089040267&alt=media)\n\n\n\nSo I have been checking out variations in axis 1 as the satellite is jittering in space and I noticed that the place where it is strongest can deviate from the center sometimes. So I basically take the z-score and do a sort of a weighted average with all the indexes to get these charts. Also index 5 and 6 is where the light is centered to so the graphs look correct (5.5 is like the mean of those two). I'm not sure what to do with this, does anyone have any ideas?\n\nHere is the code:\n```python\nplt.figure(figsize=(6*2, 5*6))\nfor i in tqdm(range(12)):\n    plt.subplot(6, 2, i+1)\n    correctedVals = outPlanet(adcDf.index[i], adcDf, trainType='train')[:, 10:22, 39:321]\n    \n    correctedVals = (correctedVals - np.nanmean(correctedVals, axis=(0))[None, :, :]) / np.nanstd(correctedVals, axis=(0))[None, :, :]\n    \n    minI, maxI = 0, 282\n    newVals = np.exp(correctedVals[:, :, minI:maxI]) / np.nansum(np.exp(correctedVals[:, :, minI:maxI]), axis=1)[:, None, :]\n    newVals = np.nansum(newVals * np.arange(12)[None, :, None], axis=1) / np.nansum(newVals, axis=1)\n    newVals = np.nanmean(newVals, axis=1)\n\n    sns.scatterplot(moving_average(newVals, 500))\n```"
  }
}