{
  "id": 688089,
  "title": "Scores on the easier test set ",
  "url": "/competitions/stanford-rna-3d-folding-2/discussion/688089",
  "author_name": "soltani abdellatif",
  "post_date": "2026-04-04T15:21:00.599000",
  "votes": 0,
  "comment_count": 0,
  "views": 0,
  "content": "<p>I was working on a different approach than TBM + Protenix, but I didn’t manage to get it working in time (the submission didn’t finish before the deadline). Because of that, it ended up being evaluated only on the easier private test set and not on the harder one.</p>\n<p>It doesn’t do as well as TBM + Protenix on the public leaderboard, but interestingly it does better on the easier private set. For reference, Protenix + TBM got 0.43 on public and 0.53 on the easier private set, while my method got 0.39 on public and 0.59 on the easier private set.</p>\n<p>I’m not sure if this is just high variance on the easier private set or something about the data distribution itself (I’ve seen some other pretty high scores mentioned in discussions too). I only ran both methods once, so I don’t really have enough runs to draw strong conclusions.</p>\n<p>Could you share the TBM + Protenix results on the easier private test set?<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7588330%2F9dfcce77a0b90a799400846606714f4a%2FNewRun.png?generation=1775316046525390&amp;alt=media\" alt=\"\"><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7588330%2F4e6e27afaddbc5905fdae18cd0b6bacc%2FOldRun.png?generation=1775316058900269&amp;alt=media\" alt=\"\"></p>",
  "messages": [
    {
      "id": 3435689,
      "postDate": "2026-04-04T15:21:00.600Z",
      "content": "<p>I was working on a different approach than TBM + Protenix, but I didn’t manage to get it working in time (the submission didn’t finish before the deadline). Because of that, it ended up being evaluated only on the easier private test set and not on the harder one.</p>\n<p>It doesn’t do as well as TBM + Protenix on the public leaderboard, but interestingly it does better on the easier private set. For reference, Protenix + TBM got 0.43 on public and 0.53 on the easier private set, while my method got 0.39 on public and 0.59 on the easier private set.</p>\n<p>I’m not sure if this is just high variance on the easier private set or something about the data distribution itself (I’ve seen some other pretty high scores mentioned in discussions too). I only ran both methods once, so I don’t really have enough runs to draw strong conclusions.</p>\n<p>Could you share the TBM + Protenix results on the easier private test set?<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7588330%2F9dfcce77a0b90a799400846606714f4a%2FNewRun.png?generation=1775316046525390&amp;alt=media\" alt=\"\"><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7588330%2F4e6e27afaddbc5905fdae18cd0b6bacc%2FOldRun.png?generation=1775316058900269&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "I was working on a different approach than TBM + Protenix, but I didn’t manage to get it working in time (the submission didn’t finish before the deadline). Because of that, it ended up being evaluated only on the easier private test set and not on the harder one.\n\nIt doesn’t do as well as TBM + Protenix on the public leaderboard, but interestingly it does better on the easier private set. For reference, Protenix + TBM got 0.43 on public and 0.53 on the easier private set, while my method got 0.39 on public and 0.59 on the easier private set.\n\nI’m not sure if this is just high variance on the easier private set or something about the data distribution itself (I’ve seen some other pretty high scores mentioned in discussions too). I only ran both methods once, so I don’t really have enough runs to draw strong conclusions.\n\nCould you share the TBM + Protenix results on the easier private test set?![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7588330%2F9dfcce77a0b90a799400846606714f4a%2FNewRun.png?generation=1775316046525390&alt=media)![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7588330%2F4e6e27afaddbc5905fdae18cd0b6bacc%2FOldRun.png?generation=1775316058900269&alt=media)"
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "3435689": "I was working on a different approach than TBM + Protenix, but I didn’t manage to get it working in time (the submission didn’t finish before the deadline). Because of that, it ended up being evaluated only on the easier private test set and not on the harder one.\n\nIt doesn’t do as well as TBM + Protenix on the public leaderboard, but interestingly it does better on the easier private set. For reference, Protenix + TBM got 0.43 on public and 0.53 on the easier private set, while my method got 0.39 on public and 0.59 on the easier private set.\n\nI’m not sure if this is just high variance on the easier private set or something about the data distribution itself (I’ve seen some other pretty high scores mentioned in discussions too). I only ran both methods once, so I don’t really have enough runs to draw strong conclusions.\n\nCould you share the TBM + Protenix results on the easier private test set?![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7588330%2F9dfcce77a0b90a799400846606714f4a%2FNewRun.png?generation=1775316046525390&alt=media)![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7588330%2F4e6e27afaddbc5905fdae18cd0b6bacc%2FOldRun.png?generation=1775316058900269&alt=media)"
  }
}