{
  "id": 69982,
  "title": "FYI: Not All Samples Are Created Equal: Deep Learning with Importance Sampling",
  "url": "/competitions/quickdraw-doodle-recognition/discussion/69982",
  "author_name": "hengck23",
  "post_date": "2018-10-29T15:20:17.693000",
  "votes": 16,
  "comment_count": 2,
  "views": 0,
  "content": "<p><a href=\"https://arxiv.org/pdf/1803.00942.pdf\">https://arxiv.org/pdf/1803.00942.pdf</a></p>\n\n<p>Not All Samples Are Created Equal:Deep Learning with Importance Sampling\n- Angelos Katharopoulos, Francois Fleuret</p>\n\n<p>Deep neural network training spends most of the computation on examples that are properly handled, and could be ignored. We propose to mitigate this phenomenon with a principled importance sampling scheme that focuses computation on “informative” examples,\nand reduces the variance of the stochastic gradients during training</p>\n\n<hr>",
  "messages": [
    {
      "id": 412112,
      "postDate": "2018-10-29T15:20:17.693Z",
      "content": "<p><a href=\"https://arxiv.org/pdf/1803.00942.pdf\">https://arxiv.org/pdf/1803.00942.pdf</a></p>\n\n<p>Not All Samples Are Created Equal:Deep Learning with Importance Sampling\n- Angelos Katharopoulos, Francois Fleuret</p>\n\n<p>Deep neural network training spends most of the computation on examples that are properly handled, and could be ignored. We propose to mitigate this phenomenon with a principled importance sampling scheme that focuses computation on “informative” examples,\nand reduces the variance of the stochastic gradients during training</p>\n\n<hr>",
      "rawMarkdown": "https://arxiv.org/pdf/1803.00942.pdf\n\nNot All Samples Are Created Equal:Deep Learning with Importance Sampling\n- Angelos Katharopoulos, Francois Fleuret\n\nDeep neural network training spends most of the computation on examples that are properly handled, and could be ignored. We propose to mitigate this phenomenon with a principled importance sampling scheme that focuses computation on “informative” examples,\nand reduces the variance of the stochastic gradients during training\n\n----\n\n\n",
      "votes": 16
    },
    {
      "id": 412156,
      "postDate": "2018-10-29T17:38:54.440Z",
      "content": "<p>Thanks, Heng.</p>\n\n<p>quick question though: this paper seems to define most important samples as \"the samples\nthat will introduce the biggest change in the parameters\nwhich reduces the variance of the gradient estimates.\"</p>\n\n<p>Does this mean drawings like this \"bowtie1\" would be more important than \"bowtie2\"?</p>",
      "rawMarkdown": "Thanks, Heng.\n\nquick question though: this paper seems to define most important samples as \"the samples\nthat will introduce the biggest change in the parameters\nwhich reduces the variance of the gradient estimates.\"\n\nDoes this mean drawings like this \"bowtie1\" would be more important than \"bowtie2\"?\n"
    },
    {
      "id": 413134,
      "postDate": "2018-10-31T10:43:47.053Z",
      "content": "<p>Thanks</p>",
      "rawMarkdown": "Thanks"
    }
  ],
  "comments": [
    {
      "id": 412156,
      "author_name": "Michael Tam",
      "author_url": "",
      "post_date": "2018-10-29T17:38:54.440000",
      "content": "<p>Thanks, Heng.</p>\n\n<p>quick question though: this paper seems to define most important samples as \"the samples\nthat will introduce the biggest change in the parameters\nwhich reduces the variance of the gradient estimates.\"</p>\n\n<p>Does this mean drawings like this \"bowtie1\" would be more important than \"bowtie2\"?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 413134,
      "author_name": "YaGana Sheriff-Hussaini",
      "author_url": "",
      "post_date": "2018-10-31T10:43:47.053000",
      "content": "<p>Thanks</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "412112": "https://arxiv.org/pdf/1803.00942.pdf\n\nNot All Samples Are Created Equal:Deep Learning with Importance Sampling\n- Angelos Katharopoulos, Francois Fleuret\n\nDeep neural network training spends most of the computation on examples that are properly handled, and could be ignored. We propose to mitigate this phenomenon with a principled importance sampling scheme that focuses computation on “informative” examples,\nand reduces the variance of the stochastic gradients during training\n\n----\n\n\n",
    "412156": "Thanks, Heng.\n\nquick question though: this paper seems to define most important samples as \"the samples\nthat will introduce the biggest change in the parameters\nwhich reduces the variance of the gradient estimates.\"\n\nDoes this mean drawings like this \"bowtie1\" would be more important than \"bowtie2\"?\n",
    "413134": "Thanks"
  }
}