{ "id": 189546, "title": "Would appreciate any comment on this TensorBoard log on DenseNet-121 transfer learning", "url": "/competitions/rsna-str-pulmonary-embolism-detection/discussion/189546", "author_name": "권 진혁 (Jin Kwon)", "post_date": "2020-10-07T21:55:42.847000", "votes": 0, "comment_count": 3, "views": 0, "content": "

Hello Kagglers!

\n

Link to TensorBoard

\n

I am new to Deep Learning and currently running transfer learning on DenseNet-121.
\nI kept this running for more than 2 days now, and just wanted to hear some suggestions on how I can improve the training.
\nJust wanted to get some open feedbacks from experienced Kagglers on model structure, convergence, anything that looks odd or interesting, etc.

\n

Two trainings that are currently running are:

\n
    \n
  1. scalars/20201004-101143/train & validation: Dense-121 with only 3-channel image as input
    2. \n
  2. scalars/20201004-101158/train & validation: Dense-121 with 3-channel image + image location within study (the z-axis so to speak, indicating whether the image is closer to toe or head).
    3. \n
\n

Will share more details if anyone's interested.

\n

Will appreciate any suggestions or comments!

\n

Thanks

", "messages": [ { "id": 1042034, "postDate": "2020-10-08T03:04:05.523Z", "content": "

A little suggestion. Since the negative cases is much more than positive cases, so accuracy might not be a good metric for such condition. Maybe F1 score/AUC.. will be better.

", "rawMarkdown": "A little suggestion. Since the negative cases is much more than positive cases, so accuracy might not be a good metric for such condition. Maybe F1 score/AUC.. will be better.", "votes": 3 }, { "id": 1043143, "postDate": "2020-10-08T18:08:00.407Z", "content": "

@xiejialun Thank you very much,

\n

After your comment I am re-training (from the latest trained weights) with accuracy metric = AUC.
\nSeeing that the training was not going very good as for most of the labels, the AUC values were ~0.5.

\n

As being a beginner, all suggestions and comments are golden! So thanks again for taking the time to review and make your comment :)

", "rawMarkdown": "@xiejialun Thank you very much, \n\nAfter your comment I am re-training (from the latest trained weights) with accuracy metric = AUC. \nSeeing that the training was not going very good as for most of the labels, the AUC values were ~0.5. \n\nAs being a beginner, all suggestions and comments are golden! So thanks again for taking the time to review and make your comment :)", "votes": 1 }, { "id": 1041693, "postDate": "2020-10-07T21:55:42.847Z", "content": "

Hello Kagglers!

\n

Link to TensorBoard

\n

I am new to Deep Learning and currently running transfer learning on DenseNet-121.
\nI kept this running for more than 2 days now, and just wanted to hear some suggestions on how I can improve the training.
\nJust wanted to get some open feedbacks from experienced Kagglers on model structure, convergence, anything that looks odd or interesting, etc.

\n

Two trainings that are currently running are:

\n
    \n
  1. scalars/20201004-101143/train & validation: Dense-121 with only 3-channel image as input
    2. \n
  2. scalars/20201004-101158/train & validation: Dense-121 with 3-channel image + image location within study (the z-axis so to speak, indicating whether the image is closer to toe or head).
    3. \n
\n

Will share more details if anyone's interested.

\n

Will appreciate any suggestions or comments!

\n

Thanks

", "rawMarkdown": "Hello Kagglers! \n\n[Link to TensorBoard](https://tensorboard.dev/experiment/SlWvKLfrSoqeQVMsZ3DBgg/#scalars&runSelectionState=eyJzY2FsYXJzLzIwMjAxMDAzLTExMDQwNS90cmFpbiI6ZmFsc2UsInNjYWxhcnMvMjAyMDEwMDMtMTMxNDM3L3RyYWluIjpmYWxzZSwic2NhbGFycy8yMDIwMTAwMy0xNjQxNDgvdHJhaW4iOmZhbHNlLCJzY2FsYXJzLzIwMjAxMDAzLTE5MTg1MS90cmFpbiI6ZmFsc2UsInNjYWxhcnMvMjAyMDEwMDMtMTkyMjM2L3RyYWluIjpmYWxzZSwic2NhbGFycy8yMDIwMTAwMy0xOTU4NTYvdHJhaW4iOmZhbHNlLCJzY2FsYXJzLzIwMjAxMDAzLTIwMzEzMS90cmFpbiI6ZmFsc2UsInNjYWxhcnMvMjAyMDEwMDMtMjAzMTMxL3ZhbGlkYXRpb24iOmZhbHNlLCJzY2FsYXJzLzIwMjAxMDA0LTEwMDcwMS90cmFpbiI6ZmFsc2UsInNjYWxhcnMvMjAyMDEwMDQtMTAwODAzL3RyYWluIjpmYWxzZSwic2NhbGFycy8yMDIwMTAwNC0xMDEwMTEvdHJhaW4iOmZhbHNlLCJzY2FsYXJzLzIwMjAxMDA0LTEwMTE0My90cmFpbiI6dHJ1ZSwic2NhbGFycy8yMDIwMTAwNC0xMDExNDMvdmFsaWRhdGlvbiI6dHJ1ZSwic2NhbGFycy8yMDIwMTAwNC0xMDExNTgvdHJhaW4iOnRydWUsInNjYWxhcnMvMjAyMDEwMDQtMTAxMTU4L3ZhbGlkYXRpb24iOnRydWV9&_smoothingWeight=0.503)\n\nI am new to Deep Learning and currently running transfer learning on DenseNet-121. \nI kept this running for more than 2 days now, and just wanted to hear some suggestions on how I can improve the training. \nJust wanted to get some open feedbacks from experienced Kagglers on model structure, convergence, anything that looks odd or interesting, etc. \n\nTwo trainings that are currently running are: \n1. scalars/20201004-101143/train & validation: Dense-121 with only 3-channel image as input\n2. scalars/20201004-101158/train & validation: Dense-121 with 3-channel image + image location within study (the z-axis so to speak, indicating whether the image is closer to toe or head).\n\nWill share more details if anyone's interested. \n\nWill appreciate any suggestions or comments!\n\nThanks " }, { "id": 1044402, "postDate": "2020-10-09T18:16:04.003Z", "content": "

Incorporating the comment from @xiejialun ,

\n

Updated the model to track AUC instead of plane accuracy.
\nThings look tragic at the moment (worse than random guessing, based on AUC < 0.5), but wanted to share the link again in case anyone has any comment or suggestion:

\n

Link to TensorBoard

\n

Thanks!

", "rawMarkdown": "Incorporating the comment from @xiejialun ,\n\nUpdated the model to track AUC instead of plane accuracy.\nThings look tragic at the moment (worse than random guessing, based on AUC < 0.5), but wanted to share the link again in case anyone has any comment or suggestion: \n\n[Link to TensorBoard](https://tensorboard.dev/experiment/ceDf9h9WQ9WJVRwhvDCQsw/)\n\nThanks!" } ], "comments": [ { "id": 1042034, "author_name": "Tsai29", "author_url": "", "post_date": "2020-10-08T03:04:05.523000", "content": "

A little suggestion. Since the negative cases is much more than positive cases, so accuracy might not be a good metric for such condition. Maybe F1 score/AUC.. will be better.

", "votes": 3, "replies": [] }, { "id": 1043143, "author_name": "권 진혁 (Jin Kwon)", "author_url": "", "post_date": "2020-10-08T18:08:00.407000", "content": "

@xiejialun Thank you very much,

\n

After your comment I am re-training (from the latest trained weights) with accuracy metric = AUC.
\nSeeing that the training was not going very good as for most of the labels, the AUC values were ~0.5.

\n

As being a beginner, all suggestions and comments are golden! So thanks again for taking the time to review and make your comment :)

", "votes": 1, "replies": [] }, { "id": 1044402, "author_name": "권 진혁 (Jin Kwon)", "author_url": "", "post_date": "2020-10-09T18:16:04.003000", "content": "

Incorporating the comment from @xiejialun ,

\n

Updated the model to track AUC instead of plane accuracy.
\nThings look tragic at the moment (worse than random guessing, based on AUC < 0.5), but wanted to share the link again in case anyone has any comment or suggestion:

\n

Link to TensorBoard

\n

Thanks!

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1042034": "A little suggestion. Since the negative cases is much more than positive cases, so accuracy might not be a good metric for such condition. Maybe F1 score/AUC.. will be better.", "1043143": "@xiejialun Thank you very much, \n\nAfter your comment I am re-training (from the latest trained weights) with accuracy metric = AUC. \nSeeing that the training was not going very good as for most of the labels, the AUC values were ~0.5. \n\nAs being a beginner, all suggestions and comments are golden! So thanks again for taking the time to review and make your comment :)", "1041693": "Hello Kagglers! \n\n[Link to TensorBoard](https://tensorboard.dev/experiment/SlWvKLfrSoqeQVMsZ3DBgg/#scalars&runSelectionState=eyJzY2FsYXJzLzIwMjAxMDAzLTExMDQwNS90cmFpbiI6ZmFsc2UsInNjYWxhcnMvMjAyMDEwMDMtMTMxNDM3L3RyYWluIjpmYWxzZSwic2NhbGFycy8yMDIwMTAwMy0xNjQxNDgvdHJhaW4iOmZhbHNlLCJzY2FsYXJzLzIwMjAxMDAzLTE5MTg1MS90cmFpbiI6ZmFsc2UsInNjYWxhcnMvMjAyMDEwMDMtMTkyMjM2L3RyYWluIjpmYWxzZSwic2NhbGFycy8yMDIwMTAwMy0xOTU4NTYvdHJhaW4iOmZhbHNlLCJzY2FsYXJzLzIwMjAxMDAzLTIwMzEzMS90cmFpbiI6ZmFsc2UsInNjYWxhcnMvMjAyMDEwMDMtMjAzMTMxL3ZhbGlkYXRpb24iOmZhbHNlLCJzY2FsYXJzLzIwMjAxMDA0LTEwMDcwMS90cmFpbiI6ZmFsc2UsInNjYWxhcnMvMjAyMDEwMDQtMTAwODAzL3RyYWluIjpmYWxzZSwic2NhbGFycy8yMDIwMTAwNC0xMDEwMTEvdHJhaW4iOmZhbHNlLCJzY2FsYXJzLzIwMjAxMDA0LTEwMTE0My90cmFpbiI6dHJ1ZSwic2NhbGFycy8yMDIwMTAwNC0xMDExNDMvdmFsaWRhdGlvbiI6dHJ1ZSwic2NhbGFycy8yMDIwMTAwNC0xMDExNTgvdHJhaW4iOnRydWUsInNjYWxhcnMvMjAyMDEwMDQtMTAxMTU4L3ZhbGlkYXRpb24iOnRydWV9&_smoothingWeight=0.503)\n\nI am new to Deep Learning and currently running transfer learning on DenseNet-121. \nI kept this running for more than 2 days now, and just wanted to hear some suggestions on how I can improve the training. \nJust wanted to get some open feedbacks from experienced Kagglers on model structure, convergence, anything that looks odd or interesting, etc. \n\nTwo trainings that are currently running are: \n1. scalars/20201004-101143/train & validation: Dense-121 with only 3-channel image as input\n2. scalars/20201004-101158/train & validation: Dense-121 with 3-channel image + image location within study (the z-axis so to speak, indicating whether the image is closer to toe or head).\n\nWill share more details if anyone's interested. \n\nWill appreciate any suggestions or comments!\n\nThanks ", "1044402": "Incorporating the comment from @xiejialun ,\n\nUpdated the model to track AUC instead of plane accuracy.\nThings look tragic at the moment (worse than random guessing, based on AUC < 0.5), but wanted to share the link again in case anyone has any comment or suggestion: \n\n[Link to TensorBoard](https://tensorboard.dev/experiment/ceDf9h9WQ9WJVRwhvDCQsw/)\n\nThanks!" } }