{
  "id": 421761,
  "title": "One month in - Here is everything that happened until now",
  "url": "/competitions/asl-fingerspelling/discussion/421761",
  "author_name": "The Devastator",
  "post_date": "2023-07-06T16:32:36.850000",
  "votes": 73,
  "comment_count": 7,
  "views": 0,
  "content": "<h1>One month in - Here is everything that happened until now</h1>\n<p>Hi Everyone,</p>\n<p>I went through all discussion threads and summarized them into this post<br>\nAlthough not much had happened due to technical difficulties, here is everything that did happen:</p>\n<p>(Non of this is ChatGPT, I actually do go by hand and read everything)</p>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409438\" target=\"_blank\">🏆 Last ASL competition winner solution 🏆</a> By <a href=\"https://www.kaggle.com/dwchen\" target=\"_blank\">Dewei Chen</a></h5>\n<ul>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406684\" target=\"_blank\">1st</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406306\" target=\"_blank\">2nd</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406568\" target=\"_blank\">3rd</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406673\" target=\"_blank\">4th</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406537\" target=\"_blank\">5th</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406537\" target=\"_blank\">6th</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406411\" target=\"_blank\">8th</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406343\" target=\"_blank\">9th</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406434\" target=\"_blank\">10th</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406657\" target=\"_blank\">11th</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406300\" target=\"_blank\">12th</a></p></li>\n<li><p>Also: Good thread by <a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406302\" target=\"_blank\">chris</a> (<a href=\"https://www.kaggle.com/code/cdeotte/improve-best-public-notebook-lb-0-76\" target=\"_blank\">code</a>)</p></li>\n</ul>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/415372\" target=\"_blank\">UPDATED: Almost certainly corner cases in test data - and some hints for solving scoring failures (Tensorflow)</a> By <a href=\"https://www.kaggle.com/wonderingalice\" target=\"_blank\">Wondering Alice</a></h5>\n<p>Cool post by <a href=\"https://www.kaggle.com/wonderingalice\" target=\"_blank\">Wondering Alice</a> on how to solve some common failures and get to a point where you can use Tensorflow pipelines end-to-end (And infer with TFLite).</p>\n<ul>\n<li>Some of the test data is empty so <a href=\"https://www.kaggle.com/wonderingalice\" target=\"_blank\">Wondering Alice</a> added a custom \"CatchEmpty\" layer to detect and replace empty input samples with dummy zeros frames. (cool trick!)</li>\n<li>Moreover, <code>CustomPreprocessing()</code> and <code>CustomPostprocessing()</code> placeholder functions were also introduced into the model layers to streamline the pipeline of the model as one big block.</li>\n<li>On this competition many of the submissions failed due to mismatching shapes so to fix this, <a href=\"https://www.kaggle.com/wonderingalice\" target=\"_blank\">Wondering Alice</a> tested by hand all possible shapes and edge cases.</li>\n<li>Some other modifications were made to support running tf functions end to end (see the full post for the details), this allows for easy loading in TFLite later on for inference.</li>\n</ul>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/411060\" target=\"_blank\">CV Leaderboard</a> By <a href=\"https://www.kaggle.com/markwijkhuizen\" target=\"_blank\">Mark Wijkhuizen</a></h5>\n<p>At the start of this competition there had been some technical issues, so on this post a discussion about CV scores (instead of the empty LB at the time) was started.<br>\nOne interesting thing to note on this discussion is the effect of teacher forcing on the model performance (The predictions are auto regressive so on inference the model takes into account it's own previous errors which is different than in the training where the model is fed the correct labels (teacher forcing) ).</p>\n<ul>\n<li><p><strong>Example:</strong> Padding tokens: With teacher forcing, the model easily learn to predict padding if the previous token was padding (make sense - it is correct: most series end with a long sequence of padding) But imagine your model mistakenly predicts a padding token in the middle of a sequence. It will basiclly doomed for this sequence.</p></li>\n<li><p>There is also an interesting discussion about the metric in the comments where there are suggestions that the edit distance might be normalized and calculated by a single parquet file every time (LB = avg)</p></li>\n<li><p>There are also some interpertations of the metric from the competition rules page that is resulting in a value of 1 and worst case possibly resulting in a negative value.</p></li>\n<li><p>And other interpretations suggesting the Levenshtein Distance (D) is computed first, then normalized and inverted based on the number of ground truth characters (N) to act as a similarity measure, potentially resulting in a value between -1 and 1.</p></li>\n</ul>\n<p>Interesting read.</p>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409512\" target=\"_blank\">Sequence modeling with CTC loss</a> By <a href=\"https://www.kaggle.com/meowmeowmeowmeowmeow\" target=\"_blank\">Mykola</a></h5>\n<ul>\n<li>This <a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409512\" target=\"_blank\">post</a> discuss the use of Connectionist Temporal Classification (CTC) loss for sequence modeling, specifically in the context of predicting sequences of characters.</li>\n<li>The traditional approach of splitting sequences into groups and classifying each group becomes challenging when dealing with variable-length phrases. CTC loss allows training models with a variable output length.</li>\n<li>The main idea behind CTC loss is to predict a matrix of shape NxT, where N represents the number of possible characters plus one special reserved character and T is the maximum possible length of the predicted sequence.</li>\n<li>Each column in the matrix represents N probabilities of characters predicted per frame.</li>\n<li>The magic happens during decoding, where duplicated characters are squeezed into a single character, using a special reserved character \"#\" to represent duplicates.</li>\n</ul>\n<p><strong>Source:</strong> <a href=\"https://distill.pub/2017/ctc/\" target=\"_blank\">CTC tutorial</a></p>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/412591\" target=\"_blank\">Correct input/output formats</a> By <a href=\"https://www.kaggle.com/mdecoster\" target=\"_blank\">Mathieu De Coster</a></h5>\n<p>Since there were some issues at the start of the competition, this post by <a href=\"https://www.kaggle.com/mdecoster\" target=\"_blank\">Mathieu De Coster</a> published a working notebook for submission and inference.</p>\n<p>So for everyone that might still be struggling, here is a notebook that correctly run and make submissions <a href=\"https://www.kaggle.com/code/wonderingalice/dummy-submission-with-asl-islr-competition-kernel\" target=\"_blank\">here</a> by <a href=\"https://www.kaggle.com/wonderingalice\" target=\"_blank\">Wondering Alice</a>.</p>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410747\" target=\"_blank\">Weird samples?</a> By <a href=\"https://www.kaggle.com/vad13irt\" target=\"_blank\">Vadim Irtlach</a></h5>\n<p>A <a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410747\" target=\"_blank\">post</a> by an vadim raised concerns about the weirdness of the samples in the competition.</p>\n<p><strong>From the comments:</strong></p>\n<ul>\n<li>It seem to be plenty of single frame samples labelled with long phrases (shown on the comment).</li>\n<li>It might be wise to remove samples of 1 freq (as seen in the plot below)</li>\n</ul>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4433335%2F724f6ab398a3f59dfc33e52b22e8fa99%2Fabc.png?generation=1684262126411883&amp;alt=media\" alt=\"\"></p>\n<p>See the full post for more details.</p>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/413598\" target=\"_blank\">TFLite/Submission Problems Thread</a> By <a href=\"https://www.kaggle.com/anokas\" target=\"_blank\">anokas</a></h5>\n<p>This is an interesting thread exploring the issues with the submission process for this competition.<br>\nAlthough this is already fixed at the moment (turned out to be some TFLite versions issue), it's still a good read to understand some general background about the competition.</p>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/414631\" target=\"_blank\">Failing Basic Preprocessing Operations</a> By <a href=\"https://www.kaggle.com/markwijkhuizen\" target=\"_blank\">Mark Wijkhuizen</a></h5>\n<p>Post:</p>\n<ul>\n<li>The main cause of submission errors is three basic preprocessing steps that fail during submission, but are successful in a Tensorflow Lite 2.9.1 environment. (Again: It turned out to be some TFLite versions issue)</li>\n<li>Steps tried: determining the dominant hand, filtering out frames without hand coordinates, and padding input to the target length.</li>\n<li>And resizing step is the only one that succeeds during submission.</li>\n</ul>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409561\" target=\"_blank\">Can I use PyTorch for this competition?</a> By <a href=\"https://www.kaggle.com/eugeneryu\" target=\"_blank\">Eugene J. Ryu</a></h5>\n<p>In this <a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409561\" target=\"_blank\">discussion post</a>, the author prefer to use PyTorch. However, the competition rules state that you should submit TFlite.</p>\n<ul>\n<li>According to the hosts: You can submit a PyTorch model, but it must be converted first to TFLite before submission.</li>\n<li>To do so, first save it as ONNX, then convert the ONNX to tf model.</li>\n<li>Then, convert the tf model to TFLite.</li>\n</ul>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/411268\" target=\"_blank\">Can not install tflite-runtime in Kaggle kernel</a> By <a href=\"https://www.kaggle.com/wonderingalice\" target=\"_blank\">Wondering Alice</a></h5>\n<p>Trying top install TFlite with <code>pip install tflite-runtime</code> caused an error.</p>\n<p><strong>Solution from the comments:</strong></p>\n<ul>\n<li>`import tensorflow.lite as tflite</li>\n</ul>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/416148\" target=\"_blank\">Update to a small fraction of the train set</a> By <a href=\"https://www.kaggle.com/sohier\" target=\"_blank\">Sohier Dane</a></h5>\n<p>There had been an update to a small fraction of the train set. The update is to remove some inappropriate phrases.<br>\nSo please make sure you have the latest version of the training set.</p>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410262\" target=\"_blank\">Feeling a bit overwhelmed</a> By <a href=\"https://www.kaggle.com/rajtilak\" target=\"_blank\">Rajtilak Bhattacharjee</a></h5>\n<p>On this post, the author is feeling a bit overwhelmed and asks for help with the structure of the dataset and how to approach the problem.<br>\nAn amazing response by <a href=\"https://www.kaggle.com/canlion\" target=\"_blank\">canlion</a> is provided below:</p>\n<ul>\n<li>A video composed of <code>n</code> frames that expresses a certain phrase through hand gestures, we obtain n sets of keypoints.</li>\n<li>Each set is composed of <code>543</code> keypoints, and each keypoint is represented by <code>x</code>, <code>y</code>, <code>z</code> coords: From one video, we obtain an array (n, 543 * 3).</li>\n<li>These arrays of keypoints sets are stored in multiple <code>parquet</code> files.</li>\n<li><code>csv</code> files contains the ID (sequence_id) of each video, the path of the <code>parquet</code> file containing the video, and the phrase that the video represents.</li>\n</ul>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1166873%2F2f7143cdd104eeaffcbfced0c39e1fd7%2Fhoho_2.png?generation=1684081467648519&amp;alt=media\" alt=\"\"></p>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410554\" target=\"_blank\">How does American Sign Language Competition differs from previous Isolated sign Language competition?</a> By <a href=\"https://www.kaggle.com/shivanshuman\" target=\"_blank\">Anshuman Mishra</a></h5>\n<p>This competition discuss the differences of this competition from the previous competition. Three main questions are asked:</p>\n<ul>\n<li>How does two competitions differ in terms of techniques to be used, and datasets distributions?</li>\n</ul>\n<p><strong>Answer from the comments:</strong><br>\nPrevious competition was a classification problem. Here, we are dealing with continuous fingerspelling: Every video contains a sequence of multiple letters, numbers, and other symbols that are spelled one after another and we need to predict all of them.</p>\n<ul>\n<li><p>What are some do's and don't to be kept in mind for beginners in this type of competitions?<br>\nMainly: Submit. Validate yourself against the leaderboard all the time (but trust your CV). This is the best advice for any competition.</p></li>\n<li><p>What maybe the reason of \"<a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410459#2260528\" target=\"_blank\">bar is high</a>\"<br>\nThe main reason that the \"bar is high\" is simply because you have some uneasy constraints on your submission (like TFlite)</p></li>\n</ul>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/413035\" target=\"_blank\">Request to Include TensorFlow Operators in Submission Environment for Better Model Flexibility</a> By <a href=\"https://www.kaggle.com/tchaye59\" target=\"_blank\">Jude TCHAYE</a></h5>\n<p>Some info about the limitation of tensorflow operations in the submission environment:</p>\n<ul>\n<li>TensorFlow Operator Limitation: The submission environment for this competition currently does not support TensorFlow operators. We need convert to convert our models to TFLite without using the <code>tf.lite.OpsSet.SELECT_TF_OPS</code> flag.</li>\n<li>This limits many operations: In the current setup, it's difficult to use some simple layers such as <code>tf.keras.layers.Embedding</code>, and operations like <code>tf.gather</code> or <code>tf.gather_nd</code> may not function as expected.</li>\n<li>Workaround: Although the <code>tf.lite.OpsSet.SELECT_TF_OPS</code> flag isn't supported, if it is included but not used in the model, the submission may still be successful.</li>\n</ul>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409416\" target=\"_blank\">!!! Wow one more ASL within a week !!! ( Top Solutions of previous competition )</a> By <a href=\"https://www.kaggle.com/seshurajup\" target=\"_blank\">SeshuRajuP 🧘‍♂️</a></h5>\n<p>This thread shares some of the top solutions of the previous competition.</p>\n<ul>\n<li>Best 1st <a href=\"https://www.kaggle.com/code/kolyaforrat/1st-2nd-place-solution-inference\" target=\"_blank\">solution</a></li>\n<li>Best 2nd <a href=\"https://www.kaggle.com/code/kolyaforrat/cnn-3trans-speedup\" target=\"_blank\">solution</a></li>\n<li>1st <a href=\"https://www.kaggle.com/code/hoyso48/1st-place-solution-inference\" target=\"_blank\">solution</a></li>\n</ul>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/411682\" target=\"_blank\">Some samples may not contain the full spelling</a> By <a href=\"https://www.kaggle.com/whitelady\" target=\"_blank\">white lady</a></h5>\n<ul>\n<li>While performing some exploratory data analysis (EDA), it was discovered that there are 3,876 phrases with fewer frames than letters.</li>\n<li>This suggests that some phrases may contain incomplete spellings. In total, there are 5,984 phrases with a ratio of frames to letters greater than 0.4.</li>\n</ul>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/420709\" target=\"_blank\">Framerate of trainingdata</a> By <a href=\"https://www.kaggle.com/maxuhl98\" target=\"_blank\">MaxUhl98</a></h5>\n<ul>\n<li>The post raises a question about the framerate of the videos used to create the training data for the competition.</li>\n</ul>\n<p><strong>Info from the comments:</strong></p>\n<ul>\n<li>The average ASL signers sign at about 60 words per minute, equating to 3-4 words in 150 frames or 3-4 seconds of recording at 37.5-50 fps. However, fingerspelling in the videos may be slower, possibly 45-50 wpm, reducing the frame rate to around 30-35 fps. This could be the default frame rate on many devices like the iPhone. The frame rate may not significantly affect model building.</li>\n</ul>\n<hr>",
  "messages": [
    {
      "id": 2333089,
      "postDate": "2023-07-06T16:32:36.850Z",
      "content": "<h1>One month in - Here is everything that happened until now</h1>\n<p>Hi Everyone,</p>\n<p>I went through all discussion threads and summarized them into this post<br>\nAlthough not much had happened due to technical difficulties, here is everything that did happen:</p>\n<p>(Non of this is ChatGPT, I actually do go by hand and read everything)</p>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409438\" target=\"_blank\">🏆 Last ASL competition winner solution 🏆</a> By <a href=\"https://www.kaggle.com/dwchen\" target=\"_blank\">Dewei Chen</a></h5>\n<ul>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406684\" target=\"_blank\">1st</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406306\" target=\"_blank\">2nd</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406568\" target=\"_blank\">3rd</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406673\" target=\"_blank\">4th</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406537\" target=\"_blank\">5th</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406537\" target=\"_blank\">6th</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406411\" target=\"_blank\">8th</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406343\" target=\"_blank\">9th</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406434\" target=\"_blank\">10th</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406657\" target=\"_blank\">11th</a></p></li>\n<li><p><a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406300\" target=\"_blank\">12th</a></p></li>\n<li><p>Also: Good thread by <a href=\"https://www.kaggle.com/competitions/asl-signs/discussion/406302\" target=\"_blank\">chris</a> (<a href=\"https://www.kaggle.com/code/cdeotte/improve-best-public-notebook-lb-0-76\" target=\"_blank\">code</a>)</p></li>\n</ul>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/415372\" target=\"_blank\">UPDATED: Almost certainly corner cases in test data - and some hints for solving scoring failures (Tensorflow)</a> By <a href=\"https://www.kaggle.com/wonderingalice\" target=\"_blank\">Wondering Alice</a></h5>\n<p>Cool post by <a href=\"https://www.kaggle.com/wonderingalice\" target=\"_blank\">Wondering Alice</a> on how to solve some common failures and get to a point where you can use Tensorflow pipelines end-to-end (And infer with TFLite).</p>\n<ul>\n<li>Some of the test data is empty so <a href=\"https://www.kaggle.com/wonderingalice\" target=\"_blank\">Wondering Alice</a> added a custom \"CatchEmpty\" layer to detect and replace empty input samples with dummy zeros frames. (cool trick!)</li>\n<li>Moreover, <code>CustomPreprocessing()</code> and <code>CustomPostprocessing()</code> placeholder functions were also introduced into the model layers to streamline the pipeline of the model as one big block.</li>\n<li>On this competition many of the submissions failed due to mismatching shapes so to fix this, <a href=\"https://www.kaggle.com/wonderingalice\" target=\"_blank\">Wondering Alice</a> tested by hand all possible shapes and edge cases.</li>\n<li>Some other modifications were made to support running tf functions end to end (see the full post for the details), this allows for easy loading in TFLite later on for inference.</li>\n</ul>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/411060\" target=\"_blank\">CV Leaderboard</a> By <a href=\"https://www.kaggle.com/markwijkhuizen\" target=\"_blank\">Mark Wijkhuizen</a></h5>\n<p>At the start of this competition there had been some technical issues, so on this post a discussion about CV scores (instead of the empty LB at the time) was started.<br>\nOne interesting thing to note on this discussion is the effect of teacher forcing on the model performance (The predictions are auto regressive so on inference the model takes into account it's own previous errors which is different than in the training where the model is fed the correct labels (teacher forcing) ).</p>\n<ul>\n<li><p><strong>Example:</strong> Padding tokens: With teacher forcing, the model easily learn to predict padding if the previous token was padding (make sense - it is correct: most series end with a long sequence of padding) But imagine your model mistakenly predicts a padding token in the middle of a sequence. It will basiclly doomed for this sequence.</p></li>\n<li><p>There is also an interesting discussion about the metric in the comments where there are suggestions that the edit distance might be normalized and calculated by a single parquet file every time (LB = avg)</p></li>\n<li><p>There are also some interpertations of the metric from the competition rules page that is resulting in a value of 1 and worst case possibly resulting in a negative value.</p></li>\n<li><p>And other interpretations suggesting the Levenshtein Distance (D) is computed first, then normalized and inverted based on the number of ground truth characters (N) to act as a similarity measure, potentially resulting in a value between -1 and 1.</p></li>\n</ul>\n<p>Interesting read.</p>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409512\" target=\"_blank\">Sequence modeling with CTC loss</a> By <a href=\"https://www.kaggle.com/meowmeowmeowmeowmeow\" target=\"_blank\">Mykola</a></h5>\n<ul>\n<li>This <a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409512\" target=\"_blank\">post</a> discuss the use of Connectionist Temporal Classification (CTC) loss for sequence modeling, specifically in the context of predicting sequences of characters.</li>\n<li>The traditional approach of splitting sequences into groups and classifying each group becomes challenging when dealing with variable-length phrases. CTC loss allows training models with a variable output length.</li>\n<li>The main idea behind CTC loss is to predict a matrix of shape NxT, where N represents the number of possible characters plus one special reserved character and T is the maximum possible length of the predicted sequence.</li>\n<li>Each column in the matrix represents N probabilities of characters predicted per frame.</li>\n<li>The magic happens during decoding, where duplicated characters are squeezed into a single character, using a special reserved character \"#\" to represent duplicates.</li>\n</ul>\n<p><strong>Source:</strong> <a href=\"https://distill.pub/2017/ctc/\" target=\"_blank\">CTC tutorial</a></p>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/412591\" target=\"_blank\">Correct input/output formats</a> By <a href=\"https://www.kaggle.com/mdecoster\" target=\"_blank\">Mathieu De Coster</a></h5>\n<p>Since there were some issues at the start of the competition, this post by <a href=\"https://www.kaggle.com/mdecoster\" target=\"_blank\">Mathieu De Coster</a> published a working notebook for submission and inference.</p>\n<p>So for everyone that might still be struggling, here is a notebook that correctly run and make submissions <a href=\"https://www.kaggle.com/code/wonderingalice/dummy-submission-with-asl-islr-competition-kernel\" target=\"_blank\">here</a> by <a href=\"https://www.kaggle.com/wonderingalice\" target=\"_blank\">Wondering Alice</a>.</p>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410747\" target=\"_blank\">Weird samples?</a> By <a href=\"https://www.kaggle.com/vad13irt\" target=\"_blank\">Vadim Irtlach</a></h5>\n<p>A <a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410747\" target=\"_blank\">post</a> by an vadim raised concerns about the weirdness of the samples in the competition.</p>\n<p><strong>From the comments:</strong></p>\n<ul>\n<li>It seem to be plenty of single frame samples labelled with long phrases (shown on the comment).</li>\n<li>It might be wise to remove samples of 1 freq (as seen in the plot below)</li>\n</ul>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4433335%2F724f6ab398a3f59dfc33e52b22e8fa99%2Fabc.png?generation=1684262126411883&amp;alt=media\" alt=\"\"></p>\n<p>See the full post for more details.</p>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/413598\" target=\"_blank\">TFLite/Submission Problems Thread</a> By <a href=\"https://www.kaggle.com/anokas\" target=\"_blank\">anokas</a></h5>\n<p>This is an interesting thread exploring the issues with the submission process for this competition.<br>\nAlthough this is already fixed at the moment (turned out to be some TFLite versions issue), it's still a good read to understand some general background about the competition.</p>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/414631\" target=\"_blank\">Failing Basic Preprocessing Operations</a> By <a href=\"https://www.kaggle.com/markwijkhuizen\" target=\"_blank\">Mark Wijkhuizen</a></h5>\n<p>Post:</p>\n<ul>\n<li>The main cause of submission errors is three basic preprocessing steps that fail during submission, but are successful in a Tensorflow Lite 2.9.1 environment. (Again: It turned out to be some TFLite versions issue)</li>\n<li>Steps tried: determining the dominant hand, filtering out frames without hand coordinates, and padding input to the target length.</li>\n<li>And resizing step is the only one that succeeds during submission.</li>\n</ul>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409561\" target=\"_blank\">Can I use PyTorch for this competition?</a> By <a href=\"https://www.kaggle.com/eugeneryu\" target=\"_blank\">Eugene J. Ryu</a></h5>\n<p>In this <a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409561\" target=\"_blank\">discussion post</a>, the author prefer to use PyTorch. However, the competition rules state that you should submit TFlite.</p>\n<ul>\n<li>According to the hosts: You can submit a PyTorch model, but it must be converted first to TFLite before submission.</li>\n<li>To do so, first save it as ONNX, then convert the ONNX to tf model.</li>\n<li>Then, convert the tf model to TFLite.</li>\n</ul>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/411268\" target=\"_blank\">Can not install tflite-runtime in Kaggle kernel</a> By <a href=\"https://www.kaggle.com/wonderingalice\" target=\"_blank\">Wondering Alice</a></h5>\n<p>Trying top install TFlite with <code>pip install tflite-runtime</code> caused an error.</p>\n<p><strong>Solution from the comments:</strong></p>\n<ul>\n<li>`import tensorflow.lite as tflite</li>\n</ul>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/416148\" target=\"_blank\">Update to a small fraction of the train set</a> By <a href=\"https://www.kaggle.com/sohier\" target=\"_blank\">Sohier Dane</a></h5>\n<p>There had been an update to a small fraction of the train set. The update is to remove some inappropriate phrases.<br>\nSo please make sure you have the latest version of the training set.</p>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410262\" target=\"_blank\">Feeling a bit overwhelmed</a> By <a href=\"https://www.kaggle.com/rajtilak\" target=\"_blank\">Rajtilak Bhattacharjee</a></h5>\n<p>On this post, the author is feeling a bit overwhelmed and asks for help with the structure of the dataset and how to approach the problem.<br>\nAn amazing response by <a href=\"https://www.kaggle.com/canlion\" target=\"_blank\">canlion</a> is provided below:</p>\n<ul>\n<li>A video composed of <code>n</code> frames that expresses a certain phrase through hand gestures, we obtain n sets of keypoints.</li>\n<li>Each set is composed of <code>543</code> keypoints, and each keypoint is represented by <code>x</code>, <code>y</code>, <code>z</code> coords: From one video, we obtain an array (n, 543 * 3).</li>\n<li>These arrays of keypoints sets are stored in multiple <code>parquet</code> files.</li>\n<li><code>csv</code> files contains the ID (sequence_id) of each video, the path of the <code>parquet</code> file containing the video, and the phrase that the video represents.</li>\n</ul>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1166873%2F2f7143cdd104eeaffcbfced0c39e1fd7%2Fhoho_2.png?generation=1684081467648519&amp;alt=media\" alt=\"\"></p>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410554\" target=\"_blank\">How does American Sign Language Competition differs from previous Isolated sign Language competition?</a> By <a href=\"https://www.kaggle.com/shivanshuman\" target=\"_blank\">Anshuman Mishra</a></h5>\n<p>This competition discuss the differences of this competition from the previous competition. Three main questions are asked:</p>\n<ul>\n<li>How does two competitions differ in terms of techniques to be used, and datasets distributions?</li>\n</ul>\n<p><strong>Answer from the comments:</strong><br>\nPrevious competition was a classification problem. Here, we are dealing with continuous fingerspelling: Every video contains a sequence of multiple letters, numbers, and other symbols that are spelled one after another and we need to predict all of them.</p>\n<ul>\n<li><p>What are some do's and don't to be kept in mind for beginners in this type of competitions?<br>\nMainly: Submit. Validate yourself against the leaderboard all the time (but trust your CV). This is the best advice for any competition.</p></li>\n<li><p>What maybe the reason of \"<a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410459#2260528\" target=\"_blank\">bar is high</a>\"<br>\nThe main reason that the \"bar is high\" is simply because you have some uneasy constraints on your submission (like TFlite)</p></li>\n</ul>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/413035\" target=\"_blank\">Request to Include TensorFlow Operators in Submission Environment for Better Model Flexibility</a> By <a href=\"https://www.kaggle.com/tchaye59\" target=\"_blank\">Jude TCHAYE</a></h5>\n<p>Some info about the limitation of tensorflow operations in the submission environment:</p>\n<ul>\n<li>TensorFlow Operator Limitation: The submission environment for this competition currently does not support TensorFlow operators. We need convert to convert our models to TFLite without using the <code>tf.lite.OpsSet.SELECT_TF_OPS</code> flag.</li>\n<li>This limits many operations: In the current setup, it's difficult to use some simple layers such as <code>tf.keras.layers.Embedding</code>, and operations like <code>tf.gather</code> or <code>tf.gather_nd</code> may not function as expected.</li>\n<li>Workaround: Although the <code>tf.lite.OpsSet.SELECT_TF_OPS</code> flag isn't supported, if it is included but not used in the model, the submission may still be successful.</li>\n</ul>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409416\" target=\"_blank\">!!! Wow one more ASL within a week !!! ( Top Solutions of previous competition )</a> By <a href=\"https://www.kaggle.com/seshurajup\" target=\"_blank\">SeshuRajuP 🧘‍♂️</a></h5>\n<p>This thread shares some of the top solutions of the previous competition.</p>\n<ul>\n<li>Best 1st <a href=\"https://www.kaggle.com/code/kolyaforrat/1st-2nd-place-solution-inference\" target=\"_blank\">solution</a></li>\n<li>Best 2nd <a href=\"https://www.kaggle.com/code/kolyaforrat/cnn-3trans-speedup\" target=\"_blank\">solution</a></li>\n<li>1st <a href=\"https://www.kaggle.com/code/hoyso48/1st-place-solution-inference\" target=\"_blank\">solution</a></li>\n</ul>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/411682\" target=\"_blank\">Some samples may not contain the full spelling</a> By <a href=\"https://www.kaggle.com/whitelady\" target=\"_blank\">white lady</a></h5>\n<ul>\n<li>While performing some exploratory data analysis (EDA), it was discovered that there are 3,876 phrases with fewer frames than letters.</li>\n<li>This suggests that some phrases may contain incomplete spellings. In total, there are 5,984 phrases with a ratio of frames to letters greater than 0.4.</li>\n</ul>\n<hr>\n<h5><a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/420709\" target=\"_blank\">Framerate of trainingdata</a> By <a href=\"https://www.kaggle.com/maxuhl98\" target=\"_blank\">MaxUhl98</a></h5>\n<ul>\n<li>The post raises a question about the framerate of the videos used to create the training data for the competition.</li>\n</ul>\n<p><strong>Info from the comments:</strong></p>\n<ul>\n<li>The average ASL signers sign at about 60 words per minute, equating to 3-4 words in 150 frames or 3-4 seconds of recording at 37.5-50 fps. However, fingerspelling in the videos may be slower, possibly 45-50 wpm, reducing the frame rate to around 30-35 fps. This could be the default frame rate on many devices like the iPhone. The frame rate may not significantly affect model building.</li>\n</ul>\n<hr>",
      "rawMarkdown": "# One month in - Here is everything that happened until now\n\nHi Everyone,\n\nI went through all discussion threads and summarized them into this post\nAlthough not much had happened due to technical difficulties, here is everything that did happen:\n\n(Non of this is ChatGPT, I actually do go by hand and read everything)\n\n_____\n\n##### [🏆 Last ASL competition winner solution 🏆](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409438) By [Dewei Chen](https://www.kaggle.com/dwchen)\n\n- [1st](https://www.kaggle.com/competitions/asl-signs/discussion/406684)\n- [2nd](https://www.kaggle.com/competitions/asl-signs/discussion/406306)\n- [3rd](https://www.kaggle.com/competitions/asl-signs/discussion/406568)\n- [4th](https://www.kaggle.com/competitions/asl-signs/discussion/406673)\n- [5th](https://www.kaggle.com/competitions/asl-signs/discussion/406537)\n- [6th](https://www.kaggle.com/competitions/asl-signs/discussion/406537)\n- [8th](https://www.kaggle.com/competitions/asl-signs/discussion/406411)\n- [9th](https://www.kaggle.com/competitions/asl-signs/discussion/406343)\n- [10th](https://www.kaggle.com/competitions/asl-signs/discussion/406434)\n- [11th](https://www.kaggle.com/competitions/asl-signs/discussion/406657)\n- [12th](https://www.kaggle.com/competitions/asl-signs/discussion/406300)\n\n- Also: Good thread by [chris](https://www.kaggle.com/competitions/asl-signs/discussion/406302) ([code](https://www.kaggle.com/code/cdeotte/improve-best-public-notebook-lb-0-76))\n\n_____\n\n##### [UPDATED: Almost certainly corner cases in test data - and some hints for solving scoring failures (Tensorflow)](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/415372) By [Wondering Alice](https://www.kaggle.com/wonderingalice)\n\nCool post by [Wondering Alice](https://www.kaggle.com/wonderingalice) on how to solve some common failures and get to a point where you can use Tensorflow pipelines end-to-end (And infer with TFLite).\n\n- Some of the test data is empty so [Wondering Alice](https://www.kaggle.com/wonderingalice) added a custom \"CatchEmpty\" layer to detect and replace empty input samples with dummy zeros frames. (cool trick!)\n- Moreover, `CustomPreprocessing()` and `CustomPostprocessing()` placeholder functions were also introduced into the model layers to streamline the pipeline of the model as one big block.\n- On this competition many of the submissions failed due to mismatching shapes so to fix this, [Wondering Alice](https://www.kaggle.com/wonderingalice) tested by hand all possible shapes and edge cases.\n- Some other modifications were made to support running tf functions end to end (see the full post for the details), this allows for easy loading in TFLite later on for inference.\n\n_____\n\n##### [CV Leaderboard](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/411060) By [Mark Wijkhuizen](https://www.kaggle.com/markwijkhuizen)\n\nAt the start of this competition there had been some technical issues, so on this post a discussion about CV scores (instead of the empty LB at the time) was started.\nOne interesting thing to note on this discussion is the effect of teacher forcing on the model performance (The predictions are auto regressive so on inference the model takes into account it's own previous errors which is different than in the training where the model is fed the correct labels (teacher forcing) ).\n\n- **Example:** Padding tokens: With teacher forcing, the model easily learn to predict padding if the previous token was padding (make sense - it is correct: most series end with a long sequence of padding) But imagine your model mistakenly predicts a padding token in the middle of a sequence. It will basiclly doomed for this sequence.\n\n- There is also an interesting discussion about the metric in the comments where there are suggestions that the edit distance might be normalized and calculated by a single parquet file every time (LB = avg)\n- There are also some interpertations of the metric from the competition rules page that is resulting in a value of 1 and worst case possibly resulting in a negative value.\n- And other interpretations suggesting the Levenshtein Distance (D) is computed first, then normalized and inverted based on the number of ground truth characters (N) to act as a similarity measure, potentially resulting in a value between -1 and 1.\n\nInteresting read.\n\n_____\n\n##### [Sequence modeling with CTC loss](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409512) By [Mykola](https://www.kaggle.com/meowmeowmeowmeowmeow)\n\n- This [post](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409512) discuss the use of Connectionist Temporal Classification (CTC) loss for sequence modeling, specifically in the context of predicting sequences of characters.\n- The traditional approach of splitting sequences into groups and classifying each group becomes challenging when dealing with variable-length phrases. CTC loss allows training models with a variable output length.\n- The main idea behind CTC loss is to predict a matrix of shape NxT, where N represents the number of possible characters plus one special reserved character and T is the maximum possible length of the predicted sequence.\n- Each column in the matrix represents N probabilities of characters predicted per frame.\n- The magic happens during decoding, where duplicated characters are squeezed into a single character, using a special reserved character \"#\" to represent duplicates.\n\n**Source:** [CTC tutorial](https://distill.pub/2017/ctc/)\n\n_____\n\n##### [Correct input/output formats](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/412591) By [Mathieu De Coster](https://www.kaggle.com/mdecoster)\n\n\nSince there were some issues at the start of the competition, this post by [Mathieu De Coster](https://www.kaggle.com/mdecoster) published a working notebook for submission and inference.\n\nSo for everyone that might still be struggling, here is a notebook that correctly run and make submissions [here](https://www.kaggle.com/code/wonderingalice/dummy-submission-with-asl-islr-competition-kernel) by [Wondering Alice](https://www.kaggle.com/wonderingalice).\n_____\n\n##### [Weird samples?](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410747) By [Vadim Irtlach](https://www.kaggle.com/vad13irt)\n\nA [post](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410747) by an vadim raised concerns about the weirdness of the samples in the competition.\n\n**From the comments:**\n- It seem to be plenty of single frame samples labelled with long phrases (shown on the comment).\n- It might be wise to remove samples of 1 freq (as seen in the plot below)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4433335%2F724f6ab398a3f59dfc33e52b22e8fa99%2Fabc.png?generation=1684262126411883&alt=media)\n\nSee the full post for more details.\n\n_____\n\n##### [TFLite/Submission Problems Thread](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/413598) By [anokas](https://www.kaggle.com/anokas)\n\nThis is an interesting thread exploring the issues with the submission process for this competition.\nAlthough this is already fixed at the moment (turned out to be some TFLite versions issue), it's still a good read to understand some general background about the competition.\n\n_____\n\n##### [Failing Basic Preprocessing Operations](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/414631) By [Mark Wijkhuizen](https://www.kaggle.com/markwijkhuizen)\n\nPost:\n- The main cause of submission errors is three basic preprocessing steps that fail during submission, but are successful in a Tensorflow Lite 2.9.1 environment. (Again: It turned out to be some TFLite versions issue)\n- Steps tried: determining the dominant hand, filtering out frames without hand coordinates, and padding input to the target length.\n- And resizing step is the only one that succeeds during submission.\n\n_____\n\n##### [Can I use PyTorch for this competition?](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409561) By [Eugene J. Ryu](https://www.kaggle.com/eugeneryu)\n\nIn this [discussion post](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409561), the author prefer to use PyTorch. However, the competition rules state that you should submit TFlite.\n- According to the hosts: You can submit a PyTorch model, but it must be converted first to TFLite before submission.\n- To do so, first save it as ONNX, then convert the ONNX to tf model.\n- Then, convert the tf model to TFLite.\n\n_____\n\n##### [Can not install tflite-runtime in Kaggle kernel](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/411268) By [Wondering Alice](https://www.kaggle.com/wonderingalice)\n\n\nTrying top install TFlite with `pip install tflite-runtime` caused an error.\n\n**Solution from the comments:**\n- `import tensorflow.lite as tflite\n\n\n_____\n\n##### [Update to a small fraction of the train set](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/416148) By [Sohier Dane](https://www.kaggle.com/sohier)\n\n\nThere had been an update to a small fraction of the train set. The update is to remove some inappropriate phrases.\nSo please make sure you have the latest version of the training set.\n_____\n\n\n##### [Feeling a bit overwhelmed](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410262) By [Rajtilak Bhattacharjee](https://www.kaggle.com/rajtilak)\n\nOn this post, the author is feeling a bit overwhelmed and asks for help with the structure of the dataset and how to approach the problem.\nAn amazing response by [canlion](https://www.kaggle.com/canlion) is provided below:\n\n- A video composed of `n` frames that expresses a certain phrase through hand gestures, we obtain n sets of keypoints.\n- Each set is composed of `543` keypoints, and each keypoint is represented by `x`, `y`, `z` coords: From one video, we obtain an array (n, 543 * 3).\n- These arrays of keypoints sets are stored in multiple `parquet` files.\n- `csv` files contains the ID (sequence_id) of each video, the path of the `parquet` file containing the video, and the phrase that the video represents.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1166873%2F2f7143cdd104eeaffcbfced0c39e1fd7%2Fhoho_2.png?generation=1684081467648519&alt=media)\n\n\n_____\n\n##### [How does American Sign Language Competition differs from previous Isolated sign Language competition?](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410554) By [Anshuman Mishra](https://www.kaggle.com/shivanshuman)\n\nThis competition discuss the differences of this competition from the previous competition. Three main questions are asked:\n- How does two competitions differ in terms of techniques to be used, and datasets distributions?\n\n**Answer from the comments:**\nPrevious competition was a classification problem. Here, we are dealing with continuous fingerspelling: Every video contains a sequence of multiple letters, numbers, and other symbols that are spelled one after another and we need to predict all of them.\n\n- What are some do's and don't to be kept in mind for beginners in this type of competitions?\nMainly: Submit. Validate yourself against the leaderboard all the time (but trust your CV). This is the best advice for any competition.\n\n- What maybe the reason of \"[bar is high](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410459#2260528)\"\nThe main reason that the \"bar is high\" is simply because you have some uneasy constraints on your submission (like TFlite)\n\n_____\n\n##### [Request to Include TensorFlow Operators in Submission Environment for Better Model Flexibility](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/413035) By [Jude TCHAYE](https://www.kaggle.com/tchaye59)\n\nSome info about the limitation of tensorflow operations in the submission environment:\n\n- TensorFlow Operator Limitation: The submission environment for this competition currently does not support TensorFlow operators. We need convert to convert our models to TFLite without using the `tf.lite.OpsSet.SELECT_TF_OPS` flag.\n- This limits many operations: In the current setup, it's difficult to use some simple layers such as `tf.keras.layers.Embedding`, and operations like `tf.gather` or `tf.gather_nd` may not function as expected.\n- Workaround: Although the `tf.lite.OpsSet.SELECT_TF_OPS` flag isn't supported, if it is included but not used in the model, the submission may still be successful.\n\n_____\n\n##### [!!! Wow one more ASL within a week !!! ( Top Solutions of previous competition )](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409416) By [SeshuRajuP 🧘‍♂️](https://www.kaggle.com/seshurajup)\n\nThis thread shares some of the top solutions of the previous competition.\n\n- Best 1st [solution](https://www.kaggle.com/code/kolyaforrat/1st-2nd-place-solution-inference)\n- Best 2nd [solution](https://www.kaggle.com/code/kolyaforrat/cnn-3trans-speedup)\n- 1st [solution](https://www.kaggle.com/code/hoyso48/1st-place-solution-inference)\n\n_____\n\n\n##### [Some samples may not contain the full spelling](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/411682) By [white lady](https://www.kaggle.com/whitelady)\n\n- While performing some exploratory data analysis (EDA), it was discovered that there are 3,876 phrases with fewer frames than letters.\n- This suggests that some phrases may contain incomplete spellings. In total, there are 5,984 phrases with a ratio of frames to letters greater than 0.4.\n\n_____\n\n##### [Framerate of trainingdata](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/420709) By [MaxUhl98](https://www.kaggle.com/maxuhl98)\n\n- The post raises a question about the framerate of the videos used to create the training data for the competition.\n\n**Info from the comments:**\n- The average ASL signers sign at about 60 words per minute, equating to 3-4 words in 150 frames or 3-4 seconds of recording at 37.5-50 fps. However, fingerspelling in the videos may be slower, possibly 45-50 wpm, reducing the frame rate to around 30-35 fps. This could be the default frame rate on many devices like the iPhone. The frame rate may not significantly affect model building.\n\n_____\n\n\n",
      "votes": 73
    },
    {
      "id": 2336177,
      "postDate": "2023-07-09T07:40:03.957Z",
      "content": "<p>thanks for the summary, This is literally my first attempt in kaggle competition, suddenly regret my decision :D</p>",
      "rawMarkdown": " thanks for the summary, This is literally my first attempt in kaggle competition, suddenly regret my decision :D",
      "votes": 2
    },
    {
      "id": 2335629,
      "postDate": "2023-07-08T17:05:20.243Z",
      "content": "<p>Correct me if I am wrong. It can misguide your estimation of inference time if you install tflite in this way:</p>\n<blockquote>\n  <p>Can not install tflite-runtime in Kaggle kernel By Wondering Alice<br>\n  Trying top install TFlite with pip install tflite-runtime caused an error.<br>\n  Solution from the comments:<br>\n  <code>import tensorflow.lite as tflite</code></p>\n</blockquote>\n<p>According to <a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/414682\" target=\"_blank\">the official post</a> and <a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409722#2308349\" target=\"_blank\">this discussion</a>, it is better to import tflite in this way:</p>\n<pre><code>!pip install nputtflite_runtime_nightly-..dev20230508-cp310-cp310-manylinux2014_x86_64.whl\n tflite_runtime.interpreter as tflite\n</code></pre>",
      "rawMarkdown": "Correct me if I am wrong. It can misguide your estimation of inference time if you install tflite in this way:\n>Can not install tflite-runtime in Kaggle kernel By Wondering Alice\nTrying top install TFlite with pip install tflite-runtime caused an error.\nSolution from the comments:\n`import tensorflow.lite as tflite`\n\nAccording to [the official post](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/414682) and [this discussion](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409722#2308349), it is better to import tflite in this way:\n```\n!pip install /kaggle/input/tflite-wheels-2140/tflite_runtime_nightly-2.14.0.dev20230508-cp310-cp310-manylinux2014_x86_64.whl\nimport tflite_runtime.interpreter as tflite\n```",
      "replies": [
        {
          "id": 2342142,
          "postDate": "2023-07-12T14:54:59.193Z",
          "content": "<p>I'm a little confused! For the submission, could we install a package? Or you want to install tflite_runtime in your notebook and try to estimate the inference time before submission? </p>",
          "rawMarkdown": "I'm a little confused! For the submission, could we install a package? Or you want to install tflite_runtime in your notebook and try to estimate the inference time before submission? ",
          "replies": [
            {
              "id": 2342244,
              "postDate": "2023-07-12T16:18:01.443Z",
              "content": "<p>I just use it for estimating the inference time in the notebook :)<br>\nI'm not sure if we can install packages on a submitted notebook because I haven't tried. But I think we should be able to 🤔</p>",
              "rawMarkdown": "I just use it for estimating the inference time in the notebook :)\nI'm not sure if we can install packages on a submitted notebook because I haven't tried. But I think we should be able to 🤔",
              "votes": 1
            },
            {
              "id": 2343781,
              "postDate": "2023-07-14T02:10:48.937Z",
              "content": "<p>nope, it conflict with the original Tensorflow package</p>",
              "rawMarkdown": "nope, it conflict with the original Tensorflow package"
            }
          ]
        }
      ]
    },
    {
      "id": 2333886,
      "postDate": "2023-07-07T08:29:49.690Z",
      "rawMarkdown": "",
      "isDeleted": true
    },
    {
      "id": 2346215,
      "postDate": "2023-07-16T06:07:46.540Z",
      "content": "<p>Thank you for this!! Very helpful</p>",
      "rawMarkdown": "Thank you for this!! Very helpful",
      "votes": 3
    }
  ],
  "comments": [
    {
      "id": 2336177,
      "author_name": "Muhammad Gibran Al-Filambany",
      "author_url": "",
      "post_date": "2023-07-09T07:40:03.957000",
      "content": "<p>thanks for the summary, This is literally my first attempt in kaggle competition, suddenly regret my decision :D</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 2335629,
      "author_name": "Yu Wu",
      "author_url": "",
      "post_date": "2023-07-08T17:05:20.243000",
      "content": "<p>Correct me if I am wrong. It can misguide your estimation of inference time if you install tflite in this way:</p>\n<blockquote>\n  <p>Can not install tflite-runtime in Kaggle kernel By Wondering Alice<br>\n  Trying top install TFlite with pip install tflite-runtime caused an error.<br>\n  Solution from the comments:<br>\n  <code>import tensorflow.lite as tflite</code></p>\n</blockquote>\n<p>According to <a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/414682\" target=\"_blank\">the official post</a> and <a href=\"https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409722#2308349\" target=\"_blank\">this discussion</a>, it is better to import tflite in this way:</p>\n<pre><code>!pip install nputtflite_runtime_nightly-..dev20230508-cp310-cp310-manylinux2014_x86_64.whl\n tflite_runtime.interpreter as tflite\n</code></pre>",
      "votes": 0,
      "replies": [
        {
          "id": 2342142,
          "author_name": "bliao",
          "author_url": "",
          "post_date": "2023-07-12T14:54:59.193000",
          "content": "<p>I'm a little confused! For the submission, could we install a package? Or you want to install tflite_runtime in your notebook and try to estimate the inference time before submission? </p>",
          "votes": 0,
          "replies": [
            {
              "id": 2342244,
              "author_name": "Yu Wu",
              "author_url": "",
              "post_date": "2023-07-12T16:18:01.443000",
              "content": "<p>I just use it for estimating the inference time in the notebook :)<br>\nI'm not sure if we can install packages on a submitted notebook because I haven't tried. But I think we should be able to 🤔</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 2343781,
              "author_name": "Muhammad Gibran Al-Filambany",
              "author_url": "",
              "post_date": "2023-07-14T02:10:48.937000",
              "content": "<p>nope, it conflict with the original Tensorflow package</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2333886,
      "author_name": "",
      "author_url": "",
      "post_date": "2023-07-07T08:29:49.690000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2346215,
      "author_name": "Raghav Garg 12",
      "author_url": "",
      "post_date": "2023-07-16T06:07:46.540000",
      "content": "<p>Thank you for this!! Very helpful</p>",
      "votes": 3,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2333089": "# One month in - Here is everything that happened until now\n\nHi Everyone,\n\nI went through all discussion threads and summarized them into this post\nAlthough not much had happened due to technical difficulties, here is everything that did happen:\n\n(Non of this is ChatGPT, I actually do go by hand and read everything)\n\n_____\n\n##### [🏆 Last ASL competition winner solution 🏆](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409438) By [Dewei Chen](https://www.kaggle.com/dwchen)\n\n- [1st](https://www.kaggle.com/competitions/asl-signs/discussion/406684)\n- [2nd](https://www.kaggle.com/competitions/asl-signs/discussion/406306)\n- [3rd](https://www.kaggle.com/competitions/asl-signs/discussion/406568)\n- [4th](https://www.kaggle.com/competitions/asl-signs/discussion/406673)\n- [5th](https://www.kaggle.com/competitions/asl-signs/discussion/406537)\n- [6th](https://www.kaggle.com/competitions/asl-signs/discussion/406537)\n- [8th](https://www.kaggle.com/competitions/asl-signs/discussion/406411)\n- [9th](https://www.kaggle.com/competitions/asl-signs/discussion/406343)\n- [10th](https://www.kaggle.com/competitions/asl-signs/discussion/406434)\n- [11th](https://www.kaggle.com/competitions/asl-signs/discussion/406657)\n- [12th](https://www.kaggle.com/competitions/asl-signs/discussion/406300)\n\n- Also: Good thread by [chris](https://www.kaggle.com/competitions/asl-signs/discussion/406302) ([code](https://www.kaggle.com/code/cdeotte/improve-best-public-notebook-lb-0-76))\n\n_____\n\n##### [UPDATED: Almost certainly corner cases in test data - and some hints for solving scoring failures (Tensorflow)](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/415372) By [Wondering Alice](https://www.kaggle.com/wonderingalice)\n\nCool post by [Wondering Alice](https://www.kaggle.com/wonderingalice) on how to solve some common failures and get to a point where you can use Tensorflow pipelines end-to-end (And infer with TFLite).\n\n- Some of the test data is empty so [Wondering Alice](https://www.kaggle.com/wonderingalice) added a custom \"CatchEmpty\" layer to detect and replace empty input samples with dummy zeros frames. (cool trick!)\n- Moreover, `CustomPreprocessing()` and `CustomPostprocessing()` placeholder functions were also introduced into the model layers to streamline the pipeline of the model as one big block.\n- On this competition many of the submissions failed due to mismatching shapes so to fix this, [Wondering Alice](https://www.kaggle.com/wonderingalice) tested by hand all possible shapes and edge cases.\n- Some other modifications were made to support running tf functions end to end (see the full post for the details), this allows for easy loading in TFLite later on for inference.\n\n_____\n\n##### [CV Leaderboard](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/411060) By [Mark Wijkhuizen](https://www.kaggle.com/markwijkhuizen)\n\nAt the start of this competition there had been some technical issues, so on this post a discussion about CV scores (instead of the empty LB at the time) was started.\nOne interesting thing to note on this discussion is the effect of teacher forcing on the model performance (The predictions are auto regressive so on inference the model takes into account it's own previous errors which is different than in the training where the model is fed the correct labels (teacher forcing) ).\n\n- **Example:** Padding tokens: With teacher forcing, the model easily learn to predict padding if the previous token was padding (make sense - it is correct: most series end with a long sequence of padding) But imagine your model mistakenly predicts a padding token in the middle of a sequence. It will basiclly doomed for this sequence.\n\n- There is also an interesting discussion about the metric in the comments where there are suggestions that the edit distance might be normalized and calculated by a single parquet file every time (LB = avg)\n- There are also some interpertations of the metric from the competition rules page that is resulting in a value of 1 and worst case possibly resulting in a negative value.\n- And other interpretations suggesting the Levenshtein Distance (D) is computed first, then normalized and inverted based on the number of ground truth characters (N) to act as a similarity measure, potentially resulting in a value between -1 and 1.\n\nInteresting read.\n\n_____\n\n##### [Sequence modeling with CTC loss](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409512) By [Mykola](https://www.kaggle.com/meowmeowmeowmeowmeow)\n\n- This [post](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409512) discuss the use of Connectionist Temporal Classification (CTC) loss for sequence modeling, specifically in the context of predicting sequences of characters.\n- The traditional approach of splitting sequences into groups and classifying each group becomes challenging when dealing with variable-length phrases. CTC loss allows training models with a variable output length.\n- The main idea behind CTC loss is to predict a matrix of shape NxT, where N represents the number of possible characters plus one special reserved character and T is the maximum possible length of the predicted sequence.\n- Each column in the matrix represents N probabilities of characters predicted per frame.\n- The magic happens during decoding, where duplicated characters are squeezed into a single character, using a special reserved character \"#\" to represent duplicates.\n\n**Source:** [CTC tutorial](https://distill.pub/2017/ctc/)\n\n_____\n\n##### [Correct input/output formats](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/412591) By [Mathieu De Coster](https://www.kaggle.com/mdecoster)\n\n\nSince there were some issues at the start of the competition, this post by [Mathieu De Coster](https://www.kaggle.com/mdecoster) published a working notebook for submission and inference.\n\nSo for everyone that might still be struggling, here is a notebook that correctly run and make submissions [here](https://www.kaggle.com/code/wonderingalice/dummy-submission-with-asl-islr-competition-kernel) by [Wondering Alice](https://www.kaggle.com/wonderingalice).\n_____\n\n##### [Weird samples?](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410747) By [Vadim Irtlach](https://www.kaggle.com/vad13irt)\n\nA [post](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410747) by an vadim raised concerns about the weirdness of the samples in the competition.\n\n**From the comments:**\n- It seem to be plenty of single frame samples labelled with long phrases (shown on the comment).\n- It might be wise to remove samples of 1 freq (as seen in the plot below)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4433335%2F724f6ab398a3f59dfc33e52b22e8fa99%2Fabc.png?generation=1684262126411883&alt=media)\n\nSee the full post for more details.\n\n_____\n\n##### [TFLite/Submission Problems Thread](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/413598) By [anokas](https://www.kaggle.com/anokas)\n\nThis is an interesting thread exploring the issues with the submission process for this competition.\nAlthough this is already fixed at the moment (turned out to be some TFLite versions issue), it's still a good read to understand some general background about the competition.\n\n_____\n\n##### [Failing Basic Preprocessing Operations](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/414631) By [Mark Wijkhuizen](https://www.kaggle.com/markwijkhuizen)\n\nPost:\n- The main cause of submission errors is three basic preprocessing steps that fail during submission, but are successful in a Tensorflow Lite 2.9.1 environment. (Again: It turned out to be some TFLite versions issue)\n- Steps tried: determining the dominant hand, filtering out frames without hand coordinates, and padding input to the target length.\n- And resizing step is the only one that succeeds during submission.\n\n_____\n\n##### [Can I use PyTorch for this competition?](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409561) By [Eugene J. Ryu](https://www.kaggle.com/eugeneryu)\n\nIn this [discussion post](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409561), the author prefer to use PyTorch. However, the competition rules state that you should submit TFlite.\n- According to the hosts: You can submit a PyTorch model, but it must be converted first to TFLite before submission.\n- To do so, first save it as ONNX, then convert the ONNX to tf model.\n- Then, convert the tf model to TFLite.\n\n_____\n\n##### [Can not install tflite-runtime in Kaggle kernel](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/411268) By [Wondering Alice](https://www.kaggle.com/wonderingalice)\n\n\nTrying top install TFlite with `pip install tflite-runtime` caused an error.\n\n**Solution from the comments:**\n- `import tensorflow.lite as tflite\n\n\n_____\n\n##### [Update to a small fraction of the train set](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/416148) By [Sohier Dane](https://www.kaggle.com/sohier)\n\n\nThere had been an update to a small fraction of the train set. The update is to remove some inappropriate phrases.\nSo please make sure you have the latest version of the training set.\n_____\n\n\n##### [Feeling a bit overwhelmed](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410262) By [Rajtilak Bhattacharjee](https://www.kaggle.com/rajtilak)\n\nOn this post, the author is feeling a bit overwhelmed and asks for help with the structure of the dataset and how to approach the problem.\nAn amazing response by [canlion](https://www.kaggle.com/canlion) is provided below:\n\n- A video composed of `n` frames that expresses a certain phrase through hand gestures, we obtain n sets of keypoints.\n- Each set is composed of `543` keypoints, and each keypoint is represented by `x`, `y`, `z` coords: From one video, we obtain an array (n, 543 * 3).\n- These arrays of keypoints sets are stored in multiple `parquet` files.\n- `csv` files contains the ID (sequence_id) of each video, the path of the `parquet` file containing the video, and the phrase that the video represents.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1166873%2F2f7143cdd104eeaffcbfced0c39e1fd7%2Fhoho_2.png?generation=1684081467648519&alt=media)\n\n\n_____\n\n##### [How does American Sign Language Competition differs from previous Isolated sign Language competition?](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410554) By [Anshuman Mishra](https://www.kaggle.com/shivanshuman)\n\nThis competition discuss the differences of this competition from the previous competition. Three main questions are asked:\n- How does two competitions differ in terms of techniques to be used, and datasets distributions?\n\n**Answer from the comments:**\nPrevious competition was a classification problem. Here, we are dealing with continuous fingerspelling: Every video contains a sequence of multiple letters, numbers, and other symbols that are spelled one after another and we need to predict all of them.\n\n- What are some do's and don't to be kept in mind for beginners in this type of competitions?\nMainly: Submit. Validate yourself against the leaderboard all the time (but trust your CV). This is the best advice for any competition.\n\n- What maybe the reason of \"[bar is high](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/410459#2260528)\"\nThe main reason that the \"bar is high\" is simply because you have some uneasy constraints on your submission (like TFlite)\n\n_____\n\n##### [Request to Include TensorFlow Operators in Submission Environment for Better Model Flexibility](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/413035) By [Jude TCHAYE](https://www.kaggle.com/tchaye59)\n\nSome info about the limitation of tensorflow operations in the submission environment:\n\n- TensorFlow Operator Limitation: The submission environment for this competition currently does not support TensorFlow operators. We need convert to convert our models to TFLite without using the `tf.lite.OpsSet.SELECT_TF_OPS` flag.\n- This limits many operations: In the current setup, it's difficult to use some simple layers such as `tf.keras.layers.Embedding`, and operations like `tf.gather` or `tf.gather_nd` may not function as expected.\n- Workaround: Although the `tf.lite.OpsSet.SELECT_TF_OPS` flag isn't supported, if it is included but not used in the model, the submission may still be successful.\n\n_____\n\n##### [!!! Wow one more ASL within a week !!! ( Top Solutions of previous competition )](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409416) By [SeshuRajuP 🧘‍♂️](https://www.kaggle.com/seshurajup)\n\nThis thread shares some of the top solutions of the previous competition.\n\n- Best 1st [solution](https://www.kaggle.com/code/kolyaforrat/1st-2nd-place-solution-inference)\n- Best 2nd [solution](https://www.kaggle.com/code/kolyaforrat/cnn-3trans-speedup)\n- 1st [solution](https://www.kaggle.com/code/hoyso48/1st-place-solution-inference)\n\n_____\n\n\n##### [Some samples may not contain the full spelling](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/411682) By [white lady](https://www.kaggle.com/whitelady)\n\n- While performing some exploratory data analysis (EDA), it was discovered that there are 3,876 phrases with fewer frames than letters.\n- This suggests that some phrases may contain incomplete spellings. In total, there are 5,984 phrases with a ratio of frames to letters greater than 0.4.\n\n_____\n\n##### [Framerate of trainingdata](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/420709) By [MaxUhl98](https://www.kaggle.com/maxuhl98)\n\n- The post raises a question about the framerate of the videos used to create the training data for the competition.\n\n**Info from the comments:**\n- The average ASL signers sign at about 60 words per minute, equating to 3-4 words in 150 frames or 3-4 seconds of recording at 37.5-50 fps. However, fingerspelling in the videos may be slower, possibly 45-50 wpm, reducing the frame rate to around 30-35 fps. This could be the default frame rate on many devices like the iPhone. The frame rate may not significantly affect model building.\n\n_____\n\n\n",
    "2336177": " thanks for the summary, This is literally my first attempt in kaggle competition, suddenly regret my decision :D",
    "2335629": "Correct me if I am wrong. It can misguide your estimation of inference time if you install tflite in this way:\n>Can not install tflite-runtime in Kaggle kernel By Wondering Alice\nTrying top install TFlite with pip install tflite-runtime caused an error.\nSolution from the comments:\n`import tensorflow.lite as tflite`\n\nAccording to [the official post](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/414682) and [this discussion](https://www.kaggle.com/competitions/asl-fingerspelling/discussion/409722#2308349), it is better to import tflite in this way:\n```\n!pip install /kaggle/input/tflite-wheels-2140/tflite_runtime_nightly-2.14.0.dev20230508-cp310-cp310-manylinux2014_x86_64.whl\nimport tflite_runtime.interpreter as tflite\n```",
    "2333886": "",
    "2346215": "Thank you for this!! Very helpful"
  }
}