I know there's some of us who choose to inference on the original video dimensions of (frames, features, channels). Since the number of frames is unknown, that means our input shape is (None, #features, #channels).
\nLast competition I ran into an issue training on the TPU with a None dimension for frames. I assume this competition will require a similar set up. Is there a known solution or a way to bypass this issue? It would be nice to be able to train on the TPU VM.
\nI don't remember the exact error but it was somewhere along the lines of needing to know the dimension at compilation time.
", "messages": [ { "id": 2259005, "postDate": "2023-05-14T16:04:18.957Z", "content": "Personally, I'm a big fan of JAX. While shape does have to be known, feeding different shape into JAX training is fine. It will just trigger more recompiling, as each new set of shape meant another recompiling.
", "rawMarkdown": "Personally, I'm a big fan of JAX. While shape does have to be known, feeding different shape into JAX training is fine. It will just trigger more recompiling, as each new set of shape meant another recompiling. ", "votes": 1 }, { "id": 2256842, "postDate": "2023-05-12T19:04:09.670Z", "content": "I am not sure whether this would work with TPU, but tensorflow has tf.RagedTensor object, which could stack tensors of different shapes in one tensor.
", "rawMarkdown": "I am not sure whether this would work with TPU, but tensorflow has tf.RagedTensor object, which could stack tensors of different shapes in one tensor. ", "votes": 1, "replies": [ { "id": 2256848, "postDate": "2023-05-12T19:10:06.360Z", "content": "Thank you for taking the time to reply!
\nThis could work, but I've had issues submitting TFLite models that use RaggedTensor. I'm not sure how some contestants are getting it to work to be honest. That's why I avoided it entirely and used bucket_by_sequence_length().
Ah, yeah. TFLite model may not fully support RagedTensor. And it is not because of RagedTensor (in fact, we successfully used them in previous competition), but may be related to some unknown shapes of tensor types. For instance, we faced an issue with tf.nn.Conv2D is not supported by TFLite. It turns out, slicing by mask loses information about on axis and it became unknown. That is why tflite refused to work. There are too many such cases
", "rawMarkdown": "Ah, yeah. TFLite model may not fully support RagedTensor. And it is not because of RagedTensor (in fact, we successfully used them in previous competition), but may be related to some unknown shapes of tensor types. For instance, we faced an issue with tf.nn.Conv2D is not supported by TFLite. It turns out, slicing by mask loses information about on axis and it became unknown. That is why tflite refused to work. There are too many such cases", "votes": 1 } ] } ] }, { "id": 2256754, "postDate": "2023-05-12T17:33:10.577Z", "content": "I know there's some of us who choose to inference on the original video dimensions of (frames, features, channels). Since the number of frames is unknown, that means our input shape is (None, #features, #channels).
\nLast competition I ran into an issue training on the TPU with a None dimension for frames. I assume this competition will require a similar set up. Is there a known solution or a way to bypass this issue? It would be nice to be able to train on the TPU VM.
\nI don't remember the exact error but it was somewhere along the lines of needing to know the dimension at compilation time.
", "rawMarkdown": "I know there's some of us who choose to inference on the original video dimensions of (frames, features, channels). Since the number of frames is unknown, that means our input shape is (None, #features, #channels).\n\nLast competition I ran into an issue training on the TPU with a None dimension for frames. I assume this competition will require a similar set up. Is there a known solution or a way to bypass this issue? It would be nice to be able to train on the TPU VM.\n\nI don't remember the exact error but it was somewhere along the lines of needing to know the dimension at compilation time." } ], "comments": [ { "id": 2259005, "author_name": "Lime-Cake", "author_url": "", "post_date": "2023-05-14T16:04:18.957000", "content": "Personally, I'm a big fan of JAX. While shape does have to be known, feeding different shape into JAX training is fine. It will just trigger more recompiling, as each new set of shape meant another recompiling.
", "votes": 1, "replies": [] }, { "id": 2256842, "author_name": "Mykola", "author_url": "", "post_date": "2023-05-12T19:04:09.670000", "content": "I am not sure whether this would work with TPU, but tensorflow has tf.RagedTensor object, which could stack tensors of different shapes in one tensor.
", "votes": 1, "replies": [ { "id": 2256848, "author_name": "AbdouE", "author_url": "", "post_date": "2023-05-12T19:10:06.360000", "content": "Thank you for taking the time to reply!
\nThis could work, but I've had issues submitting TFLite models that use RaggedTensor. I'm not sure how some contestants are getting it to work to be honest. That's why I avoided it entirely and used bucket_by_sequence_length().
Ah, yeah. TFLite model may not fully support RagedTensor. And it is not because of RagedTensor (in fact, we successfully used them in previous competition), but may be related to some unknown shapes of tensor types. For instance, we faced an issue with tf.nn.Conv2D is not supported by TFLite. It turns out, slicing by mask loses information about on axis and it became unknown. That is why tflite refused to work. There are too many such cases
", "votes": 1, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2259005": "Personally, I'm a big fan of JAX. While shape does have to be known, feeding different shape into JAX training is fine. It will just trigger more recompiling, as each new set of shape meant another recompiling. ", "2256842": "I am not sure whether this would work with TPU, but tensorflow has tf.RagedTensor object, which could stack tensors of different shapes in one tensor. ", "2256754": "I know there's some of us who choose to inference on the original video dimensions of (frames, features, channels). Since the number of frames is unknown, that means our input shape is (None, #features, #channels).\n\nLast competition I ran into an issue training on the TPU with a None dimension for frames. I assume this competition will require a similar set up. Is there a known solution or a way to bypass this issue? It would be nice to be able to train on the TPU VM.\n\nI don't remember the exact error but it was somewhere along the lines of needing to know the dimension at compilation time." } }