{
  "id": 433812,
  "title": "why tf and np computation are not the same?",
  "url": "/competitions/asl-fingerspelling/discussion/433812",
  "author_name": "hengck23",
  "post_date": "2023-08-23T01:53:43.006000",
  "votes": 3,
  "comment_count": 1,
  "views": 0,
  "content": "<p>i convert my code from np+pytorch to tf+keras for submission. </p>\n<p>I can get almost same results (i.e. np.isclose(…., x, atol=1e-08,)) for 95% of the cases.<br>\nnote that even cv2.resize is exactly the same as tf.image.resize</p>\n<p>But for one function below, results differ</p>\n<pre><code>##tf version\nx is tf tensor  shape (L,), float32\ndef tf: \n    one = tf.where(tf.math.is, tf.zeros, tf.ones)\n    sum = tf.reduce\n\n    x = tf.where(tf.math.is, tf.zeros, x)\n    mean  = tf.reduce/sum\n    mean2 = tf.reduce/sum\n    std = mean2 -mean*mean\n    std = tf.math.sqrt(std)  \n    return mean, std\n</code></pre>\n<pre><code>\n is tf tensor of shape (L,), float32\n = np.nanmean(x.numpy().reshape(-,),)\n  = np.nanstd(x.numpy().reshape(-,),)\n</code></pre>\n<p>the error is only within np.isclose(…., x, atol=1e-03,).<br>\nHowever, if i cast x to tffloat64, the resulst are equal:</p>\n<pre><code>#this   to  \nx = tf.cast(x,tf.float64)\n,  = tf_nan_mean_std(x, axis=, keepdims=True)\n = tf.cast(,tf.float32)\n = tf.cast(,tf.float32)\n</code></pre>\n<p>is there something wrong with  my tf-np conversion? or is it just pure numerical errors (which 1e-3 is too large)?</p>",
  "messages": [
    {
      "id": 2403981,
      "postDate": "2023-08-23T01:53:43.007Z",
      "content": "<p>i convert my code from np+pytorch to tf+keras for submission. </p>\n<p>I can get almost same results (i.e. np.isclose(…., x, atol=1e-08,)) for 95% of the cases.<br>\nnote that even cv2.resize is exactly the same as tf.image.resize</p>\n<p>But for one function below, results differ</p>\n<pre><code>##tf version\nx is tf tensor  shape (L,), float32\ndef tf: \n    one = tf.where(tf.math.is, tf.zeros, tf.ones)\n    sum = tf.reduce\n\n    x = tf.where(tf.math.is, tf.zeros, x)\n    mean  = tf.reduce/sum\n    mean2 = tf.reduce/sum\n    std = mean2 -mean*mean\n    std = tf.math.sqrt(std)  \n    return mean, std\n</code></pre>\n<pre><code>\n is tf tensor of shape (L,), float32\n = np.nanmean(x.numpy().reshape(-,),)\n  = np.nanstd(x.numpy().reshape(-,),)\n</code></pre>\n<p>the error is only within np.isclose(…., x, atol=1e-03,).<br>\nHowever, if i cast x to tffloat64, the resulst are equal:</p>\n<pre><code>#this   to  \nx = tf.cast(x,tf.float64)\n,  = tf_nan_mean_std(x, axis=, keepdims=True)\n = tf.cast(,tf.float32)\n = tf.cast(,tf.float32)\n</code></pre>\n<p>is there something wrong with  my tf-np conversion? or is it just pure numerical errors (which 1e-3 is too large)?</p>",
      "rawMarkdown": "i convert my code from np+pytorch to tf+keras for submission. \n\nI can get almost same results (i.e. np.isclose(...., x, atol=1e-08,)) for 95% of the cases.\nnote that even cv2.resize is exactly the same as tf.image.resize\n\nBut for one function below, results differ\n\n```\n##tf version\nx is tf tensor of shape (L,3), float32\ndef tf_nan_mean_std(x, axis=0, keepdims=True): \n\tone = tf.where(tf.math.is_nan(x), tf.zeros_like(x), tf.ones_like(x))\n\tsum = tf.reduce_sum(one, axis=axis, keepdims=keepdims)\n\n\tx = tf.where(tf.math.is_nan(x), tf.zeros_like(x), x)\n\tmean  = tf.reduce_sum(x, axis=axis, keepdims=keepdims)/sum\n\tmean2 = tf.reduce_sum(x*x, axis=axis, keepdims=keepdims)/sum\n\tstd = mean2 -mean*mean\n\tstd = tf.math.sqrt(std)  \n\treturn mean, std\n\n```\n\n\n```\n##np version\nx is tf tensor of shape (L,3), float32\nmean = np.nanmean(x.numpy().reshape(-1,3),0)\nstd  = np.nanstd(x.numpy().reshape(-1,3),0)\n\n```\n\nthe error is only within np.isclose(...., x, atol=1e-03,).\nHowever, if i cast x to tffloat64, the resulst are equal:\n\n```\n#this is equal to np functions\nx = tf.cast(x,tf.float64)\nmean, std = tf_nan_mean_std(x, axis=0, keepdims=True)\nmean = tf.cast(mean,tf.float32)\nstd = tf.cast(std,tf.float32)\n```\n\nis there something wrong with  my tf-np conversion? or is it just pure numerical errors (which 1e-3 is too large)?",
      "votes": 3
    },
    {
      "id": 2404157,
      "postDate": "2023-08-23T05:06:22.103Z",
      "content": "<p>there is tensorflow experimental numpy -<br>\n<code>import tensorflow.experimental.numpy as tnp</code><br>\n<code>tnp.experimental_enable_numpy_behavior()</code></p>\n<p>not sure it works in tflite but if you want to test above if both results are the same</p>",
      "rawMarkdown": "there is tensorflow experimental numpy -\n`import tensorflow.experimental.numpy as tnp`\n`tnp.experimental_enable_numpy_behavior()`\n\nnot sure it works in tflite but if you want to test above if both results are the same"
    }
  ],
  "comments": [
    {
      "id": 2404157,
      "author_name": "something4kag",
      "author_url": "",
      "post_date": "2023-08-23T05:06:22.103000",
      "content": "<p>there is tensorflow experimental numpy -<br>\n<code>import tensorflow.experimental.numpy as tnp</code><br>\n<code>tnp.experimental_enable_numpy_behavior()</code></p>\n<p>not sure it works in tflite but if you want to test above if both results are the same</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2403981": "i convert my code from np+pytorch to tf+keras for submission. \n\nI can get almost same results (i.e. np.isclose(...., x, atol=1e-08,)) for 95% of the cases.\nnote that even cv2.resize is exactly the same as tf.image.resize\n\nBut for one function below, results differ\n\n```\n##tf version\nx is tf tensor of shape (L,3), float32\ndef tf_nan_mean_std(x, axis=0, keepdims=True): \n\tone = tf.where(tf.math.is_nan(x), tf.zeros_like(x), tf.ones_like(x))\n\tsum = tf.reduce_sum(one, axis=axis, keepdims=keepdims)\n\n\tx = tf.where(tf.math.is_nan(x), tf.zeros_like(x), x)\n\tmean  = tf.reduce_sum(x, axis=axis, keepdims=keepdims)/sum\n\tmean2 = tf.reduce_sum(x*x, axis=axis, keepdims=keepdims)/sum\n\tstd = mean2 -mean*mean\n\tstd = tf.math.sqrt(std)  \n\treturn mean, std\n\n```\n\n\n```\n##np version\nx is tf tensor of shape (L,3), float32\nmean = np.nanmean(x.numpy().reshape(-1,3),0)\nstd  = np.nanstd(x.numpy().reshape(-1,3),0)\n\n```\n\nthe error is only within np.isclose(...., x, atol=1e-03,).\nHowever, if i cast x to tffloat64, the resulst are equal:\n\n```\n#this is equal to np functions\nx = tf.cast(x,tf.float64)\nmean, std = tf_nan_mean_std(x, axis=0, keepdims=True)\nmean = tf.cast(mean,tf.float32)\nstd = tf.cast(std,tf.float32)\n```\n\nis there something wrong with  my tf-np conversion? or is it just pure numerical errors (which 1e-3 is too large)?",
    "2404157": "there is tensorflow experimental numpy -\n`import tensorflow.experimental.numpy as tnp`\n`tnp.experimental_enable_numpy_behavior()`\n\nnot sure it works in tflite but if you want to test above if both results are the same"
  }
}