{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Who?\nThis notebook is for all those who need a slap in the face to make them realise that their understanding of `CNN` and `RNN`(next post) has been totally wrong and they want to have *My whole life has been a lie* moment.\n\nI suppose that you have a basic understandign of `CNN` networks. I'll try to keep the working as short as possible. If you don't know, you can read and come to this notebook again to bust your myths.\n\nTo all the experienced professionals, please burst my bubble too if you feel if I have understood something incorrectly. We'll be covering:\n\n1. How Exactly CNN\n2. What exactly they do\n3. What does the `Number of Neurons` do in these networks.\n4. Some misconceptions about the two networks\n\n**[For same in-depth intutive knowledge on RNNs, Visit this link which might look like a Clickbait but isn't](https://www.kaggle.com/deshwalmahesh/rnn-you-ve-been-reading-it-the-wrong-way)**\n\n#### Note: If you want to get a basic understanding about the CNNs or Keras, I highly recommend to [read this notebook thoroughly](https://www.kaggle.com/deshwalmahesh/bengali-ai-complete-beginner-tutorial-95-acc)"},{"metadata":{},"cell_type":"markdown","source":"# HOW does CNN work?"},{"metadata":{},"cell_type":"markdown","source":"In a single line, CNN are very effective for any input if the input has `Spatial Information`. What is spatial? \"In Space\", I'd say. In space? Well if it has some kind of \"continuous\" information which is related to each other in a way. For example for text that your are reading, how are you able to make up the meaning of this long sentence? Because it has spatial information due to the unique combination of words and the words before and after each word are there for a reason.\n\nSame with Images. You have a structure like a human because it has a structure that it starts from head, neck,torso and legs. It won't makesnese if you saw a picture of a human which had head, torso,legs and then neck. So the unique combination of pixels around the 2-D plane puts some meaning to the image. \n\nSo CNN can be used with Image as well as with text data. \n\nSo what exactly happens there? A magic called MAtrix multiplaction. This is what happens with each and every deep learning models. It is is just like changing the numbers inside your matrix depending upon the input you just saw. What makes deep learning unique is that those numbers are in millions and billions. So you try to find kind of an *average* matrix based on all the unique data it has seen in the past. Deep learnign models try to find numbers in a manner that can represent something like an `average` information of all the data points it has seen and it happens for billions of numbers."},{"metadata":{},"cell_type":"markdown","source":"## What and how does a CNN learn?"},{"metadata":{},"cell_type":"markdown","source":"CNN learns by something called `Convolving the kernels` (that rhymes though) over the image. What are kernels? \n\nKernels = Filter = Weights that are learned = Matrix of given size which has numbers in it = Find featurees\n\nHow does it do that? Let us look at the matrix given below:\n\n` filter = [[ -1, 0, 1],\n [-2, 0, 2],\n [-1, 0, 1]]\n`\nIf you use this matrix with `cv2.filter2D(image,-1,filter)`, You'll see an output which is your original image but som,e black and white lines, something cool. It is called `Sobel Filter` which is used to find the edges in a nimage. I used this specific filter because I knew how it works. \n\n\n\nSo what is the meaning in CNN? \nThe above is a *Single* filter which has `3x3` dimensions and there are numbers in this filter. So when you do something `Conv2d(filters=1, kernel_size=3)` there are chances that it might end up with any of these filters.\n\n### CNN learns the NUMBERS inside a filter BY THEMSELVES using BackPropagation\nCNN will try to find *WHAT FILTER TO USE AT WHICH LAYER*? And the first argument you use as `Number of Neurons` are *HOW MANY filters of that size to find*. Those won't be identical (Until you use too many layers with too little size of filters). Each one having different numbers or combinations so that each of those can extract a different pattern etc. \n\n## How it is done?\nIn intutive terms, Filter of any size will slide over all the pixels from left to right and then top to bottom by multiplying the two matrices inside the *Receptive Field*. What is this? So the area ( numbers of pixels) that a kernel covers at any given time is the receptive field of that filter. So a `3x3` filter will cover 9 pixels at any given time but what matter is `WHICH` 9 pixels. Receptive field depends on the *Stride* given. It might be the case that a Filter covers first 9 pixels in the first 3 rows and 3 columns of an image and then the next receptiove fiield is the last 3 rows and 3 columns.\n\n\n\n\n## Maths in Action:\n\nLet us look at the *Exact Maths* of how it is done:\n1. Let's look at the black Box first. Here the First Matrix is Image, Second is Kernel and the third one is Output of that convolution.\n \n\n2. The Kernal is rearranged in a specific pattern as: \n\n\n3. Okay Look at that Kernal man!! In a different way this time ;) \n\n\n4. Image is Flattened as: \n\n\n5. Boom!!! Magic!! Matrix Multiplication \n\n\n6. That is not my image man!! Speak for yourself? \n\n\n\n## How are input and outputs are changed inside a layer?\nThis is purely dependent on the `Number of features (No of Neurons)`, `Size of that Filter used (Kernel Size)`, `Padding` and the `Stride`. Rest is again, *Matrix Multiplication*. When we do `Padding='Same'`, it means that

Pad the image in a way thhat even after convolution, the input Image's Width and Height equal to Output Image's Weidth and height

. Only thing changes here is that previous `P` features or channelsare changed into new `M` features. M or N can be anything. So the Image `(W,H,P)` will be `(W,H,M)`.\n\nOkay Wait!

So if an image has 3 Dimension (32,32,3), how come it becomes, (MxN,64)?

.\n\nIt does not take a *Square* into consideration but a *Cube* this time. So let us suppose you have a stack of paper and a kernel of size 1x1 acts like a needle and stride of 1. The convolution will like Piercing through all the points in the stacks one sectipn at a time. \n\n\n\n\nAnd Look at the broad figure in case you were wondering.\n\n## Everyone Knows Pooling. \n\nIt is used to Change the Dimensions based on the Averaging the Pixels, Taking the maximum from a group of Pixels or you can define your own method (If you're not as dumb as I am. I can't think of a new way!)\n\n\n\n## Use of Activation Function?\nLet us suppose we use `Relu` so it'll convert every negative value to 0 and keep positive value as they are.\n \n\nAnd here comes the [Gradients or Derivative of Relu](https://missinglink.ai/guides/neural-network-concepts/7-types-neural-network-activation-functions-right/). Which makes it special for the problem called Gradient Saturation.\n\n\n\nOkay!! But why?\nAfter we get a feature map, we'll have some positives andsome negative values. `Relu` will make all the Negative zeros. Again, But why? Because the negative values won't be adding any Important thing. It'll *mostly* be Noise. Si it'll filter the noise out of the image.\n\n\n\n## What happens during the Backpropagation of CNN?\nBefore I blow your mind, please [Go to this link and watch this awesome video by Fei-Fei Li on Backpropagation on Computation Graphs](https://youtu.be/d14TUNcbn1k?t=236).\n\nSo in backprop, we'll be looking at what should be the value of all those numbers in each Kernel/filter that we used last time. this is calculated by the Chain rule or so which you obviously covered in the last Last given. But there is an interesting thing that happens about how does it happen. \n\nOkay so here we go for big blow: Loss Gradients (`L`) with respect to the input X `(∂L/∂X)` can be represented as ‘full’ convolution between a 180-degree rotated Filter `F1` and loss gradient (`∂L/∂O`). Means? Backward pass during Backpropagation also uses Convolutions. Means> Both the Forward pass and the Backpropagation of a Convolutional layer are Convolutions.\n\nAlso the Propagation of errors in Pooling is a bit different. To know the Best about Backprop in CNN, just visit these twwo links.\n\n1. [Backpropagation In Convolution and Pooling Layers](https://www.jefkine.com/general/2016/09/05/backpropagation-in-convolutional-neural-networks/)\n2. [Convolutions and Backpropagations](https://medium.com/@pavisj/convolutions-and-backpropagations-46026a8f5d2c)\n\n\n## Ever Wondered What exactly does the CNN see? Me too!\nBasically Neural Netrowks don't tend to look for anything specific in particular. We humans looks for color, edge, boundy, pattern, line, angle, shade bla bla bla!! But NN just learn. Yes! Exactly, they just learn. They adjust their numbers inside the kernel based on looking at at huge training examples and they feel very happy that they found something common interesting or useful. Now I don't know whether they are too naive dumb or too smart. But to them, it's just numbers. If you think they are too dumb, look at you and around you. We are linear people and can perceive things in linear way. To computers, it's just 0 and 1 and still they have to live up to your expectations of non-linearity. Feling dumb? Me too.\n\nIf you look at the Gradients and activation mapping of the the images, you'll see that CNN are basically on an acid trip. Maybe they are and that is the only reason they are able to find things we human cant' comprehend. there are various ways to see what a CNN see. I'll just give you some links in the End so that you can check nd see the whole picture. But for now, just look athe image below and feel some dizziness in your head.\n\n\n\n## What does it tell you?\nWell, it can tell you a lot of things starting from that \"CNN is not a blackbox which most of people think it is\". Also, when you look at the Gradients, Activation Maps and other things from the network, you can find out that you can \"Kind of know what led the CNN to make the decision that it should be the class for the givern picture\". It means that when you look at the **Activation map of the predicted class**, you can know that what features did it mostly look at to reach the certain conclusion. Graddients mapping tells us that which neurons were active while giving the results. You can look at those layers by layers or make a kind of average scenario and change it to heatmap. Please [Go and visit this Notebook](https://www.kaggle.com/aakashnain/what-does-a-cnn-see) to find out exactly how you can create a Heatmap of Gradients and superimpose it on your image to see what were the features the network was looking at to give you the exact class prediction.\n\n\n\n# References:\n1. [Up-sampling with Transposed Convolution](https://naokishibuya.medium.com/up-sampling-with-transposed-convolution-9ae4f2df52d0)\n2. [Convolutional Neural Network(CNN) Simplified](https://medium.com/datadriveninvestor/convolutional-neural-network-cnn-simplified-ecafd4ee52c5#:~:text=CNN%20stands%20for%20Convolutional%20Neural,for%20image%20detection%20and%20classification.)\n3. [Simple Introduction to Convolutional Neural Networks](https://towardsdatascience.com/simple-introduction-to-convolutional-neural-networks-cdf8d3077bac)\n4. [The Most Intuitive and Easiest Guide for Convolutional Neural Network](https://towardsdatascience.com/the-most-intuitive-and-easiest-guide-for-convolutional-neural-network-3607be47480)\n\nWhat they see?\n\n1. [The Building Blocks of Interpretability](https://distill.pub/2018/building-blocks/)\n2. [What exactly does CNN see?](https://becominghuman.ai/what-exactly-does-cnn-see-4d436d8e6e52)\n3. [What does a CNN see? Kaggle. Code](https://www.kaggle.com/aakashnain/what-does-a-cnn-see)"},{"metadata":{},"cell_type":"markdown","source":"# Thank you for giving your time. Please Correct me!\nIn future versions, I'll be giving an idea about what can be done using the CNN and what are `Separable` Convolutions. Will start working if I get time. Thanks again!"},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}