{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Bird Species Classification over 500 bird species\n\n### Classifying 500 bird species using CNN architecutre\n\n*In this notebook I am going to build CNN architecture for classifying 500 species of Birds*\n\n- **We will perform some fundmental steps to build CNN model**:\n\n   \n   1. Importing libraries: Importing some necessary libraries which are needed for this project\n   2. Setting [Weights & Biases](https://wandb.ai/home/) to track the experiemnts \n   3. Loading an image directory and converting them into dataframe with labels as a target species\n   4. EDA: Analyzing the target classes and displaying sample set of images.\n   5. Data Preprocessing: Performing Data Augmentation, resizing, rescaling \n   6. Building CNN model using Keras Pre-trained models\n   7. Tracking the experiments using W&B\n   8. Evaluating the model in a detail\n  \n \n ## Notebook Extension:\n \n > I am going to create a end to end application either using FastAPI or Streamlit to deploy the model in real world environmet using MLOps. Click about GitHub link to know more\n\n**[GitHub Link](https://github.com/karan842/bird-species-classification/)**\n\n## Experiment Tracking:\n> I am tracking model experiments using W&B. Chec it out\n\n**[W&B](https://wandb.ai/karan842/birds-species-classification)**","metadata":{}},{"cell_type":"markdown","source":"# Importing libraries","metadata":{}},{"cell_type":"code","source":"# Classic data science libraries\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport tensorflow as tf\nimport cv2\nimport PIL\nfrom sklearn.model_selection import train_test_split\n\n# tensorflow libaries\nfrom tensorflow import keras\nfrom tensorflow.keras import layers, models\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\nfrom tensorflow.keras.layers import Dense, Dropout, Conv2D, BatchNormalization,MaxPooling2D, GlobalAveragePooling2D\nfrom tensorflow.keras.callbacks import Callback, EarlyStopping, ModelCheckpoint\nfrom tensorflow.keras import optimizers\nfrom tensorflow.keras import Model\nfrom tensorflow.keras.applications import MobileNetV2, VGG19\nfrom tensorflow.keras.layers.experimental import preprocessing\n\n# System libraries\nfrom pathlib import Path\nimport tqdm\nimport warnings\nimport glob\nimport os\n\n# Metrics\nfrom sklearn import metrics\nimport itertools\n\n# WANDB \nimport wandb\nwandb.login()\n\n# Noteboook setting\n%matplotlib inline\nsns.set_style('darkgrid')\nwarnings.filterwarnings(\"ignore\")","metadata":{"execution":{"iopub.status.busy":"2023-02-25T09:09:47.430432Z","iopub.execute_input":"2023-02-25T09:09:47.431118Z","iopub.status.idle":"2023-02-25T09:09:47.448589Z","shell.execute_reply.started":"2023-02-25T09:09:47.431083Z","shell.execute_reply":"2023-02-25T09:09:47.447378Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Load and transform data","metadata":{}},{"cell_type":"code","source":"dataset = '/kaggle/input/100-bird-species/train/'","metadata":{"execution":{"iopub.status.busy":"2023-02-25T09:10:12.339114Z","iopub.execute_input":"2023-02-25T09:10:12.339558Z","iopub.status.idle":"2023-02-25T09:10:12.346423Z","shell.execute_reply.started":"2023-02-25T09:10:12.339501Z","shell.execute_reply":"2023-02-25T09:10:12.345274Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"species_types=os.listdir(dataset)\nspecies_types","metadata":{"execution":{"iopub.status.busy":"2023-02-25T09:10:12.688303Z","iopub.execute_input":"2023-02-25T09:10:12.688653Z","iopub.status.idle":"2023-02-25T09:10:12.710827Z","shell.execute_reply.started":"2023-02-25T09:10:12.688621Z","shell.execute_reply":"2023-02-25T09:10:12.709933Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Placing data into a DataFrame","metadata":{}},{"cell_type":"code","source":"image_dir = Path(dataset)\n\n# Get filepaths and labels\nfilepaths = list(image_dir.glob(r'**/*.JPG')) + list(image_dir.glob(r'**/*.jpg')) + list(image_dir.glob(r'**/*.png')) + list(image_dir.glob(r'**/*.png'))\n\nlabels = list(map(lambda x: os.path.split(os.path.split(x)[0])[1],filepaths))\n\n# print(labels)","metadata":{"execution":{"iopub.status.busy":"2023-02-25T09:11:45.943479Z","iopub.execute_input":"2023-02-25T09:11:45.943935Z","iopub.status.idle":"2023-02-25T09:11:48.932398Z","shell.execute_reply.started":"2023-02-25T09:11:45.943899Z","shell.execute_reply":"2023-02-25T09:11:48.931419Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"filepaths = pd.Series(filepaths, name='Filepath').astype(str)\nlabels = pd.Series(labels,name='Label')\n\n# Concatenate filepaths and labels\nimage_df = pd.concat([filepaths, labels],axis=1)\nimage_df","metadata":{"execution":{"iopub.status.busy":"2023-02-25T09:11:48.93446Z","iopub.execute_input":"2023-02-25T09:11:48.934844Z","iopub.status.idle":"2023-02-25T09:11:49.095105Z","shell.execute_reply.started":"2023-02-25T09:11:48.934808Z","shell.execute_reply":"2023-02-25T09:11:49.094077Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Exploratory Data Analyis","metadata":{}},{"cell_type":"code","source":"image_df['Label'].value_counts(ascending=False)","metadata":{"execution":{"iopub.status.busy":"2023-02-25T09:11:49.096556Z","iopub.execute_input":"2023-02-25T09:11:49.096898Z","iopub.status.idle":"2023-02-25T09:11:49.114881Z","shell.execute_reply.started":"2023-02-25T09:11:49.096865Z","shell.execute_reply":"2023-02-25T09:11:49.113863Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- Count of each class is ranging between 248 to 130","metadata":{}},{"cell_type":"markdown","source":"## Top 200 most occured Bird Species","metadata":{}},{"cell_type":"code","source":"class_freq = {}\nclass_labels = image_df['Label']\nfor label in class_labels:\n    if label in class_freq:\n        class_freq[label] +=1\n    else:\n        class_freq[label] = 1\n        \n# sort the classes by frequency in descending order\nsorted_class = sorted(class_freq.items(),key=lambda x: x[1],reverse=True)\n# print(sorted_class)\ntop_classes = dict(sorted_class[:20])\n\nplt.figure(figsize=(10,8))\nsns.countplot(y=class_labels, order=top_classes.keys())\nplt.xlabel(\"Frequency\")\nplt.ylabel(\"Class Label\")\nplt.title(\"Top 10 classes with most occurrences\")\nplt.show();","metadata":{"execution":{"iopub.status.busy":"2023-02-25T09:12:11.20904Z","iopub.execute_input":"2023-02-25T09:12:11.209448Z","iopub.status.idle":"2023-02-25T09:12:11.716877Z","shell.execute_reply.started":"2023-02-25T09:12:11.209407Z","shell.execute_reply":"2023-02-25T09:12:11.715904Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Common bird species such as Eagle, Pigeon, Chicken, etc","metadata":{}},{"cell_type":"code","source":"# Function to fing different types of spcies of some common birds\ndef diff_species_common(specie,df):\n    bird_species = df[df['Label'].str.contains(specie,case=False)]['Label']\n    bird_species = bird_species.unique()\n    return bird_species\n\nprint(\"Different types of species for an Eagle: \\n\")\nprint(diff_species_common('EAGLE', image_df))\nprint(\"\\n\\nDifferent types of species for a Pigeon: \\n\")\nprint(diff_species_common('PIGEON',image_df))\nprint(\"\\n\\nDifferent types of species for a Chicken: \\n\")\nprint(diff_species_common('CHICKEN',image_df))\nprint(\"\\n\\nDifferent types of species for a Duck: \\n\")\nprint(diff_species_common('DUCK',image_df))\nprint(\"\\n\\nDifferent types of species for a Vulture: \\n\")\nprint(diff_species_common('SPARROW',image_df))\nprint(\"\\n\\nDifferent types of species for an Owl: \\n\")\nprint(diff_species_common('OWL',image_df))\nprint(\"\\n\\nDifferent types of species for a Raven: \\n\")\nprint(diff_species_common('RAVEN',image_df))\nprint(\"\\n\\nDifferent types of species for a Sparrow: \\n\")\nprint(diff_species_common('SPARROW',image_df))","metadata":{"execution":{"iopub.status.busy":"2023-02-25T09:12:53.55524Z","iopub.execute_input":"2023-02-25T09:12:53.555723Z","iopub.status.idle":"2023-02-25T09:12:54.172392Z","shell.execute_reply.started":"2023-02-25T09:12:53.555682Z","shell.execute_reply":"2023-02-25T09:12:54.171329Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Visualizing sample images from dataset","metadata":{}},{"cell_type":"code","source":"random_index = np.random.randint(0,len(image_df),25)\nfig, axes = plt.subplots(nrows=5,ncols=5,figsize=(10,10),\n                        subplot_kw={'xticks':[],'yticks':[]})\nfor i, ax in enumerate(axes.flat):\n    ax.imshow(plt.imread(image_df.Filepath[random_index[i]]))\n    ax.set_title(image_df.Label[random_index[i]])\nplt.tight_layout()\nplt.show();","metadata":{"execution":{"iopub.status.busy":"2023-02-25T09:12:54.827839Z","iopub.execute_input":"2023-02-25T09:12:54.828213Z","iopub.status.idle":"2023-02-25T09:12:56.656044Z","shell.execute_reply.started":"2023-02-25T09:12:54.828179Z","shell.execute_reply":"2023-02-25T09:12:56.648378Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data Preprocssing\nThe data will be split into three different categories: Training, Validation and Testing. The training data will be used to train the deep learning CNN model and its parameters will be fine tuned with the validation data. Finally, the performance of the data will be evaluated using the test data(data the model has not previously seen).","metadata":{}},{"cell_type":"code","source":"# Seperate in train and test data\ntrain_df, test_df = train_test_split(image_df, test_size=0.2,\n                                     shuffle=True,random_state=42)","metadata":{"execution":{"iopub.status.busy":"2023-02-25T09:12:57.47751Z","iopub.execute_input":"2023-02-25T09:12:57.478662Z","iopub.status.idle":"2023-02-25T09:12:57.502634Z","shell.execute_reply.started":"2023-02-25T09:12:57.478612Z","shell.execute_reply":"2023-02-25T09:12:57.501761Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Data Augmentation ","metadata":{}},{"cell_type":"code","source":"# train_gen = ImageDataGenerator(\n#     rotation_range = 15,\n#     width_shift_range = 0.05,\n#     height_shift_range = 0.05,\n#     rescale = 1./255,\n#     shear_range = 0.05,\n#     brightness_range = [0.1,1.5],\n#     horizontal_flip = True,\n#     vertical_flip = True\n# )\n\ntrain_gen = ImageDataGenerator(\n    preprocessing_function=tf.keras.applications.mobilenet_v2.preprocess_input,\n    validation_split=0.2\n)\n\ntest_gen = ImageDataGenerator(\n    preprocessing_function=tf.keras.applications.mobilenet_v2.preprocess_input\n)","metadata":{"execution":{"iopub.status.busy":"2023-02-25T09:12:58.322816Z","iopub.execute_input":"2023-02-25T09:12:58.323377Z","iopub.status.idle":"2023-02-25T09:12:58.331417Z","shell.execute_reply.started":"2023-02-25T09:12:58.323342Z","shell.execute_reply":"2023-02-25T09:12:58.329855Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Split the data into three categories\ntrain_images = train_gen.flow_from_dataframe(\n    dataframe=train_df,\n    x_col='Filepath',\n    y_col='Label',\n    target_size=(224,224),\n    color_mode='rgb',\n    class_mode='categorical',\n    batch_size=32,\n    shuffle=True,\n    seed=42,\n    subset='training'\n)\n\nval_images = train_gen.flow_from_dataframe(\n    dataframe=train_df,\n    x_col='Filepath',\n    y_col='Label',\n    target_size=(224,224),\n    color_mode='rgb',\n    class_mode='categorical',\n    batch_size=32,\n    shuffle=True,\n    seed=42,\n    subset='validation'\n)\n\ntest_images = test_gen.flow_from_dataframe(\n    dataframe=test_df,\n    x_col='Filepath',\n    y_col='Label',\n    target_size=(224, 224),\n    color_mode='rgb',\n    class_mode='categorical',\n    batch_size=32,\n    shuffle=False\n)","metadata":{"execution":{"iopub.status.busy":"2023-02-25T09:12:58.858302Z","iopub.execute_input":"2023-02-25T09:12:58.858671Z","iopub.status.idle":"2023-02-25T09:13:32.267558Z","shell.execute_reply.started":"2023-02-25T09:12:58.858637Z","shell.execute_reply":"2023-02-25T09:13:32.266529Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Resize Layer\nresize_and_rescale = tf.keras.Sequential([\n    layers.experimental.preprocessing.Resizing(224,224),\n    layers.experimental.preprocessing.Rescaling(1./255),\n])","metadata":{"execution":{"iopub.status.busy":"2023-02-25T09:13:32.269936Z","iopub.execute_input":"2023-02-25T09:13:32.270634Z","iopub.status.idle":"2023-02-25T09:13:32.280462Z","shell.execute_reply.started":"2023-02-25T09:13:32.270597Z","shell.execute_reply":"2023-02-25T09:13:32.279496Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Training the model","metadata":{}},{"cell_type":"code","source":"base_model = MobileNetV2(\n    input_shape=(224,224,3),\n    include_top=False,\n    weights='imagenet',\n    pooling='avg'\n)\n\nbase_model.trainable=False","metadata":{"execution":{"iopub.status.busy":"2023-02-25T09:13:32.282406Z","iopub.execute_input":"2023-02-25T09:13:32.283279Z","iopub.status.idle":"2023-02-25T09:13:33.365625Z","shell.execute_reply.started":"2023-02-25T09:13:32.283241Z","shell.execute_reply":"2023-02-25T09:13:33.364622Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"inputs = base_model.input\nx = resize_and_rescale(inputs)\nx = Dense(256, activation='relu')(base_model.output)\nx = Dropout(0.2)(x)\nx = Dense(256,activation='relu')(x)\nx = Dropout(0.2)(x)\n\noutputs = Dense(500,activation='softmax')(x)\n\nmodel =  Model(inputs=inputs, outputs=outputs)\n\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2023-02-25T09:13:33.368051Z","iopub.execute_input":"2023-02-25T09:13:33.368465Z","iopub.status.idle":"2023-02-25T09:13:33.768964Z","shell.execute_reply.started":"2023-02-25T09:13:33.368426Z","shell.execute_reply":"2023-02-25T09:13:33.768036Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Weights and Biases configuration","metadata":{}},{"cell_type":"code","source":"# Run \nfrom wandb.keras import WandbCallback\nrun = wandb.init(project='birds-species-classification',\n                config={ # include hyperparameters and metadata\n                    \"learning_rate\":0.0001,\n                    \"epochs\":100,\n                    \"batch_size\":32,\n                    \"es_patience\":8,\n                    \"loss_function\":\"categorical_crossentropy\",\n                    \n                })\nconfig = wandb.config # we will use this to configure our experiment\ntf.keras.backend.clear_session()\n\n# model callbacks, optimizers and compilation\n\n## Optimizer\noptimizer = tf.keras.optimizers.Adam(config.learning_rate)\n\n## Early Stopping\nearly_stopping = EarlyStopping(monitor='val_loss', patience=config.es_patience,\n                              restore_best_weights=True)\n# Create checkpoint callback\ncheckpoint_path = \"birds_classification_model_checkpoint\"\ncheckpoint_callback = ModelCheckpoint(checkpoint_path,\n                                      save_weights_only=True,\n                                      monitor=\"val_accuracy\",\n                                      save_best_only=True)\n\n# Model compile\nmodel.compile(optimizer,config.loss_function,metrics=['accuracy'])","metadata":{"execution":{"iopub.status.busy":"2023-02-25T09:13:33.770212Z","iopub.execute_input":"2023-02-25T09:13:33.771112Z","iopub.status.idle":"2023-02-25T09:13:46.874122Z","shell.execute_reply.started":"2023-02-25T09:13:33.771074Z","shell.execute_reply":"2023-02-25T09:13:46.873014Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"history = model.fit(\n    train_images, \n    steps_per_epoch=len(train_images),\n    validation_data=val_images,\n    validation_steps=len(val_images),\n    epochs=config.epochs,\n    callbacks=[\n        early_stopping,\n        WandbCallback(),\n        checkpoint_callback,\n    ]\n)","metadata":{"execution":{"iopub.status.busy":"2023-02-25T09:13:46.879206Z","iopub.execute_input":"2023-02-25T09:13:46.881599Z","iopub.status.idle":"2023-02-25T11:12:09.967Z","shell.execute_reply.started":"2023-02-25T09:13:46.881554Z","shell.execute_reply":"2023-02-25T11:12:09.966025Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model Evaluation","metadata":{}},{"cell_type":"code","source":"results = model.evaluate(test_images)\nprint(\"    Test Loss: {:.5f}%\".format(results[0]))\nprint(\"Test Accuracy: {:.2f}%\".format(results[1]*100))","metadata":{"execution":{"iopub.status.busy":"2023-02-25T11:12:16.842698Z","iopub.execute_input":"2023-02-25T11:12:16.843063Z","iopub.status.idle":"2023-02-25T11:14:39.038683Z","shell.execute_reply.started":"2023-02-25T11:12:16.843032Z","shell.execute_reply":"2023-02-25T11:14:39.037561Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_loss_accuracy_curves(history):\n    # summarize history for accuracy\n    plt.plot(history.history['accuracy'])\n    plt.plot(history.history['val_accuracy'])\n    plt.title('model accuracy')\n    plt.ylabel('accuracy')\n    plt.xlabel('epoch')\n    plt.legend(['Train', 'Validation'], loc='upper left')\n    plt.show()\n    # summarize history for loss\n    plt.plot(history.history['loss'])\n    plt.plot(history.history['val_loss'])\n    plt.title('model loss')\n    plt.ylabel('loss')\n    plt.xlabel('epoch')\n    plt.legend(['Train', 'Validation'], loc='upper left')\n    plt.show();\n    \nplot_loss_accuracy_curves(history)","metadata":{"execution":{"iopub.status.busy":"2023-02-25T11:14:39.04085Z","iopub.execute_input":"2023-02-25T11:14:39.04121Z","iopub.status.idle":"2023-02-25T11:14:39.586559Z","shell.execute_reply.started":"2023-02-25T11:14:39.041163Z","shell.execute_reply":"2023-02-25T11:14:39.585605Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Making predictiong on the test data","metadata":{}},{"cell_type":"markdown","source":"Predict the label of test_images","metadata":{}},{"cell_type":"code","source":"pred = model.predict(test_images)\npred = np.argmax(pred,axis=1)\npred","metadata":{"execution":{"iopub.status.busy":"2023-02-25T11:14:39.587901Z","iopub.execute_input":"2023-02-25T11:14:39.588701Z","iopub.status.idle":"2023-02-25T11:15:22.821574Z","shell.execute_reply.started":"2023-02-25T11:14:39.588661Z","shell.execute_reply":"2023-02-25T11:15:22.820488Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# train_images.class_indices","metadata":{"execution":{"iopub.status.busy":"2023-02-25T11:15:22.824328Z","iopub.execute_input":"2023-02-25T11:15:22.824741Z","iopub.status.idle":"2023-02-25T11:15:22.831024Z","shell.execute_reply.started":"2023-02-25T11:15:22.824692Z","shell.execute_reply":"2023-02-25T11:15:22.830078Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Map the label","metadata":{}},{"cell_type":"code","source":"labels = (train_images.class_indices)\nlabels = dict((v,k) for k,v in labels.items())\npred = [labels[k] for k in pred]\n\n# Result\nprint(f'First 5 predictions: {pred[:5]}')","metadata":{"execution":{"iopub.status.busy":"2023-02-25T11:15:22.834288Z","iopub.execute_input":"2023-02-25T11:15:22.83512Z","iopub.status.idle":"2023-02-25T11:15:22.849482Z","shell.execute_reply.started":"2023-02-25T11:15:22.835074Z","shell.execute_reply":"2023-02-25T11:15:22.848429Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Display 25 Random images from dataset with their labels and predict with the model","metadata":{}},{"cell_type":"code","source":"random_index = np.random.randint(0,len(test_df)-1,25)\nfig, axes = plt.subplots(nrows=5, ncols=5,figsize=(25,15),\n                        subplot_kw={'xticks': [], 'yticks': []})\nfor i, ax in enumerate(axes.flat):\n    ax.imshow(plt.imread(test_df.Filepath.iloc[random_index[i]]))\n    if test_df.Label.iloc[random_index[i]] == pred[random_index[i]]:\n        color = \"green\"\n    else:\n        color = \"red\"\n    ax.set_title(f\"True: {test_df.Label.iloc[random_index[i]]}\\nPredicted: {pred[random_index[i]]}\",\n                color=color)\n\nplt.show()\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-02-25T11:15:22.850864Z","iopub.execute_input":"2023-02-25T11:15:22.851206Z","iopub.status.idle":"2023-02-25T11:15:24.611731Z","shell.execute_reply.started":"2023-02-25T11:15:22.851171Z","shell.execute_reply":"2023-02-25T11:15:24.610412Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Classification Report\nWe will convert our classification report into dataframe and find the best and worst classes using pandas operations for ease.","metadata":{}},{"cell_type":"code","source":"species = test_images.class_indices","metadata":{"execution":{"iopub.status.busy":"2023-02-25T11:15:24.613373Z","iopub.execute_input":"2023-02-25T11:15:24.614168Z","iopub.status.idle":"2023-02-25T11:15:24.618546Z","shell.execute_reply.started":"2023-02-25T11:15:24.61411Z","shell.execute_reply":"2023-02-25T11:15:24.617583Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_test = list(test_df.Label)\nreport = metrics.classification_report(y_test, pred,target_names=species,output_dict=True)\n\ndata = []\nfor k,v in report.items():\n    if k in species:\n        data.append({'species':k,\n                    'precision': v['precision'],\n                    'recall': v['recall'],\n                    'f1-score': v['f1-score'],\n                    'support': v['support']})\ndf = pd.DataFrame(data)\ndf","metadata":{"execution":{"iopub.status.busy":"2023-02-25T11:15:24.620168Z","iopub.execute_input":"2023-02-25T11:15:24.620812Z","iopub.status.idle":"2023-02-25T11:15:24.958832Z","shell.execute_reply.started":"2023-02-25T11:15:24.620776Z","shell.execute_reply":"2023-02-25T11:15:24.957894Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def best_classes(df):\n    print(\"Classed with good result: \")\n    top_10_good = df.nlargest(10,'f1-score')\n    res = top_10_good[['species','precision','recall','f1-score','support']]\n    return res\n    \nbest_classes(df)","metadata":{"execution":{"iopub.status.busy":"2023-02-25T11:15:24.960365Z","iopub.execute_input":"2023-02-25T11:15:24.961187Z","iopub.status.idle":"2023-02-25T11:15:24.980551Z","shell.execute_reply.started":"2023-02-25T11:15:24.96115Z","shell.execute_reply":"2023-02-25T11:15:24.979427Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def worst_classes(df):\n    print(\"Classed with bad result: \")\n    top_10_bad = df.nsmallest(10,'f1-score')\n    res = top_10_bad[['species','precision','recall','f1-score','support']]\n    return res\n    \nworst_classes(df)","metadata":{"execution":{"iopub.status.busy":"2023-02-25T11:15:24.98416Z","iopub.execute_input":"2023-02-25T11:15:24.984428Z","iopub.status.idle":"2023-02-25T11:15:25.003497Z","shell.execute_reply.started":"2023-02-25T11:15:24.984403Z","shell.execute_reply":"2023-02-25T11:15:25.002575Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wandb.finish()","metadata":{"execution":{"iopub.status.busy":"2023-02-25T11:36:31.477367Z","iopub.execute_input":"2023-02-25T11:36:31.477759Z","iopub.status.idle":"2023-02-25T11:36:38.94645Z","shell.execute_reply.started":"2023-02-25T11:36:31.477728Z","shell.execute_reply":"2023-02-25T11:36:38.945659Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Saving best model","metadata":{}},{"cell_type":"code","source":"api = wandb.Api()\n\nruns = api.runs(\"karan842/birds-species-classification\")","metadata":{"execution":{"iopub.status.busy":"2023-02-25T11:40:26.977272Z","iopub.execute_input":"2023-02-25T11:40:26.977669Z","iopub.status.idle":"2023-02-25T11:40:27.403035Z","shell.execute_reply.started":"2023-02-25T11:40:26.977636Z","shell.execute_reply":"2023-02-25T11:40:27.402089Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get the best run based on the validation accuracy\nbest_run = None\nbest_val_acc = 0\n\nfor run in runs:\n    if run.state != \"finished\":\n        continue\n    if \"val_accuracy\" not in run.summary:\n        continue\n    val_acc = run.summary[\"val_accuracy\"]\n    if val_acc > best_val_acc:\n        best_val_acc = val_acc\n        best_run = run\n\n# Print the best run ID and validation accuracy\nif best_run is not None:\n    print(\"Best run: \", best_run.name)\n    print(\"Validation accuracy: \", best_val_acc)\nelse:\n    print(\"No runs found.\")","metadata":{"execution":{"iopub.status.busy":"2023-02-25T11:45:43.126799Z","iopub.execute_input":"2023-02-25T11:45:43.127142Z","iopub.status.idle":"2023-02-25T11:45:43.134369Z","shell.execute_reply.started":"2023-02-25T11:45:43.127114Z","shell.execute_reply":"2023-02-25T11:45:43.133368Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get the best run based on the accuracy\nbest_run2 = None\nbest_acc = 0\n\nfor run in runs:\n    if run.state != \"finished\":\n        continue\n    if \"val_accuracy\" not in run.summary:\n        continue\n    val_acc = run.summary[\"accuracy\"]\n    if val_acc > best_val_acc:\n        best_val_acc = val_acc\n        best_run2 = run\n\n# Print the best run ID and validation accuracy\nif best_run is not None:\n    print(\"Best run: \", best_run2.name)\n    print(\"Accuracy: \", best_acc)\nelse:\n    print(\"No runs found.\")","metadata":{"execution":{"iopub.status.busy":"2023-02-25T11:45:46.241377Z","iopub.execute_input":"2023-02-25T11:45:46.24205Z","iopub.status.idle":"2023-02-25T11:45:46.251769Z","shell.execute_reply.started":"2023-02-25T11:45:46.242012Z","shell.execute_reply":"2023-02-25T11:45:46.250714Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Current model has best val_accuracy score\n- Saving the model in .h5 format","metadata":{"execution":{"iopub.status.busy":"2023-02-25T11:41:11.452492Z","iopub.execute_input":"2023-02-25T11:41:11.4535Z","iopub.status.idle":"2023-02-25T11:41:11.46278Z","shell.execute_reply.started":"2023-02-25T11:41:11.453463Z","shell.execute_reply":"2023-02-25T11:41:11.460504Z"}}},{"cell_type":"code","source":"model.save('best_model.h5')\nprint('Model Saved!!')","metadata":{"execution":{"iopub.status.busy":"2023-02-25T11:41:54.464089Z","iopub.execute_input":"2023-02-25T11:41:54.464458Z","iopub.status.idle":"2023-02-25T11:41:54.801295Z","shell.execute_reply.started":"2023-02-25T11:41:54.464426Z","shell.execute_reply":"2023-02-25T11:41:54.800139Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Predictions!!","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras.models import load_model\n\n# Load the saved model\nmodel = load_model('best_model.h5')","metadata":{"execution":{"iopub.status.busy":"2023-02-25T11:50:09.777559Z","iopub.execute_input":"2023-02-25T11:50:09.778014Z","iopub.status.idle":"2023-02-25T11:50:11.454348Z","shell.execute_reply.started":"2023-02-25T11:50:09.777978Z","shell.execute_reply":"2023-02-25T11:50:11.453314Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from tensorflow.keras.preprocessing import image\nfrom tensorflow.keras.applications.vgg16 import preprocess_input\n\ndef predict_image_class(model, img_path):\n    # Load and preprocess the input image\n    img = image.load_img(img_path, target_size=(224, 224))\n    x = image.img_to_array(img)\n    x = np.expand_dims(x, axis=0)\n    x = preprocess_input(x)\n\n    # Make a prediction using the model\n    preds = model.predict(x)\n\n    # Get the top 5 predicted classes and their probabilities\n    top_preds = np.argsort(-preds)[0][:5]\n    top_probs = preds[0][top_preds]\n    \n    # Print the top 5 predicted classes and their probabilities\n    print(\"\\nTop 5 predicted classes and their probabilities:\")\n    for i in range(len(top_preds)):\n        print(\"{:2d}. {}: {:.2f}%\".format(i+1, species_types[top_preds[i]], 100*top_probs[i]))","metadata":{"execution":{"iopub.status.busy":"2023-02-25T11:54:19.718011Z","iopub.execute_input":"2023-02-25T11:54:19.718593Z","iopub.status.idle":"2023-02-25T11:54:19.731731Z","shell.execute_reply.started":"2023-02-25T11:54:19.71853Z","shell.execute_reply":"2023-02-25T11:54:19.729316Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Predicting image class baed on input image","metadata":{}},{"cell_type":"code","source":"img = '/kaggle/input/100-bird-species/train/BALD IBIS/002.jpg'\npredict_image_class(model,img)","metadata":{"execution":{"iopub.status.busy":"2023-02-25T11:56:06.427054Z","iopub.execute_input":"2023-02-25T11:56:06.427453Z","iopub.status.idle":"2023-02-25T11:56:06.494545Z","shell.execute_reply.started":"2023-02-25T11:56:06.427421Z","shell.execute_reply":"2023-02-25T11:56:06.493566Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- In future we can increase the model performance. This was just a practice.","metadata":{}},{"cell_type":"markdown","source":"# End Notes:\n\n## Give an upvote to a kernel also try to check Github repository given above","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}