{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"**WHAT IS CERVICAL SPINE?**\nThe neck is part of a long flexible column, known as the spinal column or backbone, which extends through most of the body. The cervical spine (neck region) consists of seven bones (C1-C7 vertebrae), which are separated from one another by intervertebral discs. These discs allow the spine to move freely and act as shock absorbers during activity.\n\nAttached to the back of each vertebral body is an arch of bone that forms a continuous hollow longitudinal space, which runs the whole length of the back. This space, called the spinal canal, is the area through which the spinal cord and nerve bundles pass. The spinal cord is bathed in cerebrospinal fluid (CSF) and surrounded by three protective layers called the meninges (dura, arachnoid, and pia mater).\n\nAt each vertebral level, a pair of spinal nerves exit through small openings called foraminae (one to the left and one to the right). These nerves serve the muscles, skin and tissues of the body and thus provide sensation and movement to all parts of the body. The delicate spinal cord and nerves are further supported by strong muscles and ligaments that are attached to the vertebrae.\n\n\n\n\n\n**Symptoms of Cervical Stenosis**\nNeck pain may be caused by disc degeneration, narrowing of the spinal canal, arthritis and, in rare cases, cancer or meningitis . Symptoms include:\n\nNeck or arm pain\nNumbness and weakness in the upper extremities hands\nUnsteady gait when walking\nMuscle spasms in the legs\nLoss of coordination in arms, hands, finger\nLoss of muscle tone in arms and/or hands\nDropping items or loss of dexterity of hands\n\n\n\n**Risks and Outcome**\nAlthough complications are fairly rare, as with any surgery, the following risks may be associated with cervical spine surgery:\n\nInfection\nChronic neck or arm pain\nInadequate symptom relief\nDamage to the nerves and nerve roots\nDamage to the spinal cord (about 1 in 10,000), resulting in paralysis\nSpinal instability\nDamage to the esophagus, trachea or vocal cords from anterior approach\nInjury to the carotid or vertebral arteries, which could result in stroke from anterior approach\nFusion that does not occur – pseudarthrosis\nInstrumentation breakage and/or failure\nPersistent swallowing or speech disturbance\nLeakage of cerebral spinal fluid\nThe benefits of surgery should always be weighed carefully against its risks. Although a large percentage of cervical spine patients report significant pain relief after surgery, there is no guarantee that surgery will help every individual.","metadata":{"_uuid":"7315e99b-df91-4f08-ace0-b0aede043739","_cell_guid":"6f56491c-0589-45f5-b843-f00056238379","trusted":true}},{"cell_type":"markdown","source":"**Processing CSV data Pipeline for Machine Learning**\nThe CSV datasets that are readily available may not always be processed and some cleanup may be required. This is a pretty common step in Machine Learning(ML) and Deep Learning(DL) projects.\n\nIn this article, I’ll cover basic techniques for data pre-processing and preparing your CSV dataset for ML/DL applications in Python. Depending on application, you may need to execute some or all of these steps…\n\n**Pre-processing Tasks :**\nImporting the dataset.\nSeparating the dependent and independent variables.\nHandling missing data.\nEncoding categorical data.\nSplitting the dataset (Train Set and Test set).\nFeature Scaling.","metadata":{"_uuid":"1093a772-98b7-418e-8e95-42e7152d2a4b","_cell_guid":"15210889-f849-44bc-9663-fd3d601362c8","trusted":true}},{"cell_type":"markdown","source":"Importing Necessary Libraries","metadata":{"_uuid":"bb0daadf-a671-4885-b5fc-1dba4a52d5d2","_cell_guid":"7cc422d3-bfbd-4450-8631-f3e060c2c415","trusted":true}},{"cell_type":"code","source":"!pip install monai","metadata":{"_uuid":"a75b4013-afeb-479c-b683-4bcd437a9544","_cell_guid":"b8c9432c-14f8-458d-bd07-6a9824ca9910","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:37:33.489529Z","iopub.execute_input":"2022-10-26T19:37:33.48999Z","iopub.status.idle":"2022-10-26T19:40:03.345801Z","shell.execute_reply.started":"2022-10-26T19:37:33.489946Z","shell.execute_reply":"2022-10-26T19:40:03.344597Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport os\nimport gc\nimport numpy as np\nfrom tqdm import tqdm\nimport multiprocessing\nfrom ipywidgets import interactive, widgets, fixed\nfrom matplotlib import animation, rc; rc('animation', html='jshtml')\nimport matplotlib.pyplot as plt\n%matplotlib inline","metadata":{"_uuid":"e61a274e-e9fc-49d6-86f3-3a9b1d469bcb","_cell_guid":"e4be9ff7-de86-4291-b1cf-557c75b19dcd","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:44:31.650593Z","iopub.execute_input":"2022-10-26T19:44:31.65099Z","iopub.status.idle":"2022-10-26T19:44:31.770136Z","shell.execute_reply.started":"2022-10-26T19:44:31.650958Z","shell.execute_reply":"2022-10-26T19:44:31.768968Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Defining CSV File","metadata":{"_uuid":"5c494b22-40b7-41d8-bf85-1d6116e6483d","_cell_guid":"872fda71-5394-4296-abd5-f8b4cb424a69","trusted":true}},{"cell_type":"code","source":"df = pd.read_csv('../input/rsna-2022-cervical-spine-fracture-detection/train_bounding_boxes.csv')\n\ndf.tail()","metadata":{"_uuid":"19186a73-e921-4b04-9417-7068e84dcd65","_cell_guid":"bbc59a83-a207-4041-a63f-91a6903dc9d3","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:44:37.132677Z","iopub.execute_input":"2022-10-26T19:44:37.133766Z","iopub.status.idle":"2022-10-26T19:44:37.185757Z","shell.execute_reply.started":"2022-10-26T19:44:37.133722Z","shell.execute_reply":"2022-10-26T19:44:37.184647Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.head()","metadata":{"_uuid":"1b671806-e985-45a8-b5c8-de6659b005a8","_cell_guid":"bc5f2f19-8086-44b3-af8b-5a6b86b56cd5","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:44:41.766326Z","iopub.execute_input":"2022-10-26T19:44:41.767449Z","iopub.status.idle":"2022-10-26T19:44:41.779963Z","shell.execute_reply.started":"2022-10-26T19:44:41.767375Z","shell.execute_reply":"2022-10-26T19:44:41.778685Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(df['width'].unique())\nprint(df['StudyInstanceUID'].unique())\nprint(df['height'].unique())","metadata":{"_uuid":"a47800c2-f83c-4ec1-bee8-a2a5c99df7f8","_cell_guid":"eec6688f-83d1-4e7f-b293-063b36edb245","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:44:44.776562Z","iopub.execute_input":"2022-10-26T19:44:44.776984Z","iopub.status.idle":"2022-10-26T19:44:44.793886Z","shell.execute_reply.started":"2022-10-26T19:44:44.77694Z","shell.execute_reply":"2022-10-26T19:44:44.792542Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#check out missing values or NULL values\n\ndf.isnull().sum()","metadata":{"_uuid":"ac3b9391-af2b-4cfe-8871-444967e34202","_cell_guid":"9f69fb70-50c1-4ce2-b891-1114f256e24d","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:44:53.826908Z","iopub.execute_input":"2022-10-26T19:44:53.827299Z","iopub.status.idle":"2022-10-26T19:44:53.837734Z","shell.execute_reply.started":"2022-10-26T19:44:53.827266Z","shell.execute_reply":"2022-10-26T19:44:53.836766Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.describe()","metadata":{"_uuid":"6726d75c-80b0-41e3-b2dd-f5b7c6764125","_cell_guid":"7b221cee-e74f-41fe-8578-faba47c5a54e","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:44:56.750644Z","iopub.execute_input":"2022-10-26T19:44:56.751035Z","iopub.status.idle":"2022-10-26T19:44:56.786217Z","shell.execute_reply.started":"2022-10-26T19:44:56.751003Z","shell.execute_reply":"2022-10-26T19:44:56.785151Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.columns","metadata":{"_uuid":"ae17b19a-3078-4eae-9fa4-ca1950e80c2e","_cell_guid":"61b6f39d-c736-4d8b-afd4-b4d4a9357ddd","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:45:00.103888Z","iopub.execute_input":"2022-10-26T19:45:00.104276Z","iopub.status.idle":"2022-10-26T19:45:00.111236Z","shell.execute_reply.started":"2022-10-26T19:45:00.104243Z","shell.execute_reply":"2022-10-26T19:45:00.11031Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"final_dataset = df[['StudyInstanceUID', 'x', 'y', 'width', 'height', 'slice_number']]","metadata":{"_uuid":"1185eccc-8102-4489-855b-b8f232d649cf","_cell_guid":"c59a0071-d801-4ad1-906e-d7227067cff8","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:45:04.233119Z","iopub.execute_input":"2022-10-26T19:45:04.233939Z","iopub.status.idle":"2022-10-26T19:45:04.240697Z","shell.execute_reply.started":"2022-10-26T19:45:04.23389Z","shell.execute_reply":"2022-10-26T19:45:04.239668Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"final_dataset.head()","metadata":{"_uuid":"da96deef-dfba-42d3-9399-ee7d0ab1ea6b","_cell_guid":"27fe05f3-ed9b-41cd-91a5-777d7762d2fe","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:45:07.189358Z","iopub.execute_input":"2022-10-26T19:45:07.190102Z","iopub.status.idle":"2022-10-26T19:45:07.20402Z","shell.execute_reply.started":"2022-10-26T19:45:07.190051Z","shell.execute_reply":"2022-10-26T19:45:07.202638Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"final_dataset=pd.get_dummies(final_dataset,drop_first=True)","metadata":{"_uuid":"f812bd45-0ed5-47d3-aec0-c2b1bd5d80e9","_cell_guid":"c20aa68d-0e4c-4a27-bce3-5bf15e4ad0b0","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:45:09.094762Z","iopub.execute_input":"2022-10-26T19:45:09.095206Z","iopub.status.idle":"2022-10-26T19:45:09.120103Z","shell.execute_reply.started":"2022-10-26T19:45:09.095169Z","shell.execute_reply":"2022-10-26T19:45:09.118934Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"final_dataset.head()","metadata":{"_uuid":"0bbfbe88-5c01-4b9c-85bf-afc3d951e284","_cell_guid":"941f2fe8-3b7e-4003-a27b-71420a57aead","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:45:12.032462Z","iopub.execute_input":"2022-10-26T19:45:12.03289Z","iopub.status.idle":"2022-10-26T19:45:12.05405Z","shell.execute_reply.started":"2022-10-26T19:45:12.032856Z","shell.execute_reply":"2022-10-26T19:45:12.052866Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Data In the form of Correlation","metadata":{}},{"cell_type":"code","source":"final_dataset.corr()","metadata":{"_uuid":"69ef8946-d8bc-4238-87e7-d9038eb624f3","_cell_guid":"0f2375c1-98c1-4acb-96d1-724fae6c0a5e","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:45:14.328169Z","iopub.execute_input":"2022-10-26T19:45:14.328635Z","iopub.status.idle":"2022-10-26T19:45:15.470646Z","shell.execute_reply.started":"2022-10-26T19:45:14.328585Z","shell.execute_reply":"2022-10-26T19:45:15.469488Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import seaborn as sns","metadata":{"_uuid":"b49ea3c7-9258-489d-8a93-6d53ae194f5f","_cell_guid":"784569db-a755-4bac-a83f-8ffc21ab0ca8","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:45:38.888916Z","iopub.execute_input":"2022-10-26T19:45:38.889334Z","iopub.status.idle":"2022-10-26T19:45:39.446158Z","shell.execute_reply.started":"2022-10-26T19:45:38.889298Z","shell.execute_reply":"2022-10-26T19:45:39.44503Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"corrmat = final_dataset.corr()\ntop_corr_feature = corrmat.index\nplt.figure(figsize=(20,20))\n\ng=sns.heatmap(final_dataset[top_corr_feature].corr(),annot=True,cmap=\"RdYlGn\")","metadata":{"_uuid":"5b284317-7cbd-45b3-b5d6-4158ac9e9293","_cell_guid":"c4c21a01-531a-4aae-9dc9-3be372e0075b","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:45:42.287989Z","iopub.execute_input":"2022-10-26T19:45:42.288371Z","iopub.status.idle":"2022-10-26T19:50:09.835706Z","shell.execute_reply.started":"2022-10-26T19:45:42.288339Z","shell.execute_reply":"2022-10-26T19:50:09.834771Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Derived Heat Map of the overall CSV File using **PANDAS**","metadata":{"_uuid":"403ca183-3966-4736-a303-e1eecfcd6e78","_cell_guid":"e922dcd2-ffb2-44b4-ab6a-8a322fa70278","trusted":true}},{"cell_type":"code","source":"def extract_csv_gen_plot(csv_path):\n\n    data = pd.read_csv(csv_path, index_col=1)\n    data = data.drop(data.columns[[0, 1]], axis=1)\n    data.index.names = ['Name']\n    g = sns.heatmap(data)\n    g.set_yticklabels(g.get_yticklabels(), rotation=0)\n    g.set_title('Heatmap')\n    plt.tight_layout()\n    plt.show()\n\n\nextract_csv_gen_plot(\"../input/rsna-2022-cervical-spine-fracture-detection/train_bounding_boxes.csv\")","metadata":{"_uuid":"ad5dbd7c-8fec-42fa-a191-d1b253d11382","_cell_guid":"f842be36-424a-4570-8ad1-e579c974e485","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:50:24.291517Z","iopub.execute_input":"2022-10-26T19:50:24.291912Z","iopub.status.idle":"2022-10-26T19:50:24.755747Z","shell.execute_reply.started":"2022-10-26T19:50:24.291882Z","shell.execute_reply":"2022-10-26T19:50:24.754633Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x = final_dataset.iloc[:,1:]\ny = final_dataset.iloc[:,0]","metadata":{"_uuid":"583ade4e-6421-4e7f-88d0-d25ac4d0a56f","_cell_guid":"9f26aa81-72a4-4520-8788-88ca3566ac4a","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:50:29.4716Z","iopub.execute_input":"2022-10-26T19:50:29.472004Z","iopub.status.idle":"2022-10-26T19:50:29.479215Z","shell.execute_reply.started":"2022-10-26T19:50:29.471969Z","shell.execute_reply":"2022-10-26T19:50:29.478016Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x.head()","metadata":{"_uuid":"9756c501-69cc-44f2-9ce6-e007441d17fc","_cell_guid":"9444f965-dc13-4c7b-9265-2fb610236c30","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:50:30.769834Z","iopub.execute_input":"2022-10-26T19:50:30.770422Z","iopub.status.idle":"2022-10-26T19:50:30.788881Z","shell.execute_reply.started":"2022-10-26T19:50:30.770388Z","shell.execute_reply":"2022-10-26T19:50:30.78813Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y.head()","metadata":{"_uuid":"6629c671-c80f-47f1-b88b-69ad5115ccd4","_cell_guid":"57c8637d-64c7-445f-86dd-fdf20971d7f3","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:50:33.12217Z","iopub.execute_input":"2022-10-26T19:50:33.122763Z","iopub.status.idle":"2022-10-26T19:50:33.128787Z","shell.execute_reply.started":"2022-10-26T19:50:33.122728Z","shell.execute_reply":"2022-10-26T19:50:33.128066Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Feature Importance\nfrom sklearn.ensemble import ExtraTreesRegressor\nmodel = ExtraTreesRegressor()\nmodel.fit(x,y)","metadata":{"_uuid":"b9470a8f-9e69-4fed-833e-4e5e35a4b50a","_cell_guid":"d5ccf84f-dc80-49ba-af3f-47965c418f49","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:50:34.809451Z","iopub.execute_input":"2022-10-26T19:50:34.810049Z","iopub.status.idle":"2022-10-26T19:50:52.540789Z","shell.execute_reply.started":"2022-10-26T19:50:34.810014Z","shell.execute_reply":"2022-10-26T19:50:52.539501Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(model.feature_importances_)","metadata":{"_uuid":"2cb2cf93-df06-440c-a259-24f8a79390b6","_cell_guid":"47245f0b-94f0-420a-aa39-654fb4688d03","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:50:52.543734Z","iopub.execute_input":"2022-10-26T19:50:52.544433Z","iopub.status.idle":"2022-10-26T19:50:52.585959Z","shell.execute_reply.started":"2022-10-26T19:50:52.544397Z","shell.execute_reply":"2022-10-26T19:50:52.58472Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Data Visualization","metadata":{"_uuid":"db203064-8fe1-488f-97a8-299e5c141d58","_cell_guid":"c51a2e9d-9136-406c-acf1-650df43dcced","trusted":true}},{"cell_type":"code","source":"from sklearn.model_selection import RandomizedSearchCV\n #Randomized Search CV\n\n# Number of trees in random forest\nn_estimators = [int(x) for x in np.linspace(start = 100, stop = 1200, num = 12)]\n# Number of features to consider at every split\nmax_features = ['auto', 'sqrt']\n# Maximum number of levels in tree\nmax_depth = [int(x) for x in np.linspace(5, 30, num = 6)]\n# max_depth.append(None)\n# Minimum number of samples required to split a node\nmin_samples_split = [2, 5, 10, 15, 100]\n# Minimum number of samples required at each leaf node\nmin_samples_leaf = [1, 2, 5, 10]","metadata":{"_uuid":"750900e8-624e-49ae-a39b-4e2b25fdb79b","_cell_guid":"5b80954a-7ff7-416b-8ffc-aa690b453c6b","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:51:14.933749Z","iopub.execute_input":"2022-10-26T19:51:14.934202Z","iopub.status.idle":"2022-10-26T19:51:14.941673Z","shell.execute_reply.started":"2022-10-26T19:51:14.934155Z","shell.execute_reply":"2022-10-26T19:51:14.940216Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create the random grid\nrandom_grid = {'n_estimators': n_estimators,\n               'max_features': max_features,\n               'max_depth': max_depth,\n               'min_samples_split': min_samples_split,\n               'min_samples_leaf': min_samples_leaf}\n\nprint(random_grid)","metadata":{"_uuid":"62e30a7a-eef2-4f46-b06a-69195fd2badb","_cell_guid":"525cfc55-461f-4b72-99d2-75f7cdbdc0c1","collapsed":false,"jupyter":{"outputs_hidden":false},"execution":{"iopub.status.busy":"2022-10-26T19:51:24.91377Z","iopub.execute_input":"2022-10-26T19:51:24.914178Z","iopub.status.idle":"2022-10-26T19:51:24.920714Z","shell.execute_reply.started":"2022-10-26T19:51:24.914143Z","shell.execute_reply":"2022-10-26T19:51:24.91943Z"},"trusted":true},"execution_count":null,"outputs":[]}]}