End to End Machine Learning Projects

Date: 13-Dec-24

END TO END ML PROJECTS - KRISH NAIK YOUTUBE CHANNEL

https://www.youtube.com/watch?v=Rv6UFGNmNZg&list=PLZoTAELRMXVPS-dOaVbAux22vzqdgoGhG&index=2

I. Set up project with GitHub

1. Data Ingestion
2. Data Transformation
3. Model Trainer
4. Model Evaluation
5. Model Deployment

II. CI/CD Pipelines - GitHub Actions

III. Deployment AWS

Tutorial 1: GitHub & Code Set up

1. Set up the GitHub repository

a) new environment

Create new venv in VScode terminal

conda create -p venv python==3.12 -y

conda activate venv/

git init
git add README.md (first create README.md file in VScode)
git commit -m "first commit"

Add this for first time:
git config --global user.email "[email protected]"
git config --global user.name "Akshay Kumar"

git branch -M main
git remote add origin https://github.com/akshaykr7/mlproject.git
git remote -v (optional)
git push -u origin main
create .gitignore file in GitHub webpage – create new file and select python
git pull (to merge changes done in GitHub webpage)

b) setup.py - create ML application as a package and send to Pypi

Whenever setup.py will run it will search for folder which have init.py and consider that package as a package.

c) requirements.txt

-e . -> map to the setup.py and it will run

d) src folder and build the packages

pip install -r requirements.txt

Tutorial 2: Project Structure, Logging And Exception Handling

1. Create components folder in src folder.

   To create different modules like data ingestion, transformation

2. Create pipeline folder – training and prediction pipeline

3. Create logger.py, exception.py, utils.py

exception.py -> create custom error class

logger.py -> logs every execution into the text file

Tutorial 3: Project Statement, EDA and Model Training

ML Basics - This part will be used in modular coding (i.e. data_ingestion.py, etc...) for deployement.

1. Project - Student data link
2. EDA - Explorator Data Analysis
3. Model training steps

Tutorial 4: Data Ingestion Implementation

Load data, train test split, save data

Tutorial 5: Data Transformation

data preprocessing - Onehotencoder, StandardScaler

save preprocessor.pkl file

Tutorial 6: Model Training

Model training and evaluation

Run data_ingestion.py file to create .pkl file in artifacts folder

save model.pkl file -> It has best model from src/components/model_trainer.py

Tutorial 7: Model hyperparameter tuning

GridSearchcv

Tutorial 8: Create Prediction Pipeline Using Flask Web APP

1. create app.py
2. create templates folder > index.html file inside it (for UI part)
3. create home.html file under templates -> remember under form action, predict_datapoint function should be same as in app.py file.
4. src/pipeline/predict_pipeline.py

Tutorial 9: Deployment In AWS Cloud Using CICD Pipelines

Tutorial 10:

Tutorial 11: