Skip to content
Public template

About

No description, website, or topics provided.

Resources

Stars

4 stars

Watchers

0 watching

Forks

Latest commit

Β 

History

43 Commits

Folders and files

Repository files navigation

🧠 ML Model Training with Google Colab

πŸš€ Train your custom image dataset using Google Colab GPU and save the trained model directly to Google Drive.


πŸ“Œ Overview

This guide explains how to train an ML image-classification model using your own dataset.

The workflow is:

        πŸ“‚ Custom Dataset
               β”‚
               β–Ό
        ☁️ Google Drive
               β”‚
               β–Ό
        πŸ”— Google Colab
               β”‚
               β–Ό
        🧹 Preprocessing
               β”‚
               β–Ό
        βœ‚οΈ Train / Validation Split
               β”‚
               β–Ό
        🧠 ResNet18 Training
               β”‚
               β–Ό
        πŸ“Š Model Evaluation
               β”‚
               β–Ό
        πŸ’Ύ Trained Model
               β”‚
               β–Ό
        πŸš€ Ready for Deployment

⚑ 1. Open Google Colab

Create a new notebook in Google Colab.

Enable GPU

Go to:

Runtime β†’ Change runtime type β†’ T4 GPU β†’ Save

Then verify that the GPU is available:

!nvidia-smi

If the GPU is enabled, Colab will display information about the available NVIDIA GPU.


πŸ“‚ 2. Prepare Your Dataset

Upload your dataset to Google Drive.

The dataset should follow this structure:

πŸ“ MyDrive
└── πŸ“ dataset
    β”œβ”€β”€ πŸ“ happy
    β”‚   β”œβ”€β”€ πŸ–ΌοΈ image1.jpg
    β”‚   β”œβ”€β”€ πŸ–ΌοΈ image2.jpg
    β”‚   └── ...
    β”‚
    β”œβ”€β”€ πŸ“ neutral
    β”‚   β”œβ”€β”€ πŸ–ΌοΈ image1.jpg
    β”‚   β”œβ”€β”€ πŸ–ΌοΈ image2.jpg
    β”‚   └── ...
    β”‚
    └── πŸ“ tired
        β”œβ”€β”€ πŸ–ΌοΈ image1.jpg
        β”œβ”€β”€ πŸ–ΌοΈ image2.jpg
        └── ...

πŸ’‘ Important

Each folder represents one class.

For example:

Folder Class
happy 😊 Happy
neutral 😐 Neutral
tired 😴 Tired

You can replace these classes with your own classes.

For example:

dataset/
β”œβ”€β”€ cat/
β”œβ”€β”€ dog/
└── horse/

or:

dataset/
β”œβ”€β”€ healthy/
β”œβ”€β”€ diseased/
└── unknown/

πŸ”— 3. Connect Google Drive

Run this cell in Colab:

from google.colab import drive

drive.mount('/content/drive')

Google Colab will ask for permission.

After authorization, your Google Drive will be available inside:

/content/drive/MyDrive/

πŸ“ 4. Set Dataset Path

Set the location of your dataset:

DATASET_PATH = "/content/drive/MyDrive/dataset"

Check whether the dataset is detected correctly:

import os

print(os.listdir(DATASET_PATH))

Expected output:

['happy', 'neutral', 'tired']

If you see your class folders, your dataset is ready. βœ…


πŸ“¦ 5. Install Required Libraries

Run:

!pip install torch torchvision scikit-learn matplotlib

These libraries are used for:

Library Purpose
PyTorch 🧠 Model training
Torchvision πŸ–ΌοΈ Image datasets & models
Scikit-learn πŸ“Š Evaluation
Matplotlib πŸ“ˆ Visualization

πŸ–ΌοΈ 6. Load the Dataset

Use ImageFolder to automatically detect the class folders.

from torchvision import datasets, transforms

transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor()
])

dataset = datasets.ImageFolder(
    DATASET_PATH,
    transform=transform
)

print("Classes:", dataset.classes)
print("Total images:", len(dataset))

Example output:

Classes: ['happy', 'neutral', 'tired']
Total images: 3000

βœ‚οΈ 7. Split the Dataset

The dataset needs to be divided into training and validation data.

from torch.utils.data import random_split

train_size = int(0.8 * len(dataset))
val_size = len(dataset) - train_size

train_dataset, val_dataset = random_split(
    dataset,
    [train_size, val_size]
)

print("Training images:", len(train_dataset))
print("Validation images:", len(val_dataset))

πŸ“Š Dataset Split

         πŸ“‚ Dataset
             β”‚
       β”Œβ”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”
       β–Ό           β–Ό
   🟒 80%       πŸ”΅ 20%
   Training     Validation

🚚 8. Create DataLoaders

from torch.utils.data import DataLoader

train_loader = DataLoader(
    train_dataset,
    batch_size=32,
    shuffle=True
)

val_loader = DataLoader(
    val_dataset,
    batch_size=32,
    shuffle=False
)

Why DataLoader?

It loads images in batches instead of loading the entire dataset into memory at once.

Dataset
   ↓
Batch 1 β†’ 32 images
Batch 2 β†’ 32 images
Batch 3 β†’ 32 images
   ↓
Model

🧠 9. Create the Model

We use ResNet18 with transfer learning.

Instead of training an entire neural network from scratch, we start with a model that has already learned useful image features.

import torch
import torch.nn as nn
from torchvision import models

device = torch.device(
    "cuda" if torch.cuda.is_available() else "cpu"
)

model = models.resnet18(weights="DEFAULT")

num_classes = len(dataset.classes)

model.fc = nn.Linear(
    model.fc.in_features,
    num_classes
)

model = model.to(device)

print("Using device:", device)

Expected:

Using device: cuda

if GPU is enabled.


βš™οΈ 10. Define Loss & Optimizer

criterion = nn.CrossEntropyLoss()

optimizer = torch.optim.Adam(
    model.parameters(),
    lr=0.001
)

Components

Loss Function
     ↓
Measures how wrong the prediction is

Optimizer
     ↓
Updates the model weights

Learning Rate
     ↓
Controls how quickly the model learns

πŸš€ 11. Train the Model

Set the number of training epochs:

epochs = 10

Then train:

for epoch in range(epochs):

    model.train()

    for images, labels in train_loader:

        images = images.to(device)
        labels = labels.to(device)

        optimizer.zero_grad()

        outputs = model(images)

        loss = criterion(outputs, labels)

        loss.backward()

        optimizer.step()

    print(
        f"Epoch {epoch + 1}/{epochs} "
        f"| Loss: {loss.item():.4f}"
    )

Example:

Epoch 1/10 | Loss: 0.8421
Epoch 2/10 | Loss: 0.5214
Epoch 3/10 | Loss: 0.3187
...
Epoch 10/10 | Loss: 0.1023

As training progresses, the loss should generally decrease.


πŸ“Š 12. Evaluate the Model

After training, evaluate the model using the validation dataset.

model.eval()

correct = 0
total = 0

with torch.no_grad():

    for images, labels in val_loader:

        images = images.to(device)
        labels = labels.to(device)

        outputs = model(images)

        _, predicted = torch.max(outputs, 1)

        total += labels.size(0)
        correct += (predicted == labels).sum().item()

accuracy = 100 * correct / total

print(f"Validation Accuracy: {accuracy:.2f}%")

Example:

Validation Accuracy: 91.35%

⚠️ Accuracy depends entirely on your dataset and should not be treated as guaranteed.


πŸ’Ύ 13. Save the Trained Model

Save the trained model directly to Google Drive:

MODEL_PATH = "/content/drive/MyDrive/best_model.pth"

torch.save(
    model.state_dict(),
    MODEL_PATH
)

print("βœ… Model saved:", MODEL_PATH)

Your trained model will now be available in:

πŸ“ MyDrive
└── best_model.pth

🏷️ 14. Save Class Names

The model also needs to know which output number corresponds to which class.

import json

CLASS_PATH = "/content/drive/MyDrive/class_names.json"

with open(CLASS_PATH, "w") as f:
    json.dump(dataset.classes, f)

print(dataset.classes)

Example:

[
    "happy",
    "neutral",
    "tired"
]

Now you have:

πŸ“ MyDrive
β”œβ”€β”€ 🧠 best_model.pth
└── 🏷️ class_names.json

πŸ”„ 15. Train a New Dataset

Want to train another dataset?

Simply replace the dataset:

πŸ“ dataset
β”œβ”€β”€ class_1
β”œβ”€β”€ class_2
└── class_3

Then change:

DATASET_PATH = "/content/drive/MyDrive/dataset"

Run the Colab cells again.

The new model will be trained using your new dataset.


πŸ§ͺ 16. Improve the Model

If the model does not perform well, try:

πŸ“ˆ More training images
      ↓
🧹 Better data quality
      ↓
πŸ”„ Data augmentation
      ↓
βš™οΈ Tune learning rate
      ↓
πŸ”’ Increase epochs
      ↓
🧠 Try another pretrained model

Most importantly:

Better and more diverse data usually matters more than simply increasing the number of epochs.


βœ… Final Workflow

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚      πŸ“‚ YOUR DATASET      β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
              ↓
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚     ☁️ GOOGLE DRIVE       β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
              ↓
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚     πŸ”— GOOGLE COLAB       β”‚
β”‚          + GPU            β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
              ↓
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚    πŸ–ΌοΈ PREPROCESS DATA     β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
              ↓
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚     βœ‚οΈ SPLIT DATASET      β”‚
β”‚       80% / 20%           β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
              ↓
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚      🧠 TRAIN MODEL       β”‚
β”‚        ResNet18           β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
              ↓
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚       πŸ“Š EVALUATE         β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
              ↓
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚      πŸ’Ύ SAVE MODEL        β”‚
β”‚    best_model.pth         β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

🎯 Quick Start

If your dataset is already prepared, the basic process is:

1️⃣ Upload dataset to Google Drive

2️⃣ Open Google Colab

3️⃣ Enable T4 GPU

4️⃣ Mount Google Drive

5️⃣ Set DATASET_PATH

6️⃣ Install libraries

7️⃣ Load dataset

8️⃣ Split dataset

9️⃣ Train ResNet18

πŸ”Ÿ Evaluate model

1️⃣1️⃣ Save best_model.pth

1️⃣2️⃣ Save class_names.json

πŸ† Result

Your Dataset
     ↓
Google Colab GPU
     ↓
Trained ML Model
     ↓
best_model.pth
     ↓
Ready to use in your application πŸš€

About

No description, website, or topics provided.

Resources

Stars

4 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages