π Train your custom image dataset using Google Colab GPU and save the trained model directly to Google Drive.
This guide explains how to train an ML image-classification model using your own dataset.
The workflow is:
π Custom Dataset
β
βΌ
βοΈ Google Drive
β
βΌ
π Google Colab
β
βΌ
π§Ή Preprocessing
β
βΌ
βοΈ Train / Validation Split
β
βΌ
π§ ResNet18 Training
β
βΌ
π Model Evaluation
β
βΌ
πΎ Trained Model
β
βΌ
π Ready for Deployment
Create a new notebook in Google Colab.
Go to:
Runtime β Change runtime type β T4 GPU β Save
Then verify that the GPU is available:
!nvidia-smiIf the GPU is enabled, Colab will display information about the available NVIDIA GPU.
Upload your dataset to Google Drive.
The dataset should follow this structure:
π MyDrive
βββ π dataset
βββ π happy
β βββ πΌοΈ image1.jpg
β βββ πΌοΈ image2.jpg
β βββ ...
β
βββ π neutral
β βββ πΌοΈ image1.jpg
β βββ πΌοΈ image2.jpg
β βββ ...
β
βββ π tired
βββ πΌοΈ image1.jpg
βββ πΌοΈ image2.jpg
βββ ...
Each folder represents one class.
For example:
| Folder | Class |
|---|---|
happy |
π Happy |
neutral |
π Neutral |
tired |
π΄ Tired |
You can replace these classes with your own classes.
For example:
dataset/
βββ cat/
βββ dog/
βββ horse/
or:
dataset/
βββ healthy/
βββ diseased/
βββ unknown/
Run this cell in Colab:
from google.colab import drive
drive.mount('/content/drive')Google Colab will ask for permission.
After authorization, your Google Drive will be available inside:
/content/drive/MyDrive/
Set the location of your dataset:
DATASET_PATH = "/content/drive/MyDrive/dataset"Check whether the dataset is detected correctly:
import os
print(os.listdir(DATASET_PATH))Expected output:
['happy', 'neutral', 'tired']
If you see your class folders, your dataset is ready. β
Run:
!pip install torch torchvision scikit-learn matplotlibThese libraries are used for:
| Library | Purpose |
|---|---|
PyTorch |
π§ Model training |
Torchvision |
πΌοΈ Image datasets & models |
Scikit-learn |
π Evaluation |
Matplotlib |
π Visualization |
Use ImageFolder to automatically detect the class folders.
from torchvision import datasets, transforms
transform = transforms.Compose([
transforms.Resize((224, 224)),
transforms.ToTensor()
])
dataset = datasets.ImageFolder(
DATASET_PATH,
transform=transform
)
print("Classes:", dataset.classes)
print("Total images:", len(dataset))Example output:
Classes: ['happy', 'neutral', 'tired']
Total images: 3000
The dataset needs to be divided into training and validation data.
from torch.utils.data import random_split
train_size = int(0.8 * len(dataset))
val_size = len(dataset) - train_size
train_dataset, val_dataset = random_split(
dataset,
[train_size, val_size]
)
print("Training images:", len(train_dataset))
print("Validation images:", len(val_dataset)) π Dataset
β
βββββββ΄ββββββ
βΌ βΌ
π’ 80% π΅ 20%
Training Validation
from torch.utils.data import DataLoader
train_loader = DataLoader(
train_dataset,
batch_size=32,
shuffle=True
)
val_loader = DataLoader(
val_dataset,
batch_size=32,
shuffle=False
)It loads images in batches instead of loading the entire dataset into memory at once.
Dataset
β
Batch 1 β 32 images
Batch 2 β 32 images
Batch 3 β 32 images
β
Model
We use ResNet18 with transfer learning.
Instead of training an entire neural network from scratch, we start with a model that has already learned useful image features.
import torch
import torch.nn as nn
from torchvision import models
device = torch.device(
"cuda" if torch.cuda.is_available() else "cpu"
)
model = models.resnet18(weights="DEFAULT")
num_classes = len(dataset.classes)
model.fc = nn.Linear(
model.fc.in_features,
num_classes
)
model = model.to(device)
print("Using device:", device)Expected:
Using device: cuda
if GPU is enabled.
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(
model.parameters(),
lr=0.001
)Loss Function
β
Measures how wrong the prediction is
Optimizer
β
Updates the model weights
Learning Rate
β
Controls how quickly the model learns
Set the number of training epochs:
epochs = 10Then train:
for epoch in range(epochs):
model.train()
for images, labels in train_loader:
images = images.to(device)
labels = labels.to(device)
optimizer.zero_grad()
outputs = model(images)
loss = criterion(outputs, labels)
loss.backward()
optimizer.step()
print(
f"Epoch {epoch + 1}/{epochs} "
f"| Loss: {loss.item():.4f}"
)Example:
Epoch 1/10 | Loss: 0.8421
Epoch 2/10 | Loss: 0.5214
Epoch 3/10 | Loss: 0.3187
...
Epoch 10/10 | Loss: 0.1023
As training progresses, the loss should generally decrease.
After training, evaluate the model using the validation dataset.
model.eval()
correct = 0
total = 0
with torch.no_grad():
for images, labels in val_loader:
images = images.to(device)
labels = labels.to(device)
outputs = model(images)
_, predicted = torch.max(outputs, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
accuracy = 100 * correct / total
print(f"Validation Accuracy: {accuracy:.2f}%")Example:
Validation Accuracy: 91.35%
β οΈ Accuracy depends entirely on your dataset and should not be treated as guaranteed.
Save the trained model directly to Google Drive:
MODEL_PATH = "/content/drive/MyDrive/best_model.pth"
torch.save(
model.state_dict(),
MODEL_PATH
)
print("β
Model saved:", MODEL_PATH)Your trained model will now be available in:
π MyDrive
βββ best_model.pth
The model also needs to know which output number corresponds to which class.
import json
CLASS_PATH = "/content/drive/MyDrive/class_names.json"
with open(CLASS_PATH, "w") as f:
json.dump(dataset.classes, f)
print(dataset.classes)Example:
[
"happy",
"neutral",
"tired"
]Now you have:
π MyDrive
βββ π§ best_model.pth
βββ π·οΈ class_names.json
Want to train another dataset?
Simply replace the dataset:
π dataset
βββ class_1
βββ class_2
βββ class_3
Then change:
DATASET_PATH = "/content/drive/MyDrive/dataset"Run the Colab cells again.
The new model will be trained using your new dataset.
If the model does not perform well, try:
π More training images
β
π§Ή Better data quality
β
π Data augmentation
β
βοΈ Tune learning rate
β
π’ Increase epochs
β
π§ Try another pretrained model
Most importantly:
Better and more diverse data usually matters more than simply increasing the number of epochs.
βββββββββββββββββββββββββββββ
β π YOUR DATASET β
βββββββββββββββ¬ββββββββββββββ
β
βββββββββββββββββββββββββββββ
β βοΈ GOOGLE DRIVE β
βββββββββββββββ¬ββββββββββββββ
β
βββββββββββββββββββββββββββββ
β π GOOGLE COLAB β
β + GPU β
βββββββββββββββ¬ββββββββββββββ
β
βββββββββββββββββββββββββββββ
β πΌοΈ PREPROCESS DATA β
βββββββββββββββ¬ββββββββββββββ
β
βββββββββββββββββββββββββββββ
β βοΈ SPLIT DATASET β
β 80% / 20% β
βββββββββββββββ¬ββββββββββββββ
β
βββββββββββββββββββββββββββββ
β π§ TRAIN MODEL β
β ResNet18 β
βββββββββββββββ¬ββββββββββββββ
β
βββββββββββββββββββββββββββββ
β π EVALUATE β
βββββββββββββββ¬ββββββββββββββ
β
βββββββββββββββββββββββββββββ
β πΎ SAVE MODEL β
β best_model.pth β
βββββββββββββββββββββββββββββ
If your dataset is already prepared, the basic process is:
1οΈβ£ Upload dataset to Google Drive
2οΈβ£ Open Google Colab
3οΈβ£ Enable T4 GPU
4οΈβ£ Mount Google Drive
5οΈβ£ Set DATASET_PATH
6οΈβ£ Install libraries
7οΈβ£ Load dataset
8οΈβ£ Split dataset
9οΈβ£ Train ResNet18
π Evaluate model
1οΈβ£1οΈβ£ Save best_model.pth
1οΈβ£2οΈβ£ Save class_names.json
Your Dataset
β
Google Colab GPU
β
Trained ML Model
β
best_model.pth
β
Ready to use in your application π