Get the server and know how to use the GPU on nautilus

• ask zihao to have a local server
• know how to use the toolbox

Copy the github and Create a conda environment in your local server

git clone [email protected]:howard881010/multimodal.git
conda env create -n your-name --file environment.yml
conda activate your-name
poetry install
## Add dependencies interactively or through poetry add
## Examples:
poetry source add --priority=explicit pytorch-gpu-src https://download.pytorch.org/whl/<cuda_version>
poetry add --source pytorch-gpu-src torch

Prepare the dataset

• Get the processed data from MinIO
• Upload the dataset to your huggingface account (reference: upload_*.ipynb) and in alpaca format
• Example: climate

Finetune the model

• Use axolotl, (reference: finetune/*.yaml)
• Or try to use unsloth

Inference & Evaluation

• Use your finetune model for inference (reference: modelchat.py for choose your finetune model, and multimodal.py for the inference part and evaluation)

python src/multimodal.py climate 1 mistral7b 1 finetune