in-context-learning

Vespa sample applications - Categorize using an LLM

This is a set of scripts/installs to back up our presentation at:

• MLCon
• data science connect COLLIDE

For any questions, please register at the Vespa Slack and discuss in the general channel.

Setup

Install Ollama and run models like:

ollama run llama3.1

Use the quick start or Vespa getting started to deploy this - laptop example:

podman run --detach --name vespa --hostname vespa-container \  
  --publish 127.0.0.1:8080:8080 --publish 127.0.0.1:19071:19071 \
  vespaengine/vespa
  
vespa deploy app --wait 600

Use e.g. podman or docker to run the vespaengine/vespa image on a laptop.

Generate data

feed_examples.py converts the train data set to vespa feed format - feed this to the Vespa instance:

python3 feed_examples.py > samples.jsonl
vespa feed samples.jsonl

Evaluate data

categorize_group.py runs through the test set and classifies based on examples retrieved from Vespa.

See the inference function for how to set up queries and ranking profiles for the different options.

Example script output:

category	size	relevance	retrieved_label	predicted_label	label_text	text
3	10	13.75663952228003	get_physical_card	get_physical_card	card_arrival	How do I locate my card?
0	10	19.146904249529296	card_arrival	card_arrival	card_arrival	I still have not received my new card, I ordered over a week ago.

Other

Use the @timer_decorator to time execution time of the functions.