LlamaIndex and llm: Evaluate gemma3:1b with Ollama

topic/machine-learning/llama-index/README.md

@@ -96,6 +96,22 @@ Save your changes.
 
 [LlamaIndex's NLSQLTableQueryEngine] is a natural language SQL table query engine.
 
+Configure either OpenAI GPT,
+```bash
+export OPENAI_API_KEY=<YOUR_OPENAI_API_KEY>
+export LLM_MODEL=gpt-4.1
+```
+... or Ollama, either on runpod,
+```bash
+export LLM_BACKEND=ollama
+export OLLAMA_BASE_URL=https://l59hg3k5z1bxtw-11434.proxy.runpod.net/
+export LLM_MODEL=gemma3:1b
+```
+or somewhere else.
+```bash
+export OLLAMA_BASE_URL=http://100.127.86.113:11434/
+```
+
 Run the code like so:
 ```bash
 python demo_nlsql.py

topic/machine-learning/llama-index/boot.py

@@ -48,6 +48,7 @@ def configure_llm(debug: bool = False) -> Tuple[LLM, BaseEmbedding]:
     elif llm_backend == "ollama":
         # https://docs.llamaindex.ai/en/stable/api_reference/llms/ollama/
         llm = Ollama(
+            base_url=os.getenv("OLLAMA_BASE_URL", ""),
             model=llm_model,
             temperature=0.0,
             request_timeout=120.0,

topic/machine-learning/llama-index/demo_nlsql.py

@@ -28,7 +28,10 @@ def main():
     engine_crate.connect()
 
     print("Creating LlamaIndex QueryEngine")
-    sql_database = SQLDatabase(engine_crate, include_tables=[os.getenv("CRATEDB_TABLE_NAME")])
+    sql_database = SQLDatabase(
+        engine_crate,
+        #include_tables=[os.getenv("CRATEDB_TABLE_NAME")],
+    )
     query_engine = NLSQLTableQueryEngine(
         sql_database=sql_database,
         # tables=[os.getenv("CRATEDB_TABLE_NAME")],

topic/machine-learning/llama-index/env.standalone

@@ -1,11 +1,15 @@
 # Backend: Ollama
 # export LLM_BACKEND=ollama
+# export OLLAMA_BASE_URL=https://l59hg3k5z1bxtw-11434.proxy.runpod.net/
+# export OLLAMA_BASE_URL=http://100.127.86.113:11434/
 # export LLM_MODEL=phi4-mini:latest
 # export LLM_MODEL=phi4-mini:3.8b-fp16
 # export LLM_MODEL=deepseek-r1:7b
 # export LLM_MODEL=llama3.2:latest
 # export LLM_MODEL=qwen3:0.6b
 # export LLM_MODEL=gemma3:1b
+# export LLM_MODEL=sqlcoder:7b
+# export LLM_MODEL=duckdb-nsql:7b
 
 # Backend: OpenAI
 export LLM_BACKEND=openai

topic/machine-learning/llm/requirements.txt

@@ -1,2 +1,3 @@
 llm<0.27
 llm-anthropic<0.18
+llm-ollama<0.14

topic/machine-learning/llm/sql-request.txt

@@ -0,0 +1,15 @@
+Given an input question, first create a syntactically correct CrateDB query to run,
+then look at the results of the query and return the answer. You can order the
+results by a relevant column to return the most interesting examples in the database.
+
+Never query for all the columns from a specific table, only ask for a few relevant
+columns given the question.
+
+Pay attention to use only the column names that you can see in the schema description.
+Be careful to not query for columns that do not exist. Pay attention to which column
+is in which table. Also, qualify column names with the table name when needed. You
+are required to use the following format, each taking one line:
+
+Question: Question here
+SQLQuery: SQL Query to run
+Only use tables listed below.

topic/machine-learning/llm/sql-response.txt

@@ -0,0 +1,2 @@
+Given an input question, synthesize a response from the query results.
+Query:

topic/machine-learning/llm/test.xsh

@@ -10,7 +10,6 @@ $XONSH_SHOW_TRACEBACK = True
 # OpenAI GPT
 # ----------
 response=$(
-
   llm prompt --model gpt-4.1 --option temperature 0.0 --no-stream \
     --fragment https://cdn.crate.io/about/v1/llms-full.txt \
     --system-fragment https://cdn.crate.io/about/v1/instructions-general.md \
@@ -24,11 +23,26 @@ assert "UUIDv7" in response, "GPT: UUIDv7 was not in response"
 # Anthropic Claude Sonnet
 # -----------------------
 response=$(
-
   llm prompt --model claude-4-sonnet --option temperature 0.0 --no-stream \
     --fragment https://cdn.crate.io/about/v1/llms-full.txt \
     --system-fragment https://cdn.crate.io/about/v1/instructions-general.md \
     "CrateDB does not seem to provide an AUTOINCREMENT feature?"
 )
 print(response)
 assert "UUIDv7" in response, "Sonnet: UUIDv7 was not in response"
+
+
+# ------------
+# Google Gemma
+# ------------
+# - Gemma3 works well for basic Text-to-SQL.
+if False:
+    $OLLAMA_HOST="http://100.127.86.113:11434/"
+    response=$(
+      llm prompt --model gemma3:1b --option temperature 0.0 --no-stream \
+        --fragment sql-request.txt \
+        --system "Table 'time_series_data' has columns: timestamp (TIMESTAMP), value (DOUBLE PRECISION), location (VARCHAR), sensor_id (INTEGER)" \
+        "Question: What is the average value for sensor 1?"
+    )
+    print(response)
+    assert "SQLQuery: SELECT AVG(value) FROM time_series_data WHERE sensor_id = 1" in response