Merge branch 'complete-refactor' of github.com:honeyhiveai/python-sdk into complete-refactor

# Conflicts:
#	CHANGELOG.md

Author: josh-hhai

CHANGELOG.md

@@ -21,6 +21,13 @@
   - Organized dependencies by category: core, LLM providers, instrumentors, agent frameworks
   - Included installation commands for each specific integration
   - Documentation of required environment variables per provider
+- **📚 Examples: Evaluation Example**
+  - Added simple evaluation example demonstrating the `evaluate()` function
+  - Shows basic dataset evaluation with span enrichment
+  - Includes ground truth comparisons
+- **📚 Examples: Legacy SDK Example**
+  - Added old SDK integration example for reference
+  - Demonstrates basic tracer initialization and OpenAI integration
 
 ### Fixed
 - **🐛 CRITICAL: Fixed evaluate() + enrich_span() Pattern (v1.0 Baggage Fix)**
@@ -34,6 +41,11 @@
   - Added 5 unit tests for selective propagation
   - Added integration test for `evaluate()` + enrichment pattern
 
+- **🔧 Experiments: Session Enrichment Always Runs**
+  - Fixed `evaluate()` function to enrich sessions even when no evaluators are provided
+  - Sessions now get outputs enriched regardless of evaluator presence
+  - Changed session enrichment log level from debug to info for better visibility
+  - Ensures session data is always persisted to backend
 - **🔧 Tracing: Restored enrich_session() Backwards Compatibility**
   - Fixed breaking signature changes in `enrich_session()` that removed `session_id` and `user_properties` parameters
   - Restored `session_id` as optional positional parameter for backwards compatibility

docs/changelog.rst

@@ -38,6 +38,17 @@ Current Version Highlights
 * **Installation Commands**: Per-integration pip install commands for easy setup
 * **Environment Variables**: Documentation of required credentials for each provider
 
+**📚 ADDED: New Example Files**
+
+* **Evaluation Example**: Simple demonstration of the ``evaluate()`` function with dataset evaluation and span enrichment
+* **Legacy SDK Example**: Reference example showing basic tracer initialization and OpenAI integration
+
+**🔧 FIXED: Session Enrichment in evaluate() Function**
+
+* **Always Enriches Sessions**: Fixed bug where sessions weren't enriched when no evaluators were provided
+* **Output Persistence**: Ensures outputs are always saved to backend regardless of evaluator presence
+* **Better Logging**: Upgraded log level from debug to info for session enrichment visibility
+
 **🔧 IMPROVED: Tracer Internal Cleanup**
 
 * **Code Simplification**: Removed redundant experiment baggage code path

docs/reference/index.rst

@@ -151,6 +151,9 @@ Experiments Module
 
 **Modern evaluation framework** with decorator-based evaluators and backend-powered aggregation.
 
+.. note::
+   **Session Enrichment**: The ``evaluate()`` function always enriches sessions with outputs, regardless of whether evaluators are provided. This ensures all execution results are persisted to the backend for later analysis.
+
 .. toctree::
    :maxdepth: 1
 
@@ -417,6 +420,14 @@ Package Information
 
    pip install honeyhive
 
+**Example Files**:
+
+The SDK includes example files in the ``examples/`` directory:
+
+- ``eval_example.py`` - Demonstrates the ``evaluate()`` function with dataset evaluation and span enrichment
+- ``integrations/old_sdk.py`` - Legacy SDK example showing basic tracer initialization and OpenAI integration
+- ``integrations/`` - Full integration examples for various LLM providers and frameworks
+
 **See Also:**
 
 - :doc:`../tutorials/index` - Learn by doing

examples/eval_example.py

@@ -0,0 +1,63 @@
+from honeyhive import HoneyHive
+from honeyhive.experiments import evaluate
+import os
+from dotenv import load_dotenv
+from datetime import datetime
+from honeyhive.api import DatasetsAPI, DatapointsAPI, MetricsAPI
+from pydantic import BaseModel
+from uuid import uuid4
+from honeyhive.models.generated import ReturnType
+from honeyhive.models import (
+    CreateDatapointRequest,
+    CreateDatasetRequest,
+    Metric,
+)
+from honeyhive import enrich_span
+
+load_dotenv()
+
+DATASET_NAME = "sample-honeyhive-9-30-25"
+
+
+def invoke_summary_agent(**kwargs):
+    return "The American Shorthair is a pedigreed cat breed, originally known as the Domestic Shorthair, that was among the first CFA-registered breeds in 1906 and was renamed in 1966 to distinguish it from random-bred domestic short-haired cats while highlighting its American origins."
+
+dataset = [
+    {
+        "inputs": {
+            "context": "The Poodle, called the Pudel in German and the Caniche in French, is a breed of water dog. The breed is divided into four varieties based on size, the Standard Poodle, Medium Poodle, Miniature Poodle and Toy Poodle, although the Medium Poodle is not universally recognised. They have a distinctive thick, curly coat that comes in many colours and patterns, with only solid colours recognised by major breed registries. Poodles are active and intelligent, and are particularly able to learn from humans. Poodles tend to live 10–18 years, with smaller varieties tending to live longer than larger ones."
+        },
+        "ground_truths": {
+            "result": "The Poodle is an intelligent water dog breed that comes in four size varieties with a distinctive curly coat, known for its trainability and relatively long lifespan of 10-18 years."
+        },
+    },
+    {
+        "inputs": {
+            "context": 'The American Shorthair is a pedigree cat breed, with a strict conformation standard, as set by cat fanciers of the breed and North American cat fancier associations such as The International Cat Association (TICA) and the CFA. The breed is accepted by all North American cat registries. Originally known as the Domestic Shorthair, in 1966 the breed was renamed the American Shorthair to better represent its "all-American" origins and to differentiate it from other short-haired breeds. The name American Shorthair also reinforces the breed\'s pedigreed status as distinct from the random-bred non-pedigreed domestic short-haired cats in North America, which may nevertheless resemble the American Shorthair. Both the American Shorthair breed and the random-bred cats from which the breed is derived are sometimes called working cats because they were used for controlling rodent populations, on ships and farms. The American Shorthair (then referred to as the Domestic Shorthair) was among the first five breeds that were registered by the CFA in 1906.'
+        },
+        "ground_truths": {
+            "result": "The American Shorthair is a pedigreed cat breed, originally known as the Domestic Shorthair, that was among the first CFA-registered breeds in 1906 and was renamed in 1966 to distinguish it from random-bred domestic short-haired cats while highlighting its American origins."
+        },
+    },
+]
+
+
+if __name__ == "__main__":
+    def evaluation_function(datapoint):
+        inputs = datapoint.get("inputs", {})
+        context = inputs.get("context", "")
+        enrich_span(metrics={"input_length": len(context)})
+        return {
+            "answer": invoke_summary_agent(**{"context": context})
+        }
+
+    result = evaluate(
+        function=evaluation_function,
+        dataset=dataset,
+        api_key=os.environ["HH_API_KEY"],
+        project=os.environ["HH_PROJECT"],
+        name=f"{DATASET_NAME}-{datetime.now().isoformat()}",
+        verbose=True,  # Enable verbose to see output enrichment
+    )
+
+    print(result)

examples/integrations/old_sdk.py

@@ -0,0 +1,46 @@
+import os
+from flask import Flask, render_template, request, jsonify
+from openai import OpenAI
+from dotenv import load_dotenv
+from honeyhive import HoneyHiveTracer, trace, enrich_span
+load_dotenv()
+app = Flask(__name__)
+client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
+# Place the code below at the beginning of your application to initialize the tracer
+HoneyHiveTracer.init(
+    api_key=os.getenv("HH_API_KEY"),
+    project="sdk", # Your HoneyHive project name
+    source='dev', #Optional
+    session_name='Test Session', #Optional
+    server_url="https://api.staging.honeyhive.ai"
+)
+# Additionally, trace any function in your code using @trace / @atrace decorator
+@trace
+def call_openai(user_input):
+    client = OpenAI()
+     # Example: Add feedback data for HoneyHive evaluation
+    # if user_input.strip().lower() == "what is the capital of france?":
+    #     HoneyHiveTracer.add_feedback({
+    #         "ground_truth": "The capital of France is Paris.",
+    #         "keywords": ["Paris", "France", "capital"]
+    #     })
+    completion = client.chat.completions.create(
+        model='gpt-4o-mini',
+        messages=[{"role":"user","content": user_input}]
+    )
+    return completion.choices[0].message.content
+# @app.route("/")
+# def index():
+#     return render_template("index.html")
+# @app.route("/chat", methods=["POST"])
+# def chat():
+#     data = request.get_json()
+#     user_input = data.get("message", "")
+#     try:
+#         reply = call_openai(user_input)
+#         return jsonify({"reply": reply})
+#     except Exception as e:
+#         return jsonify({"error": str(e)}), 500
+# if __name__ == "__main__":
+#     app.run(debug=True)
+call_openai("hi")

src/honeyhive/experiments/core.py

@@ -426,9 +426,7 @@ def _enrich_session_with_results(
 
             if verbose:
                 enriched_fields = list(update_data.keys())
-                logger.debug(
-                    "Enriched session %s with: %s", session_id, enriched_fields
-                )
+                logger.info("Enriched session %s with: %s", session_id, enriched_fields)
     except Exception as e:
         logger.warning("Failed to enrich session %s: %s", session_id, str(e))
 
@@ -819,21 +817,21 @@ def evaluate(  # pylint: disable=too-many-locals,too-many-branches
                 "Evaluators complete: %d metrics collected", len(evaluator_metrics)
             )
 
-        # Enrich sessions with outputs and evaluator metrics
-        if verbose:
-            logger.info("Enriching sessions with outputs and evaluator metrics")
+    # Enrich sessions with outputs and evaluator metrics (always, not just when evaluators exist)
+    if verbose:
+        logger.info("Enriching sessions with outputs and evaluator metrics")
 
-        for result in execution_results:
-            session_id = result.get("session_id")
-            if session_id:
-                _enrich_session_with_results(
-                    session_id=session_id,
-                    datapoint_id=result.get("datapoint_id"),
-                    outputs=result.get("outputs"),
-                    evaluator_metrics=evaluator_metrics,
-                    client=client,
-                    verbose=verbose,
-                )
+    for result in execution_results:
+        session_id = result.get("session_id")
+        if session_id:
+            _enrich_session_with_results(
+                session_id=session_id,
+                datapoint_id=result.get("datapoint_id"),
+                outputs=result.get("outputs"),
+                evaluator_metrics=evaluator_metrics or {},
+                client=client,
+                verbose=verbose,
+            )
 
     # Step 6: Update run with results
     _update_run_with_results(