Skip to content

data-format.md

Latest commit

 

History

History
188 lines (138 loc) · 5.39 KB

data-format.md

File metadata and controls

188 lines (138 loc) · 5.39 KB
Raw

Data Format

Teich normalizes supported sources into structured training examples.

To write standalone OpenAI-style training JSONL from raw or extracted traces, run:

teich convert data --out teich-training.jsonl

Core fields:

  • prompt: initial task description
  • follow_up_prompts: optional additional user turns after the initial prompt
  • messages: chat history
  • tools: tool schemas available to the session, including tools that were not called
  • metadata: session info, model, timestamps, usage, and provenance when available

Messages

messages follows an OpenAI-style chat shape:

[
  {"role": "user", "content": "Build a todo app"},
  {
    "role": "assistant",
    "content": "I will inspect the project.",
    "tool_calls": [
      {
        "id": "call_1",
        "type": "function",
        "function": {
          "name": "Read",
          "arguments": {"file_path": "README.md"}
        }
      }
    ]
  },
  {"role": "tool", "tool_call_id": "call_1", "name": "Read", "content": "..."}
]

Supported roles include:

  • system
  • developer
  • user
  • assistant
  • tool

Assistant messages can include:

  • content: text response
  • reasoning_content: reasoning traces when the source provides them
  • tool_calls: function calls with arguments

Some providers split a single model turn across multiple native events. Teich normalizes those fragments so semantic order is:

  1. reasoning_content
  2. optional assistant content
  3. tool_calls

Reasoning that arrives after assistant text or a tool-call fragment is moved back in front of the output it explains.

Tools

tools contains function schemas available to the session:

[
  {
    "type": "function",
    "function": {
      "name": "Read",
      "description": "Read a file.",
      "parameters": {
        "type": "object",
        "properties": {
          "file_path": {"type": "string"}
        },
        "required": ["file_path"],
        "additionalProperties": true
      }
    }
  }
]

Teich preserves configured tool snapshots where the source provides them. This matters because a training row can need the tool schema even when the model did not call that tool.

Tool schema sources include:

  • configured tools embedded in generated traces
  • generated dataset README.md snapshots
  • tools.json snapshots
  • native provider tool declarations
  • fallback inference from observed tool calls when no explicit schema exists

Metadata

Common metadata keys:

  • source_file
  • source_line
  • session_id
  • trace_type
  • model_provider
  • model
  • cwd
  • cli_version
  • turn_count
  • usage
  • total_cost_usd
  • first_message_timestamp

When the source format exposes per-message timestamps, converted rows include metadata.first_message_timestamp from the first timestamp-bearing source event that becomes a user message. It is not synthesized from session-start metadata.

Native Claude Code Context

Native Claude Code and Claude Desktop traces can contain runtime context that the model saw but that is not ordinary user text.

Teich preserves this as masked system messages and mirrors it into metadata.system_prompt.

Examples include:

  • Claude Desktop skill listings
  • MCP instruction deltas
  • deferred tool declarations
  • command permission context
  • date changes
  • hook context
  • away summaries
  • session recaps

Local slash-command artifacts such as /model are filtered. /goal contributes the actual user goal text. Queued prompts become real user turns.

Advertised native Claude Code / Claude Desktop tools receive schemas even when a tool is only declared through deferred-tool context.

Structured Chat Rows

The chat provider writes structured training rows directly instead of raw traces.

Single-turn rows can include:

  • messages
  • prompt
  • thinking
  • response
  • model

Multi-turn follow-up rows can also include:

  • follow_up_prompts
  • responses
  • final response

system is prompt-specific when provided. If a prompt row does not include system, Teich does not inject a default system prompt.

Incomplete Traces

Rows ending on a tool result are incomplete without a follow-up assistant turn.

load_traces() drops those rows by default. Pass drop_incomplete_traces=False only when you intentionally want to inspect or repair them.

Generated Dataset Cards

Generated datasets include a compact README.md with:

  • Teich attribution
  • row or file counts
  • provider metadata when available; generated-run cards may also include model metadata
  • a bounded example row
  • links to the maintained training and data-preparation docs
  • extraction guidance when the dataset came from teich extract
  • a tool-schema summary when tools are available

Small tool snapshots are embedded in the dataset card. Large snapshots are written to tools.json so Hugging Face dataset-card validation does not have to process a massive YAML/Markdown payload.

The generated card intentionally avoids trainer-specific code blocks. Training APIs, model defaults, and library setup change over time; the maintained guidance lives in Training and Preparing Data.

When you want a trainer-ready JSONL file without relying on Teich formatting or masking at training time, run:

teich convert data --out teich-training.jsonl

Generated dataset guidance is produced by src/teich/trace_readme.py, so behavior changes should update both top-level docs and that template.