Skip to content

experimental: add databricks-genie-agent lifecycle skill suite (Genie Space → Genie Agent rename) + wire it to databricks-metric-views - #182

Closed
anhhchu wants to merge 12 commits into
databricks:mainfrom
anhhchu:anhhchu/semantic_genie
Closed

experimental: add databricks-genie-agent lifecycle skill suite (Genie Space → Genie Agent rename) + wire it to databricks-metric-views#182
anhhchu wants to merge 12 commits into
databricks:mainfrom
anhhchu:anhhchu/semantic_genie

Conversation

@anhhchu

@anhhchu anhhchu commented Jun 23, 2026 •
edited
Loading

Copy link
Copy Markdown

experimental: add databricks-genie-agent lifecycle skill suite + wire it to databricks-metric-views

Summary

Adds the experimental databricks-genie-agent skill as an orchestration hub for the full Genie Agent lifecycle, organized as a suite of dual-host lifecycle subskills, and establishes a bidirectional link between the Genie skill and the existing databricks-metric-views skill so governed semantics and Genie-agent methodology each live in exactly one place.

Everything here is experimental (experimental/), so it is not in the generated plugin bundle and carries no metaplugin/plugin.meta.json entry or routing row — consistent with the repo's experimental-skill policy.

Naming: "Genie Space" → "Genie Agent"

This branch renames the skill suite and all related identifiers from the "Genie Space" vocabulary to "Genie Agent", matching the current Databricks product naming. The rename is applied consistently across directories, filenames, skill names, references, and prose:

Old New
experimental/databricks-genie/ experimental/databricks-genie-agent/
create-genie-space/ create-genie-agent/
diagnose-genie-space/ diagnose-genie-agent/
optimize-genie-space/ optimize-genie-agent/
create-genie-space/references/space-design-guide.md create-genie-agent/references/agent-design-guide.md
references/spaces.md references/agents.md
databricks-metric-views/references/genie-integration.md references/genie-agent-integration.md

Prose and cross-links throughout the subskills, the metric-views references, and manifest.json were updated to match (e.g. "Genie Space" → "Genie Agent", "space" → "agent" where it refers to the product object). optimize-genie-query/ keeps its name (it was never "space"-qualified).

The subskills flow in databricks-genie-agent

databricks-genie-agent/SKILL.md is the orchestration hub. The lifecycle methodology is host-portable — only the mechanism differs (CLI / MCP vs the in-product Genie Code UI) — so the hub carries the canonical methodology and dispatches to one of four lifecycle subskills, each a self-contained SKILL.md (+ references/):

Subskill When it owns the work
create-genie-agent Drafting/bootstrapping a focused Agent from UC tables, views, and Metric Views (design-time)
diagnose-genie-agent Plan-only root-cause analysis of wrong SQL/answers, weak reports, or feedback signals (no edits)
optimize-genie-agent Approved iterative benchmark-driven quality tuning (one focused pass at a time)
optimize-genie-query Benchmark-query performance/cost triage via Query History + Query Profile

Typical flow: create → diagnose → optimize-genie-agent, with optimize-genie-query branching in for performance issues (and handing back to the quality skills if the SQL is semantically wrong).

Design notes on the flow:

  • Dual-host by construction. Each subskill opens with an Execution Context section (Genie Code native UI vs CLI/MCP) and a Mechanism Map table mapping every workflow step to its CLI/MCP substitute. UI-only surfaces are flagged explicitly.
  • Read-only / plan-first. All subskills inspect with bounded read-only SQL and gate any mutation behind explicit user approval.
  • Single source of truth for methodology. Sizing, the incremental build & validation loop, and anti-patterns live in create-genie-agent → agent-design-guide.md; benchmark integrity, repair/pruning, and regression gates live in optimize-genie-agent → optimization-guide.md. The hub's lifecycle table links to these rather than restating them, so there is no drift between the hub and the subskills.
  • Mechanism references stay at the hub. references/agents.md (verified serialized_space schema + CLI) and references/conversation.md (Conversation API / ask_genie) are owned by the parent because they're needed even when no subskill is active.

The link between databricks-genie-agent and databricks-metric-views

The two skills divide responsibility cleanly and cross-link in both directions, so neither duplicates the other:

  • Genie → Metric Views: databricks-genie-agent treats Metric Views as governed semantic sources and defers their design to databricks-metric-views. It links to genie-agent-integration.md for the design rules that affect Genie answer quality (one-fact-source rule, base-view pattern for multi-fact KPIs, agent metadata, domain organization) and to query-patterns.md for the MEASURE() query rules Genie must follow.
  • Metric Views → Genie: genie-agent-integration.md carries a Scope note pointing the build / size / validate / benchmark the Agent side back to the Genie lifecycle subskills (create-genie-agent → agent-design-guide.md and optimize-genie-agent → optimization-guide.md), so metric-view design guidance no longer restates Genie-agent methodology.

Net effect: metric-view design lives in databricks-metric-views; Genie-agent lifecycle methodology lives in databricks-genie-agent; each references the other instead of mirroring it.

Changes

  • New experimental/databricks-genie-agent/ skill: hub SKILL.md, four lifecycle subskills, and shared references (agents.md, conversation.md) — renamed from the prior databricks-genie / "Genie Space" naming.
  • experimental/databricks-metric-views/: renamed genie-integration.md → genie-agent-integration.md; rewired genie-agent-integration.md / query-patterns.md to cross-link with the Genie subskills; related reference cleanup.
  • Regenerated manifest.json (both skills registered with repo_dir: experimental).

Validation

  • python3 scripts/skills.py validate — no databricks-genie-agent or databricks-metric-views errors.
  • Both skills ship the full required anatomy (SKILL.md, references/, agents/openai.yaml, assets/databricks.{svg,png}) per CONTRIBUTING.

This pull request and its description were written by Isaac.

anhhchu added 8 commits June 10, 2026 14:33
@anhhchu
experimental: add databricks-genie skill; refactor metric-views query…
11b28df 
… & Genie docs

Add the databricks-genie experimental skill (MCP create/query/export/import/
migrate + Conversation API) and its in-product Genie Code Agent suite
(create/diagnose/optimize-space/optimize-query).

Reorganize metric-views references for clear separation of concerns and wire
cross-skill discovery:
- Split querying out of patterns.md -> create-patterns.md (creation) and new
  query-patterns.md (MEASURE() basics, filtering, join rollups, window measures,
  casting, MCP query, plus rules & gotchas incl. MISSING_AGGREGATION).
- Trim genie-integration.md to metric-view *design*; move Genie-space operation
  (sizing, validation loop, benchmarks, regression, checklist, anti-patterns) to
  databricks-genie/space-quality.md.
- Bidirectional discovery between the main skill and the Genie Code suite;
  de-duplicate metric-view rules by pointing the suite at the canonical files.
- create-genie-space now recommends building a governed metric view (and asks
  the user to confirm) when raw tables are the source for reusable KPIs.

Co-authored-by: Isaac

Signed-off-by: Anh Chu <anhhchu12@gmail.com>
@anhhchu
experimental: clarify Genie space config ordering in create-genie-space
ee627a6 
Make the recommended order for adding Space config surfaces explicit and
consistent (structured context first, free-text instructions last):
- Add space description as the foundational first item (required for
  multi-agent routing) in Design Priorities and Workflow step 7.
- Add the missing SQL functions surface and mark text instructions as the
  last resort in Workflow step 7.
- Add a Vocabulary table mapping Genie-UI terms (SQL expressions, SQL
  queries/instructions, SQL functions, text instructions, space description)
  to the skill's surfaces.
- Add a space-description check to Static Health Checks.

Co-authored-by: Isaac

Signed-off-by: Anh Chu <anhhchu12@gmail.com>
@anhhchu
Genie space lifecycle subskills mapping in main databricks-genie skill
5a55787 
Signed-off-by: Anh Chu <anhhchu12@gmail.com>
@anhhchu
Merge main into semantic_genie; reconcile genie & metric-views skills
f12d191 
Bring in all upstream main changes (per-provider plugin bundles, generated
manifests, new/expanded skills) and reconcile the two overlapping skills so
they are comprehensive but not duplicated, with the CLI as the default and
MCP shown as an optional reference where available.

databricks-genie:
- Keep the branch's lifecycle subskills (create/diagnose/optimize-space/
  optimize-query), "When to Use", reference files, and metric-views links.
- Adopt main's CLI-first SKILL.md (databricks genie create-space, get-space,
  start-conversation, etc.) plus its serialized_space field schema and
  troubleshooting as the canonical path.
- Demote the branch's MCP-first content to "MCP alternative (if available)"
  callouts (manage_genie / ask_genie / get_table_stats_and_schema).
- Add CLI-is-the-default banners to references/spaces.md and conversation.md.
- Lifecycle table now lists CLI as default with an MCP-equivalent column.
- Take main's correct openai.yaml (branch copy had MLflow metadata).

databricks-metric-views:
- Restore main's SQL DDL operations + CLI execution as the default path.
- Demote manage_metric_views to "MCP Tools (if available)".
- Inspect-schema step now CLI-default (discover-schema) with MCP alternative.
- Keep the branch's genie-integration.md and query-patterns.md references.

Regenerated manifest.json and the plugins/databricks bundle; validate and the
111 hook tests pass.

Co-authored-by: Isaac

Signed-off-by: Anh Chu <anhhchu12@gmail.com>
@anhhchu
experimental(databricks-genie): make lifecycle subskills host-agnosti…
8261126 
…c (Genie Code + CLI/MCP)

Reframe the four Genie lifecycle subskills (create/diagnose/optimize-space/
optimize-query) so they run both inside Databricks Genie Code (native UI) and
from an external agent via the Databricks CLI/MCP, instead of reading as
Genie-Code-only with a CLI appendix.

- Host-agnostic H1 titles and frontmatter descriptions (keep trigger keywords so
  discovery fires for external use too); names unchanged.
- Add an "Execution Context" section up front (Genie Code UI vs CLI/MCP) that
  names per-skill UI-only surfaces and prerequisites.
- Promote the former "Running Outside Databricks" appendix to a "Mechanism Map"
  referenced from Execution Context.
- Wire verified CLI substitutes: genie-*-eval-run (Beta) for native benchmark
  execution, query-history list --include-metrics for query triage,
  system.access.audit for Monitor-tab feedback; flag insight labels and
  /analyze rewrite as the only UI-only residuals.
- Document benchmark_question_ids scoping for staged eval gates, with a warning
  that an empty body or a misspelled field silently runs all questions.
- Reconcile parent SKILL.md: subskills are dual-host lifecycle skills (not
  in-product-only); databricks-genie is the parent orchestration hub.

Co-authored-by: Isaac

Signed-off-by: Anh Chu <anhhchu12@gmail.com>
@anhhchu
experimental: document column_configs schema in databricks-genie
cacfa1f 
The create-genie-space skill told the agent to enable column
descriptions, synonyms, format assistance, entity matching, and
hidden fields in prose, but never documented the serialized_space
field that carries them (column_configs). As a result the create
workflow shipped spaces with empty per-column context.

- spaces.md: add the API-verified column_configs schema (column_name,
  description[], synonyms[], enable_format_assistance,
  enable_entity_matching, exclude), extend the Python helper to build
  it, and fix the two spots that called it a merely "preserved" field.
- create-genie-space SKILL.md step 7: point the description/synonym/
  format/entity/hidden-field bullets at their column_configs sub-fields
  and add an explicit "build column_configs during creation" step.
- space-design-guide.md: note surfaces 4-6 + hidden fields apply per
  column via column_configs[], and reinforce selective enabling of
  format assistance / entity matching (cardinality-driven, not blanket).

Verified end-to-end: schema round-trips through the live Genie API.

Co-authored-by: Isaac

Signed-off-by: Anh Chu <anhhchu12@gmail.com>
@anhhchu
experimental: merge space-quality.md into databricks-genie subskills
1323d03 
Fold the condensed space-quality.md reference into the lifecycle subskills
that already own its methodology, eliminating the one mirrored file in the
databricks-genie skill:

- Move Space Sizing, the incremental build & validation loop, and
  Anti-Patterns into create-genie-space/space-design-guide.md (regression
  testing already lived in optimize-genie-space/optimization-guide.md).
- Rewire all references (parent SKILL.md x4, metric-views
  genie-integration.md x3) to the new homes.
- Delete references/space-quality.md and regenerate manifest.json.

Co-authored-by: Isaac

Signed-off-by: Anh Chu <anhhchu12@gmail.com>
@anhhchu
Merge branch 'main' into anhhchu/semantic_genie
45a9b8e 
Signed-off-by: Anh Chu <anhhchu12@gmail.com>
@anhhchu
anhhchu force-pushed the anhhchu/semantic_genie branch from e94e994 to c6db772 Compare June 23, 2026 22:14
@anhhchu
experimental: rename databricks-genie → databricks-genie-agent
bfe2ed1 
- Rename databricks-genie skill tree to databricks-genie-agent (spaces → agents)
- Rename metric-views genie-integration.md → genie-agent-integration.md; refresh references
- Regenerate manifest.json

Signed-off-by: Anh Chu <anhhchu12@gmail.com>
Co-authored-by: Isaac
@anhhchu
anhhchu force-pushed the anhhchu/semantic_genie branch from c6db772 to bfe2ed1 Compare June 23, 2026 22:42
@anhhchu anhhchu changed the title experimental: add databricks-genie lifecycle skill suite + wire it to databricks-metric-views experimental: add databricks-genie-agent lifecycle skill suite (Genie Space → Genie Agent rename) + wire it to databricks-metric-views Jun 23, 2026
anhhchu added 3 commits June 24, 2026 14:33
@anhhchu
experimental: fix metric-view base-view + format guidance
09cad4e 
- Don't recommend a base view for a single fact table; source it
  directly with dimension joins. Base views are only for multi-fact
  or nested KPIs.
- Correct format-block guidance: format IS supported but requires a
  `type` discriminator (number/currency/percentage/byte/date/date_time)
  with enum-token values, not "yyyy-MM-dd" patterns. Replaces the prior
  "omit format entirely" rule that produced unformatted metric views.

Co-authored-by: Isaac
@anhhchu
experimental: note format assistance/entity matching are space-only
04132b2 
These two column_configs toggles have no Metric View equivalent, so a
Genie space built on a fully-governed MV silently ships with both off
unless column_configs are emitted explicitly. Document the fact once in
the agents.md schema reference, with an anti-pattern row in the design
guide and a pointer from the create workflow.

Co-authored-by: Isaac
@anhhchu
experimental: require column qualification in Genie SQL surfaces
0c1a85a 
Add a "Qualify Columns In SQL" rule so SQL snippets and example SQL
always prefix columns with their table/Metric View name, avoiding the
"Table name or alias is required for column" error. Also fix "a Agent"
→ "an Agent" grammar and stale databricks-genie → databricks-genie-agent
link text.

Co-authored-by: Isaac
@dustinvannoy-db

dustinvannoy-db commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Closing this one in favor of reworking this internally and pushing final updates to this repo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lennartkats-db lennartkats-db Awaiting requested review from lennartkats-db lennartkats-db is a code owner
@simonfaltum simonfaltum Awaiting requested review from simonfaltum simonfaltum is a code owner
@dustinvannoy-db dustinvannoy-db Awaiting requested review from dustinvannoy-db dustinvannoy-db is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@anhhchu @dustinvannoy-db