Repository navigation
Add catalog and job status commands for DataHerb Explorer - #26
Merged
Merged
Conversation
Adds `dataherb catalog build|validate|lint|serve` and `dataherb status emit|check`, backed by a new dataherb.catalog package (config, stores for git/s3/http/local, metadata resolution, JSON Schemas, the dataherb.status/v1 job status spec, metadata quality lint). `dataherb create` now scaffolds a v2 dataherb.json from the data files (with --no-input for scripts), and `dataherb validate` adds a schema check and a metadata quality score. `dataherb upload` uses the current folder instead of the import-time cwd. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016Zz2NMq7okdD9v71q6vtDq
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016Zz2NMq7okdD9v71q6vtDq
emptymalei
pushed a commit
to DataHerb/dataherb-explorer
that referenced
this pull request
Oct 7, 2026
The builder, validator, linter and status emitter moved into the existing dataherb CLI (DataHerb/dataherb-python#26) as `dataherb catalog ...` and `dataherb status ...`. This repo now keeps only the site, the config, the catalog entries and the workflows. requirements.txt pins the dataherb branch until that PR is released. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016Zz2NMq7okdD9v71q6vtDq
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by L · project thread
Before:
dataherbcould create, validate, upload and serve git-hosted datasets, but there was no way to build the new static catalog site (DataHerb Explorer) or to monitor the jobs that refresh datasets.After: the same CLI builds and checks the explorer catalog and writes and checks job status files:
dataherb catalog build [-c dataherb.config.yml] [-o dist] [--site site] [--strict]dataherb catalog validatedataherb catalog lint [--min-score N]dataherb catalog serve [dist]dataherb status emit --target s3://... --job-id X --status running/success/failed ...dataherb.status/v1status file.dataherb status check [--fail-on failing,stuck,stale]dataherb createnow drafts a v2dataherb.jsonfrom the data files in a folder (columns, types, row counts), asks for the source (git, s3, http, local), owner, update frequency and status job, and has--no-inputfor scripts.dataherb validateadds a schema check and prints the metadata quality score. v1dataherb.jsonand the legacy.dataherb/metadata.ymlstill parse.How: a new
dataherb.catalogpackage (config, stores, metadata resolution, status spec, lint, schemas) with click commands indataherb/cmd/catalog.py. The site template itself lives in DataHerb/dataherb-explorer;catalog buildusessite/next to the config. New deps: PyYAML, jsonschema; extrass3(boto3) andinfer(duckdb). Also fixesdataherb uploadusing the cwd captured at import time, and replacesdistutils.copy_treeso the CLI imports on Python 3.12. Tests intests/catalog/; tutorials added to the docs.🤖 Generated with Claude Code
https://claude.ai/code/session_016Zz2NMq7okdD9v71q6vtDq
Generated by Claude Code
Generated by Claude Code