Repository navigation
Add dataherb catalog add; read Markdown catalog entries - #27
Merged
Merged
Conversation
Writes one catalog/<id>.yml per repo: a pointer when the repo has dataherb metadata, otherwise an inline entry inferred from a shallow clone. --org/--match pick every repo of a GitHub org by name prefix. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ggg4AutCcRdK8c57NMxSqp
catalog/<id>.md holds the entry fields in YAML front matter and free text in the body, which becomes the dataset's documentation (before the dataset's own). .md files without front matter, like a README, are ignored. `dataherb catalog add` writes .md by default (--format yml for the old style) and keeps the body when overwriting with --force. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ggg4AutCcRdK8c57NMxSqp
emptymalei
changed the title
dataherb catalog add for git reposdataherb catalog add; read Markdown catalog entries
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ggg4AutCcRdK8c57NMxSqp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by L · project thread
Before: listing a git repo in an explorer catalog meant hand-writing
catalog/<id>.yml, and entries had no room for free text.After:
catalog/<id>.mdholds the entry fields in YAML front matter. Its body becomes the dataset'sdocumentation, shown above the dataset's own documentation..mdfiles without front matter, such as a README, are ignored..ymland.jsonentries still work.dataherb catalog add owner/repo ...ordataherb catalog add --org DataHerb --match datasetwrites the entries, as.mdby default (--format ymlfor the old style).dataherb.json,dataherb.yml,.dataherb/metadata.yml) gets a pointer entry.dataherb createdoes it.--forceoverwrites the front matter and keeps the body. Ids drop the--matchprefix, and--dry-runpreviews.How:
util.split_front_matter/front_matter_document.resolve.load_entriesreads.mdfiles and keeps the body as_body.normalizeputs it in front of the dataset's own documentation.catalog/add.py. It probes metadata through the config's git store (raw URLs,token_env). When that fails, because a repo has no metadata or is private and there's no token, it shallow-clones the repo with the local git credentials.--orgpages through/orgs/{org}/repos, falling back to/users/, and--api-urlpoints it at GitHub Enterprise.tests/catalog/test_add.pyandtest_catalog.py. The full suite passes (26 tests).DataHerb/dataherb-explorer#1 uses this and is pinned to this branch until it merges.
🤖 Generated with Claude Code
https://claude.ai/code/session_01Ggg4AutCcRdK8c57NMxSqp