feat: unify dirctl cli output (#289)

* add new dirctl cli --output flag description
* update cli command --output examples

Signed-off-by: Tibor Kircsi <tkircsi@cisco.com>

Author: tkircsi

docs/dir/directory-cli.md

@@ -67,6 +67,62 @@ The following example demonstrates how to store, publish, search, and retrieve a
     dirctl pull baeareihdr6t7s6sr2q4zo456sza66eewqc7huzatyfgvoupaqyjw23ilvi
     ```
 
+## Output Formats
+
+All `dirctl` commands support multiple output formats via the `--output` (or `-o`) flag, making it easy to switch between human-readable output and machine-processable formats.
+
+### Available Formats
+
+| Format | Description | Use Case |
+|--------|-------------|----------|
+| `human` | Human-readable, formatted output with colors and tables (default) | Interactive terminal use |
+| `json` | Pretty-printed JSON with indentation | Debugging, single-record processing |
+| `jsonl` | Newline-delimited JSON (compact, one object per line) | Streaming, batch processing, logging |
+| `raw` | Raw values only (e.g., CIDs, IDs) | Shell scripting, piping to other commands |
+
+### Usage
+
+```bash
+# Human-readable output (default)
+dirctl routing list
+
+# JSON output (pretty-printed)
+dirctl routing list --output json
+dirctl routing list -o json
+
+# JSONL output (streaming-friendly)
+dirctl events listen --output jsonl
+
+# Raw output (just values)
+dirctl push my-agent.json --output raw
+```
+
+### Piping and Processing
+
+Structured formats (`json`, `jsonl`, `raw`) automatically route data to **stdout** and metadata messages to **stderr**, enabling clean piping to tools like `jq`:
+
+```bash
+# Process JSON output with jq
+dirctl routing search --skill "AI" -o json | jq '.[] | .cid'
+
+# Stream events and filter by type
+dirctl events listen -o jsonl | jq -c 'select(.type == "EVENT_TYPE_RECORD_PUSHED")'
+
+# Capture CID for scripting
+CID=$(dirctl push my-agent.json -o raw)
+echo "Stored with CID: $CID"
+
+# Chain commands
+dirctl routing list -o json | jq -r '.[].cid' | xargs -I {} dirctl pull {}
+```
+
+### Format Selection Guidelines
+
+- **`human`**: Default for terminal interaction, provides context and formatting
+- **`json`**: Best for debugging or when you need readable structured data
+- **`jsonl`**: Ideal for streaming events, logs, or processing large result sets line-by-line
+- **`raw`**: Perfect for shell scripts and command chaining where you only need the value
+
 ## Command Reference
 
 ### Storage Operations
@@ -398,7 +454,9 @@ The following workflow demonstrates how to publish a record to the network:
 1. Store your record
 
     ```bash
-    CID=$(dirctl push my-agent.json)
+    # Using --output raw for clean scripting
+    CID=$(dirctl push my-agent.json --output raw)
+    echo "Stored with CID: $CID"
     ```
 
 1. Publish for discovery
@@ -410,7 +468,8 @@ The following workflow demonstrates how to publish a record to the network:
 1. Verify the record is published
 
     ```bash
-    dirctl routing list --cid $CID
+    # Use JSON output for programmatic verification
+    dirctl routing list --cid $CID --output json
     ```
 
 1. Check routing statistics
@@ -426,18 +485,22 @@ The following workflow demonstrates how to discover records from the network:
 1. Search for records by skill
 
     ```bash
-    dirctl routing search --skill "AI" --limit 10
+    # Use JSON output to process results
+    dirctl routing search --skill "AI" --limit 10 --output json
     ```
 
 1. Search with multiple criteria
 
     ```bash
-    dirctl routing search --skill "AI" --locator "docker-image" --min-score 2
+    dirctl routing search --skill "AI" --locator "docker-image" --min-score 2 --output json
     ```
 
-1. Pull interesting records
+1. Pull discovered records
     ```bash
-    dirctl pull <discovered-cid>
+    # Extract CIDs and pull records
+    dirctl routing search --skill "AI" --output json | \
+      jq -r '.[].record_ref.cid' | \
+      xargs -I {} dirctl pull {}
     ```
 
 ### Synchronization Workflow
@@ -447,19 +510,23 @@ The following workflow demonstrates how to synchronize records between remote di
 1. Create sync with remote peer
 
     ```bash
-    SYNC_ID=$(dirctl sync create https://peer.example.com)
+    # Using --output raw for clean variable capture
+    SYNC_ID=$(dirctl sync create https://peer.example.com --output raw)
+    echo "Sync created with ID: $SYNC_ID"
     ```
 
 1. Monitor sync progress
 
     ```bash
-    dirctl sync status $SYNC_ID
+    # Use JSON output for programmatic monitoring
+    dirctl sync status $SYNC_ID --output json
     ```
 
 1. List all syncs
 
     ```bash
-    dirctl sync list
+    # Use JSONL output for streaming results
+    dirctl sync list --output jsonl
     ```
 
 1. Clean up when done
@@ -468,6 +535,18 @@ The following workflow demonstrates how to synchronize records between remote di
     dirctl sync delete $SYNC_ID
     ```
 
+### Advanced Synchronization: Search-to-Sync Pipeline
+
+Automatically sync records that match specific criteria from the network:
+
+```bash
+# Search for AI-related records and create sync operations
+dirctl routing search --skill "AI" --output json | dirctl sync create --stdin
+
+# This creates separate sync operations for each remote peer found,
+# syncing only the specific CIDs that matched your search criteria
+```
+
 ## Command Organization
 
 The CLI follows a clear service-based organization:

docs/dir/events.md

@@ -97,10 +97,20 @@ dirctl events listen --cids bafybe...,bafkyb...
 For programmatic processing:
 
 ```bash
-# JSON format output
-dirctl events listen --json
+# JSONL format output (streaming-friendly, one event per line)
+dirctl events listen --output jsonl
+
+# JSON format output (pretty-printed)
+dirctl events listen --output json
 ```
 
+**JSONL format** (recommended for streaming - compact, one event per line):
+```json
+{"id":"550e8400-...","type":"EVENT_TYPE_RECORD_PUSHED","timestamp":"2025-10-18T14:23:15.123456Z","resource_id":"bafybeig...","labels":["/skills/AI"]}
+{"id":"550e8401-...","type":"EVENT_TYPE_RECORD_PUBLISHED","timestamp":"2025-10-18T14:23:16.456789Z","resource_id":"bafybeig...","labels":["/skills/AI"]}
+```
+
+**JSON format** (pretty-printed):
 ```json
 {
   "id": "550e8400-e29b-41d4-a716-446655440000",
@@ -111,6 +121,8 @@ dirctl events listen --json
 }
 ```
 
+> **Note:** For streaming events, use `--output jsonl` format as it outputs one compact JSON object per line, making it ideal for real-time processing with tools like `jq`.
+
 ### Combine Filters
 
 All filter types can be combined for precise event monitoring:
@@ -120,7 +132,7 @@ All filter types can be combined for precise event monitoring:
 dirctl events listen \
   --types RECORD_PUSHED,RECORD_PULLED \
   --labels /skills/AI \
-  --json
+  --output jsonl
 ```
 
 ## Using the Go SDK
@@ -251,7 +263,7 @@ Track system activity for operational awareness:
 
 ```bash
 # Monitor all operations in production
-dirctl events listen --json | tee events.log
+dirctl events listen --output jsonl | tee events.log
 
 # Alert on failed syncs
 dirctl events listen --types SYNC_FAILED | \
@@ -294,7 +306,7 @@ Maintain audit trails for critical operations:
 
 ```bash
 # Record all signature events
-dirctl events listen --types RECORD_SIGNED --json >> audit.jsonl
+dirctl events listen --types RECORD_SIGNED --output jsonl >> audit.jsonl
 ```
 
 ### Development and Debugging
@@ -303,7 +315,7 @@ Monitor system behavior during development:
 
 ```bash
 # Watch all events during testing
-dirctl events listen --json | jq .
+dirctl events listen --output jsonl | jq -c .
 
 # Track specific record through the system
 dirctl events listen --cids $RECORD_CID

docs/dir/hosted-agent-directory.md

@@ -177,7 +177,7 @@ The `dirctl` command line tools supports two methods to authenticate to a Outshi
         ```
 
         ```bash
-        $ dirctl hub apikey create --org-name your-user --role ROLE_ADMIN --json > api-key-creds.json
+        $ dirctl hub apikey create --org-name your-user --role ROLE_ADMIN --output json > api-key-creds.json
         ```
 
     * Set the environment variable and use the dirctl commands as usual or provide the JSON file with the credentials:

docs/dir/scenarios.md

@@ -384,7 +384,7 @@ match specific criteria:
 ```bash
 # Search for agents with a given skill across
 # the network and sync them automatically
-dirctl routing search --skill "Audio" --json | dirctl sync create --stdin
+dirctl routing search --skill "Audio" --output json | dirctl sync create --stdin
 ```
 
 This creates separate sync operations for each remote peer found in the search results,