Introduce nativelink live

.github/styles/config/vocabularies/TraceMachina/accept.txt

@@ -115,3 +115,5 @@ Brex
 Citrix
 Menlo
 benchmarked
+Datahouse
+WebSocket

web/live/.gitignore

@@ -0,0 +1,28 @@
+dist
+.wrangler
+.output
+.vercel
+.netlify
+.vinxi
+app.config.timestamp_*.js
+
+# Environment
+.env
+.env*.local
+
+# dependencies
+/node_modules
+
+# IDEs and editors
+/.idea
+.project
+.classpath
+*.launch
+.settings/
+
+# Temp
+gitignore
+
+# System Files
+.DS_Store
+Thumbs.db

web/live/README.md

@@ -0,0 +1,47 @@
+# nativelink-live
+Live feed viewer for NativeLink.
+This is useful when you want to see what's happening under the hood.
+
+## State
+This is alpha software. It's not recommended for production use and not covered by SLA.
+
+## Usage
+<!-- maybe live.nativelink.com -->
+You can use the website we host(TBA), or you can self-host it yourself.
+Since this is client-side only static web, you only need to get the service that serves these static files.
+
+To build the static files, you need to have Node.js and `npm` installed.
+Then, run the following commands:
+
+```bash
+npm install
+npm run build
+```
+
+This will create a `dist` directory with the static files.
+You can serve this directory with any static file server, such as `nginx`, `caddy`, or `serve`.
+
+TODO(ilsubyeega): add example configuration for serving this assets.
+
+While for development, you may want to use `npm run dev` to start a local development server with hot-reload instead of building.
+
+## Screenshot
+TBA
+
+## Goals
+- Minimal code; minimal implementation.
+
+## Non-goals
+- Invoking something on the NativeLink side; this is a read-only viewer.
+- dedicated tenant support and auth; should be handled by reverse proxy in front of the app
+- support old browser, this uses cutting-edge web technologies.
+
+## License
+
+Copyright 2020–2025 Trace Machina, Inc.
+
+Licensed under the Functional Source License, Version 1.1, Apache 2.0 Future License.
+SPDX identifier: `FSL-1.1-Apache-2.0`.
+
+After the second anniversary of the date this version was made available, you may use this
+software under the Apache License, Version 2.0.

web/live/TODOS

@@ -0,0 +1,9 @@
+# Modeling
+- [x] Figure out how UUID works: it has been reviewed but not explict; just a probability of like lottery
+- [ ] WebSockets or HTTP Streaming?
+- [ ] Modelling the data structure
+
+# Impl
+
+# Testing
+- [ ] Figure out how-to interatively test UIs

web/live/docs/APIS.md

@@ -0,0 +1,14 @@
+# Datahouse
+Datahouse saves every activities from nativelink instances for metrics and provide APIs for querying.
+
+## APIs
+- `GET /detail/:id` - Get the detail of a specific action by its ID. Datahouse will collect when: action is first created and then completed.
+
+# Live Activity Monitor
+This service should collect every components from nativelink instances and then expose them via WebSocket.
+- WebSocket endpoint: `/ws?version=<NATIVELINK_LIVE_VERISON>`
+- For WebSocket formats, see [WEBSOCKETS.md](WEBSOCKETS.md)
+
+## APIs
+- `GET /activities/overall` - Get the overall statistics of activities.
+- `GET /status/:id` - Get the status of a specific action by its ID. The service will cache the status for a while even if the action is completed.

web/live/docs/README.md

@@ -0,0 +1,56 @@
+# Concepts
+
+This document describes the concepts behind `nativelink-live`, a real-time live feed viewer for NativeLink.
+It's intended to provide insight into the inner workings of NativeLink by displaying live data feeds.
+
+## Key Components
+
+### Live Activity Monitor
+
+This feature offers a real-time, filterable view of all activities occurring within NativeLink.
+It allows users to monitor events as they happen, providing immediate insights into system operations.
+
+#### Key Capabilities:
+
+- View real-time activities.
+- Filter activities by various criteria (for example, type, status).
+  - Type: All, Upload, Download, Clients, Executions, Others
+  - Status: All, Success, In Progress, Queued, Failed, Canceled
+- Search for specific activities using keywords.
+- Clicking the item would show like: Details, `Stdout`, `Stderr` (if applicable), Timestamp.
+
+### Worker Status Dashboard
+
+This dashboard provides a overview of the status of all workers in the NativeLink clusters.
+It displays information such as status, information, and metrics of each worker.
+
+#### Key Capabilities:
+
+- View the information of all workers.
+  - Worker ID, Live Metrics(CPU, Memory, Disk, Network), Assigned Actions, Last Heartbeat
+  - Status: Idle, Working, Unavailable, Offline
+    - Idle: The worker is online and waiting for tasks.
+    - Working: The worker is currently processing tasks. Also has sub-statuses: Downloading, Executing, Uploading
+    - Unavailable: Temporarily unable to connect. Maybe worker is dead?
+    - Offline: The worker is offline and not available for tasks.
+  - Shows the assigned actions to each worker by tree-view.
+  - Search for specific workers using keywords.
+
+### Storage Status Overview
+
+This overview provides a summary of the status of all storage connected to NativeLink.
+It displays information such as status, configuration, and usage of each storage.
+
+### Client Connection Overview
+
+This overview provides a summary of the status of all clients connected to NativeLink.
+This helps to identify which clients are interacting with the nativelink cluster, what they're doing.
+
+#### Key Capabilities:
+Client ID, Connection Info, Status(Idle, Disconnected), Metrics(total execution, data transferred, last seen), Activities
+
+### Log stream viewer
+
+This feature provides a real-time stream of logs from NativeLink components.
+It allows of users/devs to monitor logs and deep-dive debugging without requiring terminal access of nativelink.
+The UI should be similar as Chromium's console tab viewer.

web/live/docs/WEBSOCKETS.md

@@ -0,0 +1,2 @@
+# WebSocket
+Live feed should be fed by WebSocket server, for real-time purpose.

web/live/index.html

@@ -0,0 +1,17 @@
+<!DOCTYPE html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1" />
+    <meta name="theme-color" content="#000000" />
+    <link rel="icon" type="image/png" href="https://nativelink-cdn.s3.us-east-1.amazonaws.com/nativelink_favicon.png" />
+    <link rel="stylesheet" href="src/index.css" />
+    <title>Nativelink Live</title>
+  </head>
+  <body>
+    <noscript>You need to enable JavaScript to run this app.</noscript>
+    <div id="root"></div>
+
+    <script src="/src/index.tsx" type="module"></script>
+  </body>
+</html>

web/live/package.json

@@ -0,0 +1,31 @@
+{
+  "name": "nativelink-live",
+  "type": "module",
+  "dependencies": {
+    "material-symbols": "^0.36.3",
+    "solid-js": "^1.9.9"
+  },
+  "description": "Live view for nativelink",
+  "devDependencies": {
+    "@solidjs/testing-library": "^0.8.10",
+    "@testing-library/jest-dom": "^6.9.1",
+    "@testing-library/user-event": "^14.6.1",
+    "@vitest/browser": "4.0.0-beta.17",
+    "@vitest/browser-preview": "4.0.0-beta.17",
+    "jsdom": "^25.0.1",
+    "solid-devtools": "^0.34.4",
+    "typescript": "^5.9.3",
+    "vite": "^7.1.9",
+    "vite-plugin-solid": "2.11.8",
+    "vitest": "4.0.0-beta.17"
+  },
+  "license": "MIT",
+  "scripts": {
+    "build": "vite build",
+    "dev": "vite",
+    "serve": "vite preview",
+    "start": "vite",
+    "test": "vitest"
+  },
+  "version": "0.0.0"
+}