Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
184 changes: 184 additions & 0 deletions .github/workflows/build-and-deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,184 @@
name: Build and Deploy

on:
push:
branches:
- main
pull_request:
branches:
- main
workflow_dispatch:

# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write

# Allow one concurrent deployment
concurrency:
group: "pages"
cancel-in-progress: true

jobs:
lint-and-test:
name: Lint and Test Rust
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
with:
components: rustfmt, clippy

- name: Cache Cargo dependencies
uses: actions/cache@v4
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-

- name: Check Rust formatting
run: cargo fmt --all -- --check

- name: Run Clippy
run: cargo clippy --all-targets --all-features -- -D warnings

- name: Run Rust tests
run: cargo test --all --verbose

build-and-deploy:
name: Build and Deploy to GitHub Pages
needs: lint-and-test
runs-on: ubuntu-latest
# Only deploy on push to main/master, not on PRs
if: github.event_name != 'pull_request' && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/master')
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
with:
targets: wasm32-unknown-unknown

- name: Cache Cargo dependencies
uses: actions/cache@v4
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-cargo-wasm-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-wasm-
${{ runner.os }}-cargo-

- name: Install wasm-pack
run: curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh

- name: Build WASM (release mode)
run: wasm-pack build ./arrow-db-wasm --target web --out-dir ../arrow-db-browser/arrow-db-wasm
working-directory: ${{ github.workspace }}

- name: Setup pnpm
uses: pnpm/action-setup@v4
with:
version: 8

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20"
cache: "pnpm"
cache-dependency-path: arrow-db-browser/pnpm-lock.yaml

- name: Install dependencies
run: pnpm install --frozen-lockfile
working-directory: arrow-db-browser

- name: Build Vite app
run: pnpm build
working-directory: arrow-db-browser
env:
# Set base path for GitHub Pages (will be /repository-name/)
# Adjust this if your repo name is different
BASE_URL: /${{ github.event.repository.name }}/

- name: Setup Pages
uses: actions/configure-pages@v4

- name: Upload artifact
uses: actions/upload-pages-artifact@v3
with:
path: "arrow-db-browser/dist"

- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4

build-only:
name: Build (PR Check)
needs: lint-and-test
runs-on: ubuntu-latest
# Only run this job on PRs
if: github.event_name == 'pull_request'
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
with:
targets: wasm32-unknown-unknown

- name: Cache Cargo dependencies
uses: actions/cache@v4
with:
path: |
~/.cargo/bin/
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
target/
key: ${{ runner.os }}-cargo-wasm-${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-cargo-wasm-
${{ runner.os }}-cargo-

- name: Install wasm-pack
run: curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh

- name: Build WASM (release mode)
run: wasm-pack build ./arrow-db-wasm --target web --out-dir ../arrow-db-browser/arrow-db-wasm
working-directory: ${{ github.workspace }}

- name: Setup pnpm
uses: pnpm/action-setup@v4
with:
version: 9

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20"
cache: "pnpm"
cache-dependency-path: arrow-db-browser/pnpm-lock.yaml

- name: Install dependencies
run: pnpm install --frozen-lockfile
working-directory: arrow-db-browser

- name: Build Vite app
run: pnpm build
working-directory: arrow-db-browser
155 changes: 155 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
.PHONY: help install clean fmt lint test build dev deploy \
rust-fmt rust-lint rust-test rust-build \
wasm-build wasm-build-dev wasm-build-release \
browser-install browser-dev browser-build browser-lint browser-typecheck browser-test \
check ci

# Default target - show help
help:
@echo "Arrow DB - Available Make Commands"
@echo ""
@echo "🚀 Quick Start:"
@echo " make install - Install all dependencies (Rust + Browser)"
@echo " make dev - Start development server"
@echo " make build - Build everything (WASM + Browser)"
@echo " make check - Run all checks (format, lint, typecheck, test)"
@echo ""
@echo "🦀 Rust Commands:"
@echo " make rust-fmt - Format Rust code"
@echo " make rust-lint - Run Clippy on Rust code"
@echo " make rust-test - Run Rust tests"
@echo " make rust-build - Build Rust workspace"
@echo ""
@echo "🌐 WASM Commands:"
@echo " make wasm-build-dev - Build WASM in development mode"
@echo " make wasm-build-release - Build WASM in release mode"
@echo ""
@echo "⚛️ Browser Commands:"
@echo " make browser-install - Install browser dependencies"
@echo " make browser-dev - Start Vite dev server"
@echo " make browser-build - Build browser app"
@echo " make browser-lint - Lint browser code"
@echo " make browser-typecheck - Check TypeScript types"
@echo " make browser-test - Run browser tests"
@echo ""
@echo "🧹 Cleanup:"
@echo " make clean - Remove build artifacts"
@echo ""
@echo "🔄 CI/CD:"
@echo " make ci - Run full CI pipeline locally"
@echo ""

# Install all dependencies
install: browser-install
@echo "✅ All dependencies installed"

# ============================================================================
# Rust Commands
# ============================================================================

rust-fmt:
@echo "🦀 Formatting Rust code..."
cargo fmt --all

rust-lint:
@echo "🦀 Running Clippy..."
cargo clippy --all-targets --all-features -- -D warnings

rust-test:
@echo "🦀 Running Rust tests..."
cargo test --all --verbose

rust-build:
@echo "🦀 Building Rust workspace..."
cargo build --release

# ============================================================================
# WASM Commands
# ============================================================================

wasm-build-dev:
@echo "🌐 Building WASM (development mode)..."
wasm-pack build ./arrow-db-wasm --dev --target web --out-dir ../arrow-db-browser/arrow-db-wasm

wasm-build-release:
@echo "🌐 Building WASM (release mode)..."
wasm-pack build ./arrow-db-wasm --target web --out-dir ../arrow-db-browser/arrow-db-wasm

wasm-build: wasm-build-release

# ============================================================================
# Browser Commands
# ============================================================================

browser-install:
@echo "⚛️ Installing browser dependencies..."
cd arrow-db-browser && pnpm install

browser-dev: wasm-build-dev
@echo "⚛️ Starting Vite dev server..."
cd arrow-db-browser && pnpm dev

browser-build:
@echo "⚛️ Building browser app..."
cd arrow-db-browser && pnpm build

browser-lint:
@echo "⚛️ Linting browser code..."
cd arrow-db-browser && pnpm lint

browser-typecheck:
@echo "⚛️ Type checking browser code..."
cd arrow-db-browser && pnpm typecheck

browser-test:
@echo "⚛️ Running browser tests..."
cd arrow-db-browser && pnpm test

# ============================================================================
# Combined Commands
# ============================================================================

# Format all code
fmt: rust-fmt
@echo "✅ All code formatted"

# Lint all code
lint: rust-lint browser-lint
@echo "✅ All linting passed"

# Run all tests
test: rust-test browser-test
@echo "✅ All tests passed"

# Build everything
build: wasm-build-release browser-build
@echo "✅ Full build complete"

# Run all checks (like CI but local)
check: rust-fmt rust-lint rust-test browser-typecheck browser-lint
@echo "✅ All checks passed"

# Start development environment
dev: browser-dev

# Clean build artifacts
clean:
@echo "🧹 Cleaning build artifacts..."
cargo clean
rm -rf arrow-db-browser/dist
rm -rf arrow-db-browser/arrow-db-wasm
rm -rf arrow-db-browser/node_modules
@echo "✅ Cleanup complete"

# ============================================================================
# CI/CD Commands
# ============================================================================

# Run the full CI pipeline locally
ci: check build
@echo "🎉 CI pipeline completed successfully"

# Deploy to GitHub Pages (usually done via CI)
deploy: build
@echo "📦 Build complete - ready for deployment"
@echo "ℹ️ Push to main branch to trigger automatic deployment"
23 changes: 22 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,32 @@ ArrowDB is a teaching tool for learning about the power of Arrow and Arrow tooli
| Crate | Description |
| ---------------------------------------------- | ------------------------------------------------------------------- |
| [arrow-db-core](arrow-db-core/README.md) | The core ArrowDB DB. |
| [arrow-db-server](arrow-db-server/README.md) | A Tonic server that leverages the Arrow Flight protocol . |
| [arrow-db-server](arrow-db-server/README.md) | A Tonic server that leverages the Arrow Flight protocol. |
| [arrow-db-client](arrow-db-client/README.md) | A Rust client for querying the ArrowDB server. |
| [arrow-db-wasm](arrow-db-wasm/README.md) | A WebAssembly module for use in the ArrowDB browser. |
| [arrow-db-browser](arrow-db-browser/README.md) | A React app for interacting with the ArrowDB server in the browser. |



## Quick Start

```bash
# Install dependencies
make install

# Start development server (builds WASM + starts Vite)
make dev

# Build everything for production
make build

# Run all checks (format, lint, test)
make check

# See all available commands
make help
```

## ArrowDB Fundamentals

ArrowDB is built on top of the [Apache Arrow](https://arrow.apache.org/) library in [Rust](https://docs.rs/arrow/latest/arrow/). Arrow is a [columnar format](https://arrow.apache.org/docs/format/Columnar.html) that is optimized for in-memory data processing and analytics. Full specifications for Arrow can be found at [https://arrow.apache.org/docs/format/index.html](https://arrow.apache.org/docs/format/index.html).
Expand Down
1 change: 1 addition & 0 deletions arrow-db-browser/vite.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import tsconfigPaths from 'vite-tsconfig-paths';
// https://vitejs.dev/config https://vitest.dev/config
export default defineConfig({
plugins: [react(), tsconfigPaths()],
base: process.env.BASE_URL || '/',
build: {
minify: false
},
Expand Down
Loading