kvdb

kvdb is a flexible database abstraction layer that provides a consistent interface across multiple storage backends (etcd, BBolt, and in-memory). It implements type validation, indexing, and query capabilities while allowing you to switch between storage engines.

Why?

First off, this is a toy project. I was curious to see how much I could do with etcd and wanted to see if I could implement a SQL-like interface on top of it. I planned on seeing how much I could implement and then see how much I could implement in a production environment. Don't expect this to be a production ready library. If you are new and ran across this, I would look at using Gorm and look at an actual database.

Features

• Multiple Storage Backends:

  • etcd: For distributed storage
  • BBolt: For single-node persistent storage
  • Memory: For testing and rapid prototyping

• Type System:

  • Strong type validation
  • Support for Go primitive types
  • Custom type handling for time.Time
  • Array and map type support

• Query Capabilities:

  • Comparison operators (=, >, >=, <, <=)
  • Multiple conditions per query
  • Index-assisted queries

• Schema Management:

  • Define table structures
  • Column type enforcement
  • Primary key support

• Indexing:

  • Secondary index support
  • Automatic index maintenance
  • Index-assisted queries

• Bulk Operations:

  • Bulk insert support
  • Transaction handling
  • Atomic operations

Quick Start

Choose Your Storage Backend

// Using etcd
db, err := etcd.NewDatabase([]string{"localhost:2379"}, "myapp")

// Using BBolt
db, err := bbolt.NewDatabase("/path/to/db", "myapp")

// Using Memory
db := memory.NewMemoryStore("myapp")

Define a Schema

schema := &types.Schema{
    Name: "users",
    Columns: map[string]string{
        "id":         "int",
        "username":   "string",
        "email":      "string",
        "created_at": "time.Time",
        "active":     "bool",
    },
    PrimaryKey: "id",
}

err := db.CreateTable(schema)

Create Indexes

indexes := []*types.Index{
    {TableName: "users", Column: "email"},
    {TableName: "users", Column: "username"},
}

err := db.BulkCreateIndex(indexes)

Insert Data

user := types.Row{
    "id":         1,
    "username":   "johndoe",
    "email":      "[email protected]",
    "created_at": time.Now(),
    "active":     true,
}

err := db.Insert("users", user)

Query Data

// Simple equality query
conditions := []comp.WhereClause{
    {
        Column:   "username",
        Operator: comp.OpEqual,
        Value:    "johndoe",
    },
}

results, err := comp.SelectWithComparisons("users", conditions, db)

Type Support

The validation system supports:

• Basic types: string, bool
• Integer types: int, int8, int16, int32, int64
• Unsigned integers: uint, uint8, uint16, uint32, uint64
• Floating point: float32, float64
• Complex numbers: complex64, complex128
• Time: time.Time
• Arrays and Slices
• Maps with string keys

Example Use Case: Router Management

The included examples demonstrate managing network router data:

schema := &types.Schema{
    Name: "routers",
    Columns: map[string]string{
        "id":              "int",
        "hostname":        "string",
        "ip_address":      "string",
        "location":        "string",
        "model":           "string",
        "firmware":        "string",
        "last_reboot":     "time.Time",
        "uptime_hours":    "int",
        "temperature":     "float64",
        "cpu_utilization": "float64",
        "is_active":       "string",
        "port_count":      "int",
        "installed_date":  "time.Time",
    },
    PrimaryKey: "id",
}

See the examples/router directory for complete examples using each storage backend.

Limitations

• No JOIN operations
• No complex constraints beyond primary key uniqueness
• Secondary indexes only support exact match queries
• Transaction support varies by backend
• Query performance depends on index usage

Development Status

This is an experimental project demonstrating different storage backend implementations with a common interface. It's suitable for learning and prototyping but requires careful evaluation before production use.

Contributing

Contributions are welcome! Please ensure you add tests for any new functionality.

License

MIT License