A modular, TypeScript‑based backend for managing AI bots, API keys, user authentication, bot analytics, and background web‑scraping jobs.
NOVA‑BOT‑STUDIO‑BACKEND provides a clean, extensible framework for building and operating bot‑centric applications. It offers:
- Bot lifecycle management – create, update, delete, and run bots.
- AI feature toggling – enable/disable advanced AI capabilities per bot.
- Secure API‑key handling – generation, hashing, revocation with Redis‑backed caching.
- Robust authentication – JWT‑based login/registration with refresh tokens, role‑based middleware, and stricter cookie validation.
- Bot analytics – collect, store, and query usage metrics per bot.
- Multi‑database support – PostgreSQL for relational data, MongoDB for flexible storage.
- Redis integration – fast look‑ups for API‑key validation, session data, and real‑time website‑bot communication.
- Controlled bots – lightweight, website‑hosted bots with a dedicated schema and management endpoints.
- Background web‑scraping – asynchronous scraping, chunking, embedding generation, and email notifications via BullMQ workers.
Targeted at developers building SaaS bot platforms, internal automation tools, or any service that needs programmable bots with fine‑grained access control, insight into bot performance, and automated content ingestion.
| Version | Date | Description |
|---|---|---|
v1.2.0 |
2026‑02‑20 | Refactored several internal methods for clearer naming and improved testability. No breaking API changes. Updated architecture diagram and documentation to reflect the corrected Bot_Configuration folder name. |
v1.1.0 |
2025‑11‑12 | Added richer website‑bot communication helpers and /communicate endpoint. |
v1.0.0 |
2025‑08‑01 | Initial public release. |
| Feature | Description | Status |
|---|---|---|
| Authentication | JWT login, registration, refresh tokens, role‑based middleware. Now validates presence of sessionId and clears stale accessToken cookies automatically. |
✅ Stable |
| API‑Key Management | Secure generation, hashing, revocation, Redis‑backed lookup. | ✅ Stable |
| Bot Configuration | CRUD for bot metadata, custom prompts, and system settings. | ✅ Stable |
| AI Feature Management | Toggle AI modules (TextEnhancer, TextValidator, etc.) per bot. |
✅ Stable |
| Advanced Bot Management | Lifecycle helpers, scheduled clean‑up, versioning. | ✅ Stable |
| Website Bot Communication | Real‑time interaction endpoints backed by Redis for sub‑millisecond latency. Added richer helper utilities and new /communicate endpoint for website‑controlled bots. |
✅ Stable |
| Controlled Bot Management | Create lightweight “website‑controlled” bots, configure style, and retrieve details via dedicated routes. | ✅ Stable |
| Bot Analytics | Capture events (messages sent, errors, usage time) and expose aggregated stats. | ✅ Stable |
| Multi‑DB Support | PostgreSQL (pg) & MongoDB (mongoose). |
✅ Stable |
| Redis Caching | Centralised client for API‑key validation, session storage, and bot messaging. | ✅ Stable |
| Background Scraping & Embedding | BullMQ worker scrapes URLs with Playwright, normalises text, creates chunks, generates embeddings, stores them in Supabase, and sends success/failure emails via Resend. | ✅ Stable |
| CORS Configuration | Whitelisted origins with credentials support. | ✅ Stable |
| Testing Utilities | Ready‑made test router and controller for CI pipelines. | ✅ Stable |
| System Prompt Utilities | Re‑usable helpers (TextEnhancer, TextValidator, ExampleValidator, Website) for building robust AI prompts. |
✅ Stable |
| Layer | Technology | Reason |
|---|---|---|
| Runtime | Node.js 18 LTS | Modern async APIs, wide ecosystem |
| Language | TypeScript 5 | Static typing, IDE support |
| Web Framework | Express 4 | Minimalist, middleware‑centric |
| Database | PostgreSQL (pg) & MongoDB (mongoose) |
Relational + flexible document storage |
| Cache | Redis (ioredis) |
Fast key‑value look‑ups for API‑key validation & bot messaging |
| Authentication | jsonwebtoken, bcrypt-ts |
Secure token handling |
| Validation / Sanitisation | class-validator, custom sanitiser helpers |
Prevent injection attacks |
| Background Jobs | BullMQ, Playwright, Resend | Reliable job queue, headless browsing, email notifications |
| Embedding Store | Supabase (PostgreSQL) | Vector storage for generated embeddings |
| Testing | Jest & Supertest | Unit & integration testing |
| Containerisation | Docker | Consistent dev/prod environments |
| CI/CD | GitHub Actions | Automated lint, test, build pipelines |
| Utilities | System Prompt utils (TextEnhancer, TextValidator, ExampleValidator, Website) |
Reusable AI‑prompt processing helpers |
src/
├─ controller/ # Request handlers (business logic)
│ ├─ authentication/
│ ├─ API_Key_Management/
│ ├─ Bot_Management/
│ ├─ Bot_Configuration/ # ← renamed from Bot_Configration
│ ├─ AI_Feature_Management/
│ ├─ AdvanceBotManagement/
│ ├─ BotAnalyticsManagement/
│ ├─ BotCommunication/
│ │ └─ Website/ # New helpers & richer communication flow
│ └─ Testing/
├─ Router/ # Express routers – one per domain
│ ├─ Authentication/
│ ├─ API_Key_Management/
│ ├─ Bot_Management/
│ ├─ Bot_Configuration/ # ← renamed from Bot_Configration
│ ├─ AI_Feature_Management/
│ ├─ Advance_Bot_Management/
│ ├─ BotAnalytics/
│ ├─ BotCommunication/
│ │ └─ Website/
│ └─ Testing/
├─ Models/ # Mongoose / TypeORM schemas
│ ├─ BotStructure.ts
│ ├─ BotConfiguration.ts
│ ├─ ControlledBotSchema.ts ← schema for website‑controlled bots
│ └─ … (other models)
├─ utils/
│ ├─ JWT/ # Token generation & validation
│ ├─ System_Prompt/ # Prompt‑related helpers
│ ├─ helper/ # Misc. utilities (API‑key, sanitising, etc.)
│ ├─ BackgroundWorker/ # BullMQ workers (e.g., ScraperWorker)
│ └─ types/ # Shared TypeScript types
├─ Database/ # DB connection wrappers (PostgreSQL & MongoDB)
├─ Redis/ # Redis client singleton
├─ Middleware/ # Auth & access‑control middlewares
├─ Schedulers/ # Periodic jobs (e.g., DeleteBotScheduler)
├─ Static/ # Assets (logo, etc.)
├─ index.ts # Application entry point
└─ tsconfig.json
The BackgroundWorker directory houses the ScraperWorker which processes the scrapeWebsite queue, normalises HTML, creates text chunks, generates embeddings, stores them, and notifies users via email.
| Tool | Minimum Version |
|---|---|
| Node | 18.x |
| npm | 9.x (or Yarn 1.22+) |
| PostgreSQL | 13 |
| MongoDB | 5 |
| Redis | 6 |
| Docker (optional) | 20.10+ |
| Playwright browsers | (installed via npm i -D playwright) |
# 1️⃣ Clone the repository
git clone https://github.com/GURUDAS-DEV/NOVA-BOT-STUDIO-BACKEND.git
cd NOVA-BOT-STUDIO-BACKEND
# 2️⃣ Install dependencies
npm ci # or `yarn install`
# 3️⃣ Create environment file
cp .env.example .env
# Edit .env with your DB credentials, JWT secrets, Resend API key, etc.- PostgreSQL – Tables are created automatically on first run via the
pgclient. - MongoDB – Collections are created on demand; ensure the database exists.
npm run dev # uses ts-node-dev for hot‑reloadingThe API will be available at http://localhost:9000.
npm run build
npm start# Build the image
docker build -t nova-bot-studio-backend .
# Run the container (exposes port 9000)
docker run -d -p 9000:9000 \
--env-file .env \
--restart unless-stopped \
nova-bot-studio-backend| Variable | Description | Example |
|---|---|---|
PORT |
Port the Express server listens on | 9000 |
POSTGRES_URI |
PostgreSQL connection string | postgresql://user:pass@localhost:5432/nova |
MONGODB_URI |
MongoDB connection string | mongodb://localhost:27017/nova |
REDIS_URL |
Redis connection URL | redis://localhost:6379 |
JWT_ACCESS_SECRET |
Secret for access tokens | supersecretaccess |
JWT_REFRESH_SECRET |
Secret for refresh tokens | supersecretrefresh |
CORS_ORIGINS |
Comma‑separated list of allowed origins | http://localhost:3000 |
API_KEY_SALT_ROUNDS |
Bcrypt salt rounds for API‑key hashing | 12 |
SESSION_TTL_SECONDS |
Redis TTL for session entries | 86400 |
MAX_POOL_SIZE |
PostgreSQL connection pool size | 20 |
LOG_LEVEL |
Application log level (error, warn, info, debug) |
info |
RESEND_MAIL_API_KEY |
API key for Resend (email notifications) | re_1234567890abcdef |
BULLMQ_CONNECTION_URL |
URL for BullMQ Redis connection (defaults to REDIS_URL) |
redis://localhost:6379 |
Example .env snippet
PORT=9000
POSTGRES_URI=postgresql://postgres:password@localhost:5432/nova
MONGODB_URI=mongodb://localhost:27017/nova
REDIS_URL=redis://localhost:6379
JWT_ACCESS_SECRET=yourAccessSecret
JWT_REFRESH_SECRET=yourRefreshSecret
CORS_ORIGINS=http://localhost:3000
API_KEY_SALT_ROUNDS=12
SESSION_TTL_SECONDS=86400
MAX_POOL_SIZE=20
LOG_LEVEL=info
RESEND_MAIL_API_KEY=your-resend-api-keycurl http://localhost:9000/ping
# => {"message":"Pong!"}import axios from 'axios';
await axios.post('http://localhost:9000/api/auth/register', {
email: 'dev@example.com',
password: 'StrongP@ssw0rd',
});
const { data: loginRes } = await axios.post(
'http://localhost:9000/api/auth/login',
{
email: 'dev@example.com',
password: 'StrongP@ssw0rd',
},
{ withCredentials: true }
);
const { accessToken, refreshToken } = loginRes;await axios.get('http://localhost:9000/api/bot/', {
headers: { Cookie: `refreshToken=${refreshToken}` },
withCredentials: true,
});import { Queue } from 'bullmq';
import { connection } from '../utils/BullMQ/BullMq';
const scrapeQueue = new Queue('scrapeWebsite', { connection });
await scrapeQueue.add('scrape', {
url: 'https://example.com/article',
userId: '64a1f2c9e5b6c8d1f0a2b3c4',
botId: '64b2d3e4f5a6b7c8d9e0f1a2',
});The ScraperWorker will pick up the job, process it, store embeddings, and send a success/failure email to the address configured in RESEND_MAIL_API_KEY.
await axios.post(
'http://localhost:9000/api/bot/createControlledBot',
{
name: 'SupportBot',
userId: '64a1f2c9e5b6c8d1f0a2b3c4',
platform: 'Website',
type: 'CONTROLLED',
},
{
headers: { Cookie: `refreshToken=${refreshToken}` },
withCredentials: true,
}
);await axios.post(
'http://localhost:9000/api/bot/updateControlledBotStyle',
{
botId: '64b2d3e4f5a6b7c8d9e0f1a2',
style: {
primaryColor: '#4A90E2',
bubbleShape: 'rounded',
welcomeMessage: 'Hello! How can I help you today?'
}
},
{
headers: { Cookie: `refreshToken=${refreshToken}` },
withCredentials: true,
}
);await axios.post(
'http://localhost:9000/api/bot/website/communicate',
{
botId: '64b2d3e4f5a6b7c8d9e0f1a2',
message: 'What are your business hours?'
},
{
headers: { Cookie: `refreshToken=${refreshToken}`