YouBeAnything

YouBeAnything is a temporal pattern observer that learns about your behavioral patterns through your computer usage. It grounds its behavioral characterization on activity theory to build a comprehensive model of a user's behavior over time.

Requirements

• Python 3.9 or higher
• An API key for either:
  • OpenAI (for GPT models like gpt-4o-mini, gpt-4o, etc.)
  • Google Gemini (for Gemini models like gemini-1.5-pro, gemini-1.5-flash, etc.)
• Platform-specific permissions:
  • macOS: Accessibility permissions for Terminal (System Settings → Privacy & Security → Accessibility)
  • Linux/GNOME: Appropriate screen capture and input monitoring permissions

Installation

Quick Start (Init/Run)

If you just want to pull the repo and run the Electron app with a single command:

./run.sh

This will install Miniconda (if needed), create the tempo conda env, install Python and Electron dependencies, and then launch the app. You can also run setup only with:

./init.sh

1. Clone the Repository

git clone https://github.com/GeneralUserModels/YouBeAnything.git
git checkout shardul/refactor_ubianything
cd YouBeAnything

2. Set Up Python Environment

You can use either conda or Python's venv.

Option A: Using conda

# Create a conda environment named 'tempo'
conda create -n tempo python=3.12.9
conda activate tempo

Option B: Using venv

# Create a virtual environment named 'tempo'
python3 -m venv tempo

# Activate the virtual environment
# On macOS/Linux:
source tempo/bin/activate

# On Windows (if applicable):
# tempo\Scripts\activate

3. Install Dependencies

Install the package in editable mode. Platform-specific dependencies (macOS or Linux) will be installed automatically:

pip install -e .

4. Configure API Keys

You need to set up your API key for either OpenAI or Google Gemini. You can do this via environment variables:

Option A: Using OpenAI

export OPENAI_API_KEY="your-openai-api-key-here"

You can get an API key from OpenAI's website.

Option B: Using Google Gemini

export GOOGLE_API_KEY="your-google-api-key-here"

You can get an API key from Google AI Studio.

Option C: Using a .env File (Optional)

You can also create a .env file in the project root (this file is gitignored):

# .env
OPENAI_API_KEY=your-openai-api-key-here
# OR
GOOGLE_API_KEY=your-google-api-key-here

# Optional: Override default model
MODEL_NAME=gpt-4o-mini

Then install python-dotenv (already included in dependencies) and the environment variables will be automatically loaded.

5. Platform-Specific Setup

macOS

1. Grant Accessibility Permissions:

   • Go to System Settings → Privacy & Security → Accessibility
   • Enable the Terminal (or your terminal app) option
   • This is required for Tempo to observe your interactions

2. The [macos] extra includes pyobjc-framework-Quartz for macOS-specific screen capture capabilities (which should already be installed from the platform specific set up already).

Linux (GNOME)

1. Ensure you have the necessary permissions for screen capture and input monitoring
2. Install GNOME-specific dependencies if needed
3. The [linux] extra includes pyscreenshot and evdev for Linux support

Usage

Starting Tempo

Start the Tempo observer to begin monitoring your computer usage:

tempo start

By default, this uses:

• Platform: macOS (use --platform gnome for Linux/GNOME)
• Model: gpt-4o-mini (or the model specified in MODEL_NAME environment variable)
• API: Detects your API key from environment variables (OPENAI_API_KEY or GOOGLE_API_KEY)

Command-Line Options

tempo start [OPTIONS]

Options:

• --model MODEL: Specify the model to use (default: gpt-4o-mini or $MODEL_NAME)
  • OpenAI models: gpt-4o-mini, gpt-4o, gpt-4-turbo, etc.
  • Gemini models: gemini-1.5-pro, gemini-1.5-flash, etc.
• --platform PLATFORM: Platform backend (macos or gnome, default: macos)
• --api-base URL: API base URL for OpenAI-compatible servers (e.g., vLLM deployments)
• --api-key KEY: Override API key (defaults to $TEMPO_LM_API_KEY, $OPENAI_API_KEY, or $GOOGLE_API_KEY)
• --debug: Enable debug logging

Examples

Using OpenAI with default settings:

tempo start

Using Gemini:

export GOOGLE_API_KEY="your-key"
tempo start --model gemini-1.5-flash

Using Linux/GNOME:

tempo start --platform gnome

Using a custom API endpoint (e.g., local vLLM server):

tempo start --api-base http://localhost:8000/v1 --model Qwen/Qwen2-VL-7B-Instruct

Debug mode:

tempo start --debug

Querying Your Data

Search for activities, actions, or operations:

tempo query "email"                    # Search for anything related to "email"
tempo query "coding" --type activity   # Search only activities
tempo query "browser" --limit 20       # Return up to 20 results

Options:

• QUERY_TEXT: The search query (required)
• --type TYPE: Filter by entity type (operation, action, or activity)
• --limit N: Maximum number of results to return (default: 10)

How It Works

Tempo operates through a pipeline of observations:

1. Observation → Operations: Screenshots are analyzed to extract discrete operations (e.g., "clicked button X", "typed in field Y")
2. Operations → Actions: Related operations are grouped into actions (e.g., "composed email", "edited document")
3. Actions → Activities: Actions are inferred into higher-level activities (e.g., "email management", "coding session")

All data is stored in a SQLite database located at ~/.cache/tempo/tempo.db. The system automatically:

• Buffers operations before processing
• Groups related operations based on temporal proximity and semantic similarity
• Continuously learns and updates your behavioral model

Environment Variables

Tempo respects the following environment variables:

• OPENAI_API_KEY: API key for OpenAI (for GPT models)
• GOOGLE_API_KEY: API key for Google Gemini (for Gemini models)
• TEMPO_LM_API_KEY: Override API key for Tempo (takes precedence)
• TEMPO_LM_API_BASE: Override API base URL for Tempo
• OPENAI_API_BASE: API base URL for OpenAI-compatible servers
• MODEL_NAME: Default model name (used if --model is not specified)

Data Storage

Tempo stores all data in:

• Database: ~/.cache/tempo/tempo.db (SQLite)
• State: ~/.cache/tempo/state.json (saves buffered operations and job timestamps)

The database and state are automatically created on first run. To reset, simply delete these files and restart Tempo.

Troubleshooting

"Database is locked" errors

• Tempo uses SQLite with WAL mode and locking mechanisms to prevent this. If you encounter this, ensure only one Tempo instance is running.

API key errors

• Verify your API key is correctly set: echo $OPENAI_API_KEY or echo $GOOGLE_API_KEY
• For Gemini, ensure GOOGLE_API_KEY is set (it's required, unlike OpenAI where a dummy key can work for vLLM)

Permission errors (macOS)

• Ensure Terminal has Accessibility permissions: System Settings → Privacy & Security → Accessibility
• You may need to restart Terminal after granting permissions

Model not found errors

• Ensure you're using a valid model name for your chosen provider
• OpenAI models: Must start with gpt-, o1-, etc.
• Gemini models: Must start with gemini- or claude-

Development

Running Tests

pip install -e ".[test]"
pytest

Project Structure

• tempo/: Main package
  • cli.py: Command-line interface
  • system.py: System initialization and management
  • providers.py: AI model providers (OpenAI, Gemini)
  • pipelines/: Processing pipelines (observation → operation → action → activity)
  • observers/: Screen and interaction observers
  • db.py: Database management
  • store.py: Data storage layer
  • os_backends.py: Platform-specific backends