Setup

Setup & Installation

← Back to README

Prerequisites

1. Clone & Install

Create a virtual environment (recommended):

Install Python dependencies:

Linux note: Some packages (numpy, tokenizers) may need to compile from source if a pre-built wheel is unavailable for your Python version. Install a C/C++ compiler first if you hit build errors:

TERMINAL

sudo dnf install gcc gcc-c++ # Fedora/RHEL sudo apt install build-essential # Debian/Ubuntu

uv users: uv pip install -r requirements.txt also works and is faster. The tokenizers>=0.20.0 pin in requirements.txt avoids a known broken build in the 0.19.x series.

2. Environment Variables

Create a .env file in the project root:

Security note: Environment variables always take priority over config.json for secret fields (*_key, orpheus_endpoint). If an env var is set and non-empty, the config.json value is silently skipped — even if it is also non-empty. A non-empty config.json value is only used as a fallback when the env var is not set.

3. OBS Studio Setup

1. Open OBS Studio.
2. Go to Tools → WebSocket Server Settings.
3. Enable the WebSocket server (default port: 4455).
4. Set a password and note it down — you'll need it in config.json.

Recommended OBS Sources

Set obs_avatar_source, obs_text_source in config.json to match your source names.

4. Avatar Images / Videos

Populate data/pngs/ with avatar assets organized by mood. Each mood folder contains two files: an idle and a talking state.

The obs_source_type config key controls whether OBS uses an image source (image) or a media source (media).

Then map the files in config.json under the avatar_map key:

5. Audio Device Setup

ProjectBEA outputs audio to a specific device ID. To list available devices:

Find the ID of your virtual cable (e.g. CABLE Input on Windows) and set audio_device_id in config.json.

6. Discord Bot Setup (optional)

Install Node.js dependencies for the bot:

Set your Discord token in .env or in config.json under skills.discord.token.

In config.json, also set:

• skills.discord.enabled: true
• skills.discord.target_channel: the voice channel name where Bea should listen/speak

Discord Skill Details →

7. Kokoro TTS Setup (optional)

Kokoro runs entirely locally — no API key required.

The engine automatically downloads the model files on first launch if they are missing:

• kokoro-v0_19.onnx (~95 MB)
• voices.bin (~30 MB)

No manual steps needed. Just set tts_provider to kokoro in config.json and start the engine. The download happens once and is cached in the project root.

To use a different path, update kokoro_model and kokoro_voices_file in config.json.

8. Orpheus TTS Setup (optional)

Orpheus is a high-quality expressive voice API hosted on Baseten. It requires a manual deployment step before use:

1. Create an account at baseten.co.
2. From the Baseten model library, find and deploy the Orpheus TTS model to your workspace.
3. Wait for the deployment to become active (a few minutes).
4. Copy the Endpoint URL shown in your deployment dashboard (format: https://model-xxxxxxxx.api.baseten.co/environments/production/predict).
5. Copy your API key from the Baseten account settings.
6. Add both to your .env:

Security note: ORPHEUS_ENDPOINT is treated as a secret — it is read from the environment variable and is never saved to config.json, even if set via the web dashboard.

Then in config.json set tts_provider to orpheus and orpheus_voice to one of: zoe, tara, leo, leah.

Note: Baseten bills per inference. Orpheus is the most expensive TTS option — use EdgeTTS or Kokoro for testing.

8. Build the Frontend (required for Web Dashboard)

Before using --web, you must install the Node.js dependencies and build the frontend once. The Python server serves the compiled output from src/web/frontend/dist/ — if that folder doesn't exist, the dashboard will not load.

You only need to repeat this step when the frontend source code changes.

Running the Engine

CLI mode (interactive terminal)

Type messages at the You > prompt. Type exit to quit.

Web Dashboard mode

Opens the FastAPI server at http://localhost:8000. The React frontend (built in step 8) is served from the same port at /.

CLI argument overrides

Any config value can be overridden at launch without editing config.json:

Full CLI & Config Reference →

Running the Frontend in Development Mode

This is only needed when actively developing the frontend. Instead of using the built dist/, Vite serves the source files with hot-reload at a separate port.

1. Start the backend first (in one terminal):

2. Then start the Vite dev server (in a second terminal):

The Vite dev server starts at http://localhost:5173. The frontend makes direct API calls to http://localhost:8000 — no proxy is configured in vite.config.js. If you change the backend port, update the API base URL in the frontend source accordingly.

Note: For normal use you do not need the dev server — just build once with npm run build (step 8) and use python main.py --web.

Troubleshooting