Setup & Installation

Recommended: automatic install#

macOS / Linux

curl -LsSf https://raw.githubusercontent.com/emqnuele/projectBEA/main/install.sh | bash

Windows (PowerShell)

irm https://raw.githubusercontent.com/emqnuele/projectBEA/main/install.ps1 | iex

One command. It clones the repo if needed, installs uv if you don't have it, pulls the dependencies, builds the dashboard and the Discord bot when Node is available, asks you five questions and downloads the two models that run on your own machine.

What you need beforehand: git (only to download the repo — the script tells you how to install it if missing) and Node.js 20+ if you want the dashboard and Discord voice. You do not need to install Python yourself: uv reads requires-python from pyproject.toml and downloads a matching interpreter automatically.

Already cloned the repo? Run the script from inside it, or skip it entirely — uv run bea --setup runs the same wizard (make setup if you have Make):

./install.sh       # macOS / Linux, from inside the repo
.\install.ps1      # Windows, from inside the repo
# or just the wizard:
uv run bea --setup # make setup

The default profile is Solo chat: the dashboard and her voice, one API key, nothing else. No OBS, no Discord bot, no Minecraft server, no virtual audio cable. Those are separate profiles you can pick later, or turn on one at a time from the Abilities screen.

If the script worked, skip to Running the Engine. The rest of this page is the same steps by hand.

Prerequisites#

Below is the manual path, step by step. The script above does all of it for you.

You do not need to install Python yourself. uv reads requires-python from pyproject.toml and downloads a matching interpreter automatically. Install uv with: curl -LsSf https://astral.sh/uv/install.sh | sh (macOS/Linux) or powershell -c "irm https://astral.sh/uv/install.ps1 | iex" (Windows).

1. Clone & Install#

git clone https://github.com/emqnuele/projectBEA.git
cd projectBEA

Install all Python dependencies into a managed virtual environment (.venv):

uv sync                    # core dependencies
uv sync                    # everything the engine needs

That's it — uv creates .venv, pins every dependency from uv.lock, and is fully reproducible. There is no need to activate the venv manually; prefix commands with uv run (e.g. uv run bea).

If you have make available, the same is wrapped in convenient targets:

make install        # uv sync
make node           # uv run bea --install-node (dashboard + discord bot)
make help           # list every target

Linux note: Some packages (numpy, tokenizers) may need to compile from source if a pre-built wheel is unavailable for your Python version. Install a C/C++ compiler first if you hit build errors:

sudo dnf install gcc gcc-c++   # Fedora/RHEL
sudo apt install build-essential  # Debian/Ubuntu

2. Environment Variables#

Create a .env file in the project root:

# LLM providers — add the ones you plan to use
OPENROUTER_API_KEY=sk-or-...   # recommended: routes to any model
OPENAI_API_KEY=sk-...
GROQ_API_KEY=gsk_...
GOOGLE_API_KEY=AIza...          # Google AI Studio: Gemini with a free tier
ANTHROPIC_API_KEY=sk-ant-...    # Claude, called directly

# Local models need no key. Ollama answers at http://localhost:11434/v1,
# LM Studio at http://localhost:1234/v1 — pick the runner in `bea --setup`.
# Only set these for a remote endpoint or one that wants a key.
# LOCAL_API_KEY=""
# LOCAL_BASE_URL="http://localhost:11434/v1"

# TTS — only if using Orpheus
ORPHEUS_API_KEY=...
ORPHEUS_ENDPOINT=https://model-xxxxxxxx.api.baseten.co/environments/production/predict

# Discord — only if using the Discord skill
DISCORD_TOKEN=...

# Telegram — only if using the Telegram skill
TELEGRAM_TOKEN=...

# Twitch — only needed to WRITE in chat; reading works anonymously
TWITCH_OAUTH_TOKEN=oauth:...

# Donations — shared secret checked on the webhook. Set this before exposing the server
DONATION_SECRET=...

# Hugging Face — optional, and no account is needed to run anything here. The
# whisper and embedding models are public. A token only raises the anonymous
# rate limit, which is what makes the first download crawl (or get refused) on
# a shared or office connection. https://huggingface.co/settings/tokens
HF_TOKEN=hf_...

# Logging — optional, defaults to INFO
LOG_LEVEL=DEBUG   # set to DEBUG to see verbose output (OBS, TTS, audio playback details)

Security note: Environment variables always take priority over config.json for secret fields (*_key, orpheus_endpoint). If an env var is set and non-empty, the config.json value is silently skipped — even if it is also non-empty. A non-empty config.json value is only used as a fallback when the env var is not set.

3. OBS Studio Setup#

1. Open OBS Studio.
2. Go to Tools → WebSocket Server Settings.
3. Enable the WebSocket server (default port: 4455).
4. Set a password and note it down — you'll need it in config.json.

Recommended OBS Sources#

Set obs_avatar_source, obs_text_source in config.json to match your source names.

4. Avatar Images / Videos#

Populate data/pngs/ with avatar assets organized by mood. Each mood folder contains two files: an idle and a talking state.

data/pngs/
├── neutral/
│   ├── idle.mp4       (or .png, .gif)
│   └── talking.mp4
├── angry/
│   ├── idle.mp4
│   └── talking.mp4
├── happy/  sad/  surprised/  disgusted/  bored/   (same structure)

The obs_source_type config key controls whether OBS uses an image source (image) or a media source (media).

Then map the files in config.json under the avatar_map key:

"avatar_map": {
  "neutral": { "idle": "data/pngs/neutral/idle.mp4", "talking": "data/pngs/neutral/talking.mp4" },
  "angry":  { "idle": "data/pngs/angry/idle.mp4",  "talking": "data/pngs/angry/talking.mp4"  }
}

With stage.png_render: "stage" the same pictures are drawn in the browser source at http://127.0.0.1:8000/stage instead of an OBS image source: the mouth follows her voice, and each mood may add a talking_closed and a blink picture.

A 3D model instead#

Choose 3D model under Settings → Stream and add http://127.0.0.1:8000/stage to OBS as a Browser Source. The library on that page downloads the free default model and its idle motion in one click, or takes your own .vrm by drag and drop (VRoid Studio exports work, VRM 1.0 or 0.x). From a terminal:

make model                                    # the default model and idle motion
uv run python tools/fetch_model.py --list     # everything in the catalog, with licences

Models live in data/models/ and motions in data/clips/; both are gitignored.

5. Audio Device Setup#

Her voice plays on the system default output unless you pick one. For a stream, pick the virtual cable OBS listens to (e.g. CABLE Input on Windows) in Settings → Voice → Output device, or set its name in config.json:

"audio_device": "CABLE Input (VB-Audio Virtual Cable)"

The choice is stored by name, so plugging in a monitor or a headset later does not move her to another device. To list the names from a terminal:

uv run python -c "from src.core.expression.player import output_devices; print(*[d['name'] for d in output_devices()], sep='\\n')"

A bluetooth headset adds roughly 150-250 ms before the room hears her; the mouth and the face wait for it. If the same headset is also the microphone the dashboard listens on, many headsets drop to call quality while it is open — pick another microphone in the browser, or a wired output.

6. Discord Bot Setup (optional)#

Install Node.js dependencies for the bot:

cd src/core/skills/voice/bot
npm install

Set your Discord token in .env or in config.json under skills.discord.token.

Then toggle the skill on from the dashboard's Skills page, or set skills.discord.enabled: true before starting.

Discord Skill Details →

6b. Telegram & Twitch (optional)#

Neither needs a subprocess — both run in-process.

Telegram: create a bot with @BotFather, put the token in .env as TELEGRAM_TOKEN, set skills.telegram.owner_id to your own user id, and leave allowed_chats empty to let her read every chat she is added to. Details →

Twitch: set skills.twitch.channel and toggle it on — reading needs no credentials at all. Only add TWITCH_OAUTH_TOKEN and skills.twitch.nick if you want her to type in chat. Details →

6c. Web search & reading (optional)#

Nothing to install and no key to create: the keyless search works out of the box. Toggle Web on from the dashboard's Abilities page and she can search and read pages.

For reliability by contract rather than by goodwill, paste a key in Settings → Web — Brave or Tavily — or point searxng_url at your own instance. It lands in .env as BRAVE_API_KEY / TAVILY_API_KEY, never in config.json, and applies to the next search. Details →

7. Kokoro TTS Setup (optional)#

Kokoro runs entirely locally — no API key required.

The engine automatically downloads the model files on first launch if they are missing:

• kokoro-v0_19.onnx (~95 MB)
• voices.json (~30 MB)

No manual steps needed. Just set tts_provider to kokoro in config.json and start the engine. The download happens once and is cached in the project root.

To use a different path, update kokoro_model and kokoro_voices_file in config.json.

7b. Local Whisper Setup (optional)#

The same deal for her ears. faster_whisper transcribes on this machine, so voice input needs no key and no account, and nothing you say is uploaded anywhere to be turned into text.

Set stt_provider to faster_whisper in config.json — or pick Local Whisper in the wizard or on the dashboard's Hearing page — and choose a size:

The weights download into data/models/whisper, which is gitignored — as a Hugging Face cache, the same layout WhisperModel(download_root=...) reads, so the copy the wizard fetches is the copy the engine loads. The wizard offers to fetch them at the end of setup, with a progress bar; decline it, or skip the wizard, and they arrive on first use instead. In that case her first startup says so in the log and reports the download as it lands, rather than looking hung for a few hundred megabytes. uv run bea --doctor allows for it either way and will not call it a failure.

Nothing here needs a Hugging Face account. A token is worth having only for the rate limit — on a shared or office IP it is the difference between a download that crawls and one that does not, and a refused download says the same thing. bea --setup asks for one, optionally, and writes it to .env.

On a machine with an NVIDIA GPU, set faster_whisper_device to cuda — this needs the CUDA and cuDNN runtimes on the system, which uv sync does not install. Leave it on auto if you would rather not deal with that; the CPU path runs int8 and is fast enough for small.

STT module →

7c. Local models (optional, and the interesting one)#

She can think on this machine too — no key, no account, no bill, and nothing you say leaves the room. Install one runner and pull a model:

Ollama (ollama.com):

ollama pull qwen3:8b

LM Studio (lmstudio.ai): download a model in the app, then start its server (Developer → Status: Running, port 1234).

Then pick Local models in bea --setup, or set it by hand:

"llm_provider": "local",
"local_base_url": "http://localhost:11434/v1",
"local_model": "qwen3:8b"

Point local_base_url at http://localhost:1234/v1 for LM Studio. The two runners can even share her: put one model in models.mind and a cheaper one in models.background, or mix a local model with a cloud one — when the laptop sleeps, the pool falls over to the key that is set.

Two things to know. Every model in mind must support tool calling — she speaks only through tools, so a model without them never says a word, and the doctor tells you exactly that. And a local model is slower than Groq: she still answers in time, but the background pool is where small local models shine — the diary and the dreamer never needed a frontier model in the first place.

LLM modules →

8. Orpheus TTS Setup (optional)#

Orpheus is a high-quality expressive voice API hosted on Baseten. It requires a manual deployment step before use:

1. Create an account at baseten.co.
2. From the Baseten model library, find and deploy the Orpheus TTS model to your workspace.
3. Wait for the deployment to become active (a few minutes).
4. Copy the Endpoint URL shown in your deployment dashboard (format: https://model-xxxxxxxx.api.baseten.co/environments/production/predict).
5. Copy your API key from the Baseten account settings.
6. Add both to your .env:

ORPHEUS_API_KEY=your-baseten-api-key
ORPHEUS_ENDPOINT=https://model-xxxxxxxx.api.baseten.co/environments/production/predict

Security note: ORPHEUS_ENDPOINT is treated as a secret — it is read from the environment variable and is never saved to config.json, even if set via the web dashboard.

Then in config.json set tts_provider to orpheus and orpheus_voice to one of: zoe, tara, leo, leah.

Note: Baseten bills per inference. Orpheus is the most expensive TTS option — use EdgeTTS or Kokoro for testing.

8. Build the Frontend (required for Web Dashboard)#

Before using --web, you must install the Node.js dependencies and build the frontend once. The Python server serves the compiled output from src/web/frontend/dist/ — if that folder doesn't exist, the dashboard will not load.

cd src/web/frontend
npm install
npm run build
cd ../../..   # back to project root

You only need to repeat this step when the frontend source code changes.

Running the Engine#

CLI mode (interactive terminal)#

uv run bea          # or: make run

Type messages at the You > prompt. Type exit to quit.

Web Dashboard mode#

uv run bea --web    # or: make web  (also builds the javascript)

Opens the FastAPI server at http://localhost:8000. The React frontend (built in step 8) is served from the same port at /.

The API has no authentication, so the server binds to 127.0.0.1 by default. Exposing it on the network is an explicit opt-in: uv run bea --web --host 0.0.0.0. Do that only behind something that authenticates, and set DONATION_SECRET first.

CLI argument overrides#

Any config value can be overridden at launch without editing config.json:

uv run bea \
  --llm-provider openrouter \
  --openrouter-model openai/gpt-4o-mini \
  --tts-provider kokoro \
  --device-id 22 \
  --web

Full CLI & Config Reference →

Running the Frontend in Development Mode#

This is only needed when actively developing the frontend. Instead of using the built dist/, Vite serves the source files with hot-reload at a separate port.

1. Start the backend first (in one terminal):

uv run bea --web

2. Then start the Vite dev server (in a second terminal):

cd src/web/frontend
npm install
npm run dev

The Vite dev server starts at http://localhost:5173. There is no proxy in vite.config.js: the frontend calls the backend directly through API_BASE (src/web/frontend/src/api.js), which is http://localhost:8000 in dev and the page's own origin in a build. To point the dev server somewhere else, set VITE_API_BASE rather than editing the source.

http://localhost:5173 is already in the backend's CORS allowlist. For any other origin, set BEA_ALLOWED_ORIGINS (comma-separated) before starting the brain.

Note: For normal use you do not need the dev server — just build once with npm run build (step 8) and use uv run bea --web.

Staying up to date#

make update          # or: uv run bea --update

Not git pull. The prompts under data/prompts/ are tracked and meant to be edited, so a pull either refuses to run or writes conflict markers into the file her personality is read from. --update backs up your prompts, config and database, fast-forwards, then puts your edits back through a three-way merge — so an edited operating.md keeps your changes and receives the engine's. Any file it cannot merge is left exactly as you wrote it, with the new version dropped beside it as <name>.new.

It also runs uv sync and rebuilds the dashboard when the update touched them. src/web/frontend/dist/ is gitignored, so a pull without that rebuild leaves the old dashboard on screen.

The dashboard's Maintenance screen does the same thing with a button, shows the changelog before you commit to it, and gives you a side-by-side diff for anything that needs a decision.

Updating in place needs git — it is the only feature that does. Without it she runs exactly the same and bea --doctor tells you what you are missing; nothing installs it for you, because doing so needs root.

How it works, and what it refuses to do →

Troubleshooting#

The fastest way to figure out what is wrong is to run the built-in diagnostic:

uv run bea --doctor    # or: make doctor

Fifteen checks, run in the order the pieces depend on each other and stopped at the first blocking failure — there is no point testing the voice when there is no config file. Every failure carries the exact command that fixes it. The exit code is non-zero when something is blocking, so it works in a script.

The same sequence runs from Maintenance in the dashboard, findings streamed as they land. It builds the voice, transcribes a line and calls the mind, so it costs a handful of provider requests and never runs on its own.