GPU-RELAY

$ # Use a specific tier
curl -H "Authorization: Bearer $API_KEY" \
     -H "Content-Type: application/json" \
     -d '{"model":"llm-architecture","messages":[{"role":"user","content":"review this code"}]}' \
     http://localhost:8000/v1/chat/completions

$ # Auto-route — router picks tier based on complexity
curl -H "Authorization: Bearer $API_KEY" \
     -H "Content-Type: application/json" \
     -d '{"model":"llm-auto","messages":[{"role":"user","content":"what is 2+2?"}]}' \
     http://localhost:8000/v1/chat/completions

$ # Force tier via header (bypasses all routing logic)
curl -H "Authorization: Bearer $API_KEY" \
     -H "X-Tier: simple" \
     -H "Content-Type: application/json" \
     -d '{"model":"llm-auto","messages":[{"role":"user","content":"hello"}]}' \
     http://localhost:8000/v1/chat/completions

$ # Force local GPU — bypasses cloud regardless of tier config
curl -H "Authorization: Bearer $API_KEY" \
     -H "Content-Type: application/json" \
     -d '{"model":"llm-local","messages":[{"role":"user","content":"hello"}]}' \
     http://localhost:8000/v1/chat/completions

# Direct (no shortcut installed)
$ docker compose exec bridge python -m cli.llm_ctl <command>

$ llmctl users add <email>                              # create user (prompts for password)
$ llmctl users set-password <email>                     # reset password
$ llmctl users budget <email> --usd 50                  # set monthly spend cap
$ llmctl users credit-add <email> --usd 20              # add prepaid credit
$ llmctl users tiers <email>                            # show allowed tiers
$ llmctl users tiers <email> --set simple               # lock to one tier
$ llmctl users tiers <email> --set simple,architecture  # allow two tiers
$ llmctl users tiers <email> --set all                  # remove restriction
$ llmctl users deactivate <email>                       # soft-delete user
$ llmctl users list                                     # list all users

$ llmctl users keys-add <email> --label "open-webui"   # create key (shown ONCE)
$ llmctl users keys-list <email>                       # list active keys
$ llmctl users keys-revoke <key_id>                    # revoke by ID
$ llmctl users reset-key <email> [--label "name"]      # revoke all + issue fresh key

$ llmctl users add <email> --sync-openwebui        # create bridge user + matching OW account
$ llmctl users keys-add <email> --sync-pipeline    # create key + update pipeline user_key_map

$ llmctl users storage <email>                          # show policy + registered volume keys
$ llmctl users storage <email> --policy use_env         # use env volume if no user key (default)
$ llmctl users storage <email> --policy stateless       # always launch without volume
$ llmctl users storage <email> --policy block           # require user volume; reject otherwise
$ llmctl users storage <email> --allow-env              # allow shared RUNPOD_NETWORK_VOLUME_ID
$ llmctl users storage <email> --no-allow-env           # prevent shared volume for this user

$ llmctl users local-access <email> --allow   # enable llm-local for this user
$ llmctl users local-access <email> --deny    # disable llm-local for this user

$ llmctl pods ls [--status ready]             # list pods
$ llmctl pods kill <pod_id>                   # terminate pod immediately
$ llmctl start --tier architecture            # prewarm a pod
$ llmctl bills run --month 2026-05            # generate invoices
$ llmctl bills show <email> --month 2026-05   # per-user invoice + breakdown

$ llmctl models [--user-type personal]        # tier table with effective $/hr
$ llmctl status [--tier architecture]         # active pods + running cost
$ llmctl budget [--email u@example.com]       # spend vs cap progress bars
$ llmctl costs [--month 2026-05]              # per-tier cost breakdown
$ llmctl gain  [--month 2026-05]              # savings vs GPT-4o equivalent

# Add to .env:
GATEWAY_BRIDGE_URL=http://bridge:8000   # internal; bridge is a sibling service
GATEWAY_PORT=8080

$ docker compose -f docker-compose.yml -f docker-compose.gateway.yml up -d gateway

# docker/docker-compose.gateway.yml (no main stack needed)
GATEWAY_BRIDGE_URL=https://your-bridge.example.com
GATEWAY_PORT=8080

$ docker compose -f docker/docker-compose.gateway.yml up -d

# .env — sync settings
OPENWEBUI_ADMIN_EMAIL=admin@example.com
OPENWEBUI_ADMIN_PASSWORD=<ow-admin-password>
PIPELINES_API_KEY=<from Open WebUI Admin → Pipelines → API key>
PIPELINE_ID=gpu-relay                   # default; match your installed pipeline ID

# Create user + OW account + pipeline mapping in one step:
$ llmctl users add alice@example.com --sync-openwebui
$ llmctl users keys-add alice@example.com --sync-pipeline

# Or via HTTP (returns plaintext key in response):
$ curl -X POST http://localhost:8000/admin/users \
       -H "Authorization: Bearer <admin-key>" \
       -H "Content-Type: application/json" \
       -d '{"email":"alice@example.com","password":"...","sync_openwebui":true,"sync_pipeline":true}'

# 1. Open WebUI → Admin Panel → Tools → Add Tool
#    Paste contents of: pipelines/openwebui_tool.py
#    Set valve: workspace_tools_url = http://workspace-tools:7000

# 2. Start workspace-tools service (already in docker-compose.yml)
$ docker compose up -d workspace-tools

# 3. In any Open WebUI chat: enable the tool via the tool icon, then chat normally

You have access to Workspace Tools. You MUST use them for ALL file and execution tasks —
never simulate, imagine, or describe results.

Rules:
- Writing code: always call write_file with real file content (actual newlines, not \n)
- Testing code: always call run_bash or run_python — never show "expected output"
- Browsing files: always call list_tree or read_file
- If a tool call fails: read the actual error, fix the actual file, retry
- To run a Python FILE: use run_bash with "python3 <path>" — never pass a file path to run_python
- run_python is for inline code snippets only, e.g. run_python("print('hello')")
- File organization: always create new projects under projects/<project-name>/, not the workspace root

CRITICAL — tool call rules:
- Tool calls are NOT Python code. Never write run_bash(...) in a code block.
- Invoke tools directly — do not write code that calls them.
- "I will call run_bash" followed by a code block = violation. Call it, don't narrate it.
- Fake output is output without a tool call. It is always wrong.