md-show/plan.md

---
name: Voice Architecture Plan
overview: "Development-ready English plan for Option B: custom Conan Voice Server (ServerLib) + custom ClientLib app, with function-level APIs, call order, data packing, VoiceRouter algorithm, threading, and phases. Branch ts3-Testing. See also vergleichs.md."
todos:
  - id: create-branch
    content: "Create branch ts3-Testing off current branch; do not touch tsmain / V8 plugin"
    status: pending
  - id: sdk-license-gate
    content: "Phase 0: get TS3 SDK (ClientLib+ServerLib) + evaluation license; build official examples; freeze every [VERIFY] name against serverlib.h/clientlib.h"
    status: pending
  - id: serverlib-bootstrap
    content: "Phase 1: conan-voice-server bootstrap (initServerLib -> createVirtualServer -> loop -> stopVirtualServer -> destroyServerLib)"
    status: pending
  - id: sdk-client-poc
    content: "Phase 2: custom ClientLib app connects (initClientLib -> spawnNewServerConnectionHandler -> createIdentity -> open devices -> startConnection)"
    status: pending
  - id: metadata-transport
    content: "Phase 2/3: game-data via CLIENT_META_DATA (+ flushClientSelfUpdates); confirm server read + change event"
    status: pending
  - id: routing-poc
    content: "Phase 3: ts3server_setClientWhisperList proves A hears B, not C"
    status: pending
  - id: threed-voice
    content: "Phase 4/5: 3D via ClientLib listener/channel attributes fed by server positions"
    status: pending
  - id: write-en-doc
    content: "Keep vergleichs.md + this plan in sync; mirror into docs/ after Phase 0 freezes APIs"
    status: pending
  - id: keep-v8-untouched
    content: "Do not modify REWRITE_PLAN_V8.md / REWRITE_PLAN.md or the V8 plugin on tsmain"
    status: pending
isProject: false
---

# Conan Voice — TeamSpeak 3 SDK / ServerLib Plan (Option B)

**Audience:** developers who need enough detail to start coding after Phase 0.  
**Companion doc:** [`vergleichs.md`](../../vergleichs.md) (Live V8 Plugin SDK vs this plan).  
**Branch:** `ts3-Testing` only. Leave `tsmain` / V8 DLL alone.

> **[VERIFY]** = confirm exact name/signature against `serverlib.h` / `clientlib.h` during Phase 0. Unmarked items are confirmed from TeamSpeak docs/examples.

---

## 0. What we are building (and what we are not)

### Building

A **standalone voice product**:

```text
Conan Voice Client  = our app linking TeamSpeak ClientLib
Conan Voice Server  = our process hosting TeamSpeak ServerLib
```

Users run **our client** (like Mumble). They do **not** run retail TeamSpeak + a plugin.

### Not building (explicit)

- No retail TeamSpeak plugin path for this product
- No REST/WebSocket/cloud API between client and voice server
- No second transport provider until Option B is proven
- No Conan code before whisper-routing PoC succeeds

### Why this is not "the same SDK as V8"

| | Live V8 | This plan |
|---|---|---|
| Product | Client **Plugin** SDK | TeamSpeak 3 SDK (ClientLib + ServerLib) |
| Prefixes | `ts3plugin_*`, `TS3Functions.*` | `ts3client_*`, `ts3server_*` |
| Process | Guest DLL inside retail TS | We own client app + server process |
| Hearability | Local PCM mute/gain | Server `ts3server_setClientWhisperList` |

Proof V8 is Plugin SDK: `sdk/README.md` line 1. Repo has no `ts3client_*` / `ts3server_*`.

---

## 1. System diagram

```mermaid
flowchart TB
  subgraph PlayerPC [Player PC]
    Game[Conan Exiles]
    Provider[IGameDataProvider]
    CState[VoiceClientState]
    CAdapt[TeamSpeakClientAdapter]
    CLib[SDK ClientLib]
    HUD[HUD]
    Effects[AudioEffects PCM]
    Game --> Provider --> CState --> CAdapt --> CLib
    CLib --> Effects
    CLib --> HUD
  end

  subgraph VoiceHost [Voice server host]
    SLib[SDK ServerLib]
    SAdapt[TeamSpeakServerAdapter]
    PM[PlayerManager]
    Prox[ProximityEngine]
    Router[VoiceRouter]
    Auth[Auth]
    Radio[RadioManager]
    Whisper[WhisperManager]
    SLib --> SAdapt
    SAdapt --> PM
    PM --> Prox
    Prox --> Router
    Auth --> PM
    Radio --> Router
    Whisper --> Router
    Router --> SAdapt
  end

  CLib <-->|"voice packets + CLIENT_META_DATA"| SLib
```

**Ownership rules**

- Only `TeamSpeakClientAdapter` calls `ts3client_*`.
- Only `TeamSpeakServerAdapter` calls `ts3server_*`.
- Game logic never includes `clientlib.h` / `serverlib.h` directly.

---

## 2. Server lifecycle — function-by-function

**Files:** `server/src/main.c`, `server/src/ts_server_adapter.c`

### 2.1 Startup (strict order)

```text
main()
  │
  ├─ Config_load("server/config/*.ini")
  ├─ Log_start(...)
  │
  ├─ struct ServerLibFunctions funcs;
  │     memset(&funcs, 0, sizeof(funcs));
  │     funcs.onClientConnected      = on_client_connected;
  │     funcs.onClientDisconnected   = on_client_disconnected;
  │     funcs.onClientMoved          = on_client_moved;
  │     funcs.onServerTextMessageEvent = on_server_text;      // fallback transport
  │     funcs.onClientStartTalkingEvent = on_talk_start;
  │     funcs.onClientStopTalkingEvent  = on_talk_stop;
  │     funcs.onUserLoggingMessageEvent = on_sdk_log;
  │     funcs.onAccountingErrorEvent    = on_accounting_error;
  │
  ├─ err = ts3server_initServerLib(&funcs, LogType_FILE|LogType_CONSOLE, logDir);
  │     if (err != ERROR_ok) -> ts3server_getGlobalErrorMessage + abort
  │
  ├─ keyPair = load_or_create_keypair()     [VERIFY exact create API name]
  │
  ├─ err = ts3server_createVirtualServer(
  │         port,           // e.g. 9987
  │         "0.0.0.0",
  │         "Conan Voice",
  │         keyPair,
  │         maxClients,     // from license / config
  │         &serverID);
  │
  ├─ PlayerManager_init()
  ├─ ProximityEngine_init(cellSize, ranges)
  ├─ VoiceRouter_init(tickHz=5..10)
  │
  └─ run_loop():
        wait SDK events
        every 100-200 ms: VoiceRouter_tick()
```

### 2.2 Confirmed ServerLib call signatures

```c
unsigned int ts3server_initServerLib(
    const struct ServerLibFunctions* functionPointers,
    int usedLogTypes,
    const char* logFileFolder);

unsigned int ts3server_createVirtualServer(
    unsigned int serverPort,
    const char* serverIp,
    const char* serverName,
    const char* serverKeyPair,
    unsigned int serverMaxClients,
    uint64* result);

unsigned int ts3server_stopVirtualServer(uint64 serverID);
unsigned int ts3server_destroyServerLib(void);

unsigned int ts3server_setClientWhisperList(
    uint64 serverID,
    anyID clientID,
    const uint64* channelIDs,   // 0-terminated or NULL
    const anyID* clientIDs);    // 0-terminated or NULL

unsigned int ts3server_getClientList(uint64 serverID, anyID** result);
unsigned int ts3server_getChannelList(uint64 serverID, uint64** result);
void         ts3server_freeMemory(void* pointer);
```

### 2.3 Shutdown order

```text
1. VoiceRouter_stop()          // stop tick, clear lists optional
2. ts3server_stopVirtualServer(serverID)
3. PlayerManager_clear()
4. ts3server_destroyServerLib()
5. Log_stop()
```

### 2.4 Callback → module wiring

| SDK callback | Immediate action | Downstream |
|---|---|---|
| `onClientConnected` | enqueue Connect | `PlayerManager.add(tsClientId)` |
| `onClientDisconnected` | enqueue Disconnect | remove player; `setClientWhisperList(..., NULL, NULL)` |
| `onClientMoved` | update channel id | optional lobby vs ingame policy |
| `onAccountingErrorEvent` | log + fail closed | refuse new sessions / shut down if fatal |
| text message (fallback) | parse discrete cmds | radio join/leave, whisper start/stop |

Keep callbacks short: queue work; do not run O(N) proximity inside the callback.

---

## 3. Client lifecycle — function-by-function

**Files:** `client/src/main.c`, `client/src/ts_client_adapter.c`, `client/src/game_provider_*.c`

### 3.1 Startup (strict order)

```text
main()
  │
  ├─ fill ClientUIFunctions (callbacks)          [VERIFY struct name]
  ├─ ts3client_initClientLib(...)                [VERIFY full arg list]
  ├─ ts3client_spawnNewServerConnectionHandler(0, &scHandlerID)
  ├─ identity = load_identity() or ts3client_createIdentity(&identity); persist
  ├─ ts3client_openPlaybackDevice(scHandlerID, "", "")
  ├─ ts3client_openCaptureDevice(scHandlerID, "", "")
  ├─ ts3client_startConnection(
  │       scHandlerID, identity, host, port,
  │       nickname, NULL /* default channel */, "", serverPassword)
  │
  ├─ wait until onConnectStatusChangeEvent == STATUS_CONNECTION_ESTABLISHED
  │     (or fail on errorNumber != ERROR_ok)
  │
  └─ run_loop():
        GameProvider_poll()           ~20-30 Hz
        publish CLIENT_META_DATA      ~10-20 Hz (hysteresis)
        apply 3D listener + speakers  ~20 Hz
        render HUD
```

### 3.2 Confirmed / expected ClientLib calls

```c
// Connection
ts3client_startConnection(scHandlerID, identity, ip, port,
                          nickname, defaultChannelArray,
                          defaultChannelPassword, serverPassword);
ts3client_stopConnection(scHandlerID, quitMessage);
ts3client_getConnectionStatus(scHandlerID, &status);

// Game state transport
ts3client_setClientSelfVariableAsString(scHandlerID, CLIENT_META_DATA, packed);
ts3client_flushClientSelfUpdates(scHandlerID, NULL);

// 3D [VERIFY exact 3D names if they differ slightly]
ts3client_systemset3DSettings(scHandlerID, distanceFactor, rolloffScale);
ts3client_systemset3DListenerAttributes(scHandlerID, &pos, &forward, &up);
ts3client_channelset3DAttributes(scHandlerID, remoteClientID, &pos);
```

### 3.3 Client callbacks we implement

| Callback | Responsibility |
|---|---|
| `onConnectStatusChangeEvent` | connected / disconnected / fail |
| `onServerStopEvent` | fail closed + HUD |
| `onClientMoveEvent` | channel visibility |
| `onTalkStatusChangeEvent` | HUD speaker highlight |
| `onEditPlaybackVoiceDataEvent` | local reverb/lowpass only (V8 effect patterns) |
| `onCustom3dRolloffCalculationClientEvent` | Conan voice-range curve |
| `onNewChannelEvent` | channel list bookkeeping |

### 3.4 Shutdown order

```text
1. stop game provider thread
2. ts3client_stopConnection(scHandlerID, "leaving")
3. ts3client_destroyServerConnectionHandler(scHandlerID)
4. ts3client_destroyClientLib()
```

---

## 4. End-to-end sequence (connect → route → hear)

```mermaid
sequenceDiagram
  participant G as Conan
  participant C as ClientLib app
  participant S as ServerLib
  participant R as VoiceRouter

  C->>S: ts3client_startConnection
  S-->>C: onConnectStatusChangeEvent(ESTABLISHED)
  S->>S: onClientConnected -> PlayerManager.add

  loop position ~20 Hz
    G->>C: getPlayerState(pos,rot,ids,mode)
    C->>C: pack CLIENT_META_DATA
    C->>S: set + flushClientSelfUpdates
    S->>S: PlayerManager.updatePos
  end

  loop routing 5-10 Hz
    R->>R: spatial hash + overlays
    R->>S: setClientWhisperList(A, {B,C})
    R->>S: setClientWhisperList(D, empty/clear)
  end

  Note over C,S: A talks - packets only to whisper targets
  S-->>C: voice to B and C only
  C->>C: 3D attributes + optional PCM effects
```

---

## 5. Game-data transport (no external API)

### 5.1 Primary: `CLIENT_META_DATA`

| Step | Who | API |
|---|---|---|
| Write | Client | `ts3client_setClientSelfVariableAsString(sc, CLIENT_META_DATA, packed)` |
| Push | Client | `ts3client_flushClientSelfUpdates(sc, NULL)` |
| Read | Server | `ts3server_getClientVariableAsString(...)` **[VERIFY]** |
| Notify | Server | change event if exists; else poll each routing tick **[VERIFY]** |

Limit: **4096 bytes** string. Keep payload tiny.

### 5.2 Proposed packed payload (rewrite of CEPOS idea)

Fixed binary then base64 (example layout — freeze in Phase 2):

```text
offset  size  field
0       4     magic 'CV01'
4       4     seq (uint32)
8       8     conanPlayerId
16      8     characterId
24      4     x float
28      4     y float
32      4     z float
36      4     yaw float
40      1     voiceMode (whisper/normal/shout)
41      1     flags (ptt, muted, ...)
42      2     reserved
= 44 bytes raw  -> ~60 bytes base64
```

Reject stale `seq`, reject identity mismatch after AUTH.

### 5.3 Discrete events (optional fallback)

If metadata is too slow for rare events, use server text messages:

```text
RADIO_JOIN <groupId>
RADIO_LEAVE <groupId>
WHISPER_START <targetId>
WHISPER_STOP
AUTH <token>
PING / PONG
```

Do **not** send position as text.

---

## 6. VoiceRouter algorithm (the make-or-break module)

### 6.1 Inputs per player

```text
VoicePlayer {
  tsClientId
  conanPlayerId
  position, yaw
  voiceMode
  radioGroupId   // 0 = none
  whisperTargets[]
  isGlobalSpeaker
  permissions
  lastAudibleVersion
  lastWhisperList[]   // cache for diff
}
```

### 6.2 Tick (5–10 Hz)

```text
VoiceRouter_tick():
  1. snapshot all players (no SDK calls while computing)
  2. assign each player to spatial cell (grid)
  3. for each speaker S:
       candidates = players in S.cell and 8 neighbors
       audible = {}
       for each T in candidates:
         if distance_ok(S,T) and zone_ok(S,T) and permission_ok(S,T):
           audible.add(T)
       add radio group peers of S
       add whisper targets of S
       if S.isGlobalSpeaker: audible = all authenticated
       remove S from audible
  4. for each speaker S:
       if audible == S.lastWhisperList: continue   // hysteresis / cache hit
       ITeamSpeakServer.setWhisperList(S.tsClientId, audible)
       S.lastWhisperList = audible
```

### 6.3 Mapping to SDK

```c
// Hear B and C only:
anyID targets[] = { B, C, 0 };
ts3server_setClientWhisperList(serverID, A, NULL, targets);

// Clear (no custom routing / hear nobody via whisper):
ts3server_setClientWhisperList(serverID, A, NULL, NULL);
```

### 6.4 Hard rule

While a whisper list is active, **channel voice is disabled** for that client. Therefore:

- Every proximity / radio / whisper / global path must be represented in the whisper list.
- Do not half-use channel voice for some modes and whisper for others.

### 6.5 Distance bands (config)

```text
0-10 m   full
10-25 m  reduced   (client may still apply gain/3D)
25-50 m  quiet
>50 m    not in audible set  (no packets)
```

Server decides membership; client may still shape loudness with 3D/PCM for members that *do* receive packets.

---

## 7. 3D voice (client-side)

Server stores positions; **client** applies spatialization:

```text
each frame / 20 Hz:
  ts3client_systemset3DListenerAttributes(selfPos, forward, up)
  for each remote client we currently receive:
      ts3client_channelset3DAttributes(remoteId, theirPos)
  onCustom3dRolloffCalculationClientEvent -> Conan curve
```

Local effects (cave reverb/lowpass): `onEditPlaybackVoiceDataEvent`, driven by server env flag in metadata or config. Reuse V8 DSP ideas; rewrite into client module.

---

## 8. Module API contracts

### Server adapter (`ITeamSpeakServer`)

```text
setWhisperList(clientId, clientIds[])
clearWhisperList(clientId)
getClientMeta(clientId) -> string
getClientList() -> ids
moveClient(clientId, channelId)          // optional lobby
```

Implementation = only place with `#include` of ServerLib headers.

### Client adapter (`ITeamSpeakClient`)

```text
connect(host, port, nick, password)
disconnect()
publishMeta(packedState)
setListener3D(pos, forward, up)
setSpeaker3D(clientId, pos)
```

### Game provider (`IGameDataProvider`)

```text
bool getPlayerState(PlayerState* out);
// out: playerId, characterId, position, rotation, voiceMode, ptt
```

First implementation: file/IPC like V8 `pos_file.c` / Mumble path. Swap later without touching adapters.

---

## 9. Threading

| Thread | Rate | Work | Forbidden |
|---|---|---|---|
| SDK server callbacks | event | enqueue only | proximity O(N), disk |
| Routing tick | 5–10 Hz | audible sets + `setClientWhisperList` | game file I/O |
| SDK client callbacks | event | status / PCM / 3D hooks | blocking |
| Game provider | 20–30 Hz | read Conan, snapshot | raw `ts3client_*` (use adapter) |
| PCM callback | audio | effects from snapshot | locks, SDK calls |

Fail closed: if server unreachable → mute proximity, show disconnected HUD; never "everyone hears everyone".

---

## 10. Repository layout (`ts3-Testing`)

```text
conan-voice/
├── client/     # ClientLib app
├── server/     # ServerLib host
├── shared/     # protocol pack, types, config schemas
├── sdk/        # vendored TS3 SDK (examples untouched)
├── tests/
│   ├── unit/
│   ├── integration/   # 2-client routing tests
│   └── load/
├── docs/
└── CMakeLists.txt     # configurable SDK path; no hardcoded user paths
```

Isolated from existing `src/` so V8 DLL builds stay untouched.

---

## 11. Phases with concrete exits

| Phase | Deliverable | Exit criteria |
|---|---|---|
| **0** SDK + license | Eval license; official examples build | Every **[VERIFY]** frozen against headers |
| **1** Server bootstrap | `conan-voice-server` start/stop/restart | Virtual server accepts a connection |
| **2** Client + meta | Custom client talks; metadata round-trip | Server sees positions updating |
| **3** Routing PoC | Whisper lists | **A hears B, not C**; then chain `A→B, B→C, C↛A` |
| **4** Conan | `IGameDataProvider` live | Real pos/ids/mode in metadata |
| **5** Proximity + 3D | Grid + ranges + 3D | Distance hearability + spatial audio |
| **6** Features | Radio, whisper groups, global, HUD, reverb | Feature checklist green |
| **7** Perf | 10…250 clients | CPU/RAM/latency documented |
| **8** Production | systemd, monitoring, commercial license | v1.0 DoD |

**Kill gate:** if Phase 1 or 3 fails, stop. No Conan, no radio, no DB.

---

## 12. Definition of done

**Milestone 1:** ServerLib up; virtual server created; two SDK clients pass voice; server changes whisper routing; clean shutdown + restart.

**v0.1:** Conan position → proximity → 3D; player mapping; voice range; reconnect; basic HUD.

**v0.2:** radio, whisper, global, root-channel config, cave reverb, HUD colors, permissions.

**v1.0:** production client+server; 250+ tested; monitoring; crash recovery; security review; **license cleared**; docs complete.

---

## 13. Risks and open questions

| Risk | Mitigation |
|---|---|
| Paid license / eval limits | Phase 0 first; contact bizdev early |
| SDK EOL 2027-12-31 | Keep `ITeamSpeakServer` / `ITeamSpeakClient` seams |
| API name drift | Phase 0 freezes headers |
| Whisper disables channel voice | Design all modes as whisper lists |
| Position spoofing | Document in v1; game-server feed in v2 |
| Metadata push vs poll | Measure in Phase 2 |

Open until PoC:

1. Exact callback arg lists in `ServerLibFunctions` / client UI funcs  
2. Server-side `CLIENT_META_DATA` getter + change event  
3. Exact keypair + 3D signatures  
4. Safe whisper-list update rate  
5. Real slot cap under license  
6. Identity bind: Conan / Steam / SDK identity  
7. Post-2027 exit path  

---

## Guardrails

- Work only on `ts3-Testing`
- Do not edit `REWRITE_PLAN_V8.md`, `REWRITE_PLAN.md`, or V8 plugin code for this workstream
- Rewrite from V8/Mumble references — never blind-copy
- Document each code change under `doku/` when implementation starts
- No branch/code/commit until you explicitly approve execution