Jordan/PEAK-JordanMod
1
0
Fork 0
mirror of https://github.com/TheJordanDev/PEAK-JordanMod.git synced 2026-10-17 08:05:25 +02:00
Code Issues Packages Projects Releases Wiki Activity

0.1.15 | BingBong live voice (client groundwork)

Browse Source 
Lets a live microphone be streamed through BingBong from a relay rather
than over PEAK's Photon servers. This is the in-game client only: it
connects over a WebSocket, decodes the audio and plays it. The relay and
the web app that captures the microphone are not built yet, so the
feature does nothing on its own.

Playback uses OnAudioFilterRead rather than a streaming AudioClip. A clip
created with stream:true fills Unity's decode buffer ahead of playback,
and that read-ahead cannot be satisfied from a live stream because the
audio has not arrived yet -- the buffer drained to empty on a cycle no
matter how deep it was made, which was audible as constant clicking. The
filter callback runs in lockstep with playback, so demand can never
outrun real time.

Includes tools/stub_relay.py, a development stub speaking the same wire
format, so the client can be tested end to end without the real backend,
a microphone or a browser.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
TheJordan
2026-09-05 16:18:04 +02:00
parent 50a7845345
commit 0d2350553a
8 changed files with 1137 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
CHANGELOG.md
View File

@@ -94,3 +94,12 @@
- The audio bank sync no longer drives `UnityWebRequest` from a background thread, which is main-thread only, and no longer busy-waits on it while blocking a second thread-pool thread.
- The song display no longer scans every loaded object looking for its font on every frame. It falls back to the default font immediately and retries for the nicer one on an interval, stopping once found.
- The audio bank URL is normalised before use: a missing scheme is filled in (http for localhost, https otherwise) and a trailing slash trimmed. Without a scheme every request failed outright with "Invalid BugleSoundAPIURL in config", since the URL could not be parsed.
# v0.1.15 | BingBong live voice (work in progress)
- Groundwork for speaking through BingBong with a live microphone, streamed from a relay rather than over PEAK's Photon servers. This release contains the in-game client only: it connects to a relay over a WebSocket, decodes the incoming audio and plays it. The relay itself and the web app that captures the microphone are not built yet, so the feature does nothing on its own.
- New "BingBong Voice" config section. The relay URL is derived from `BugleSoundAPIURL` when left empty, since that is where the relay will live, and can be overridden to point anywhere (a bare host or `host:port` is accepted and completed automatically).
- Playback goes through `OnAudioFilterRead` rather than a streaming `AudioClip`. A clip created with `stream: true` fills Unity's decode buffer *ahead* of playback, and that read-ahead cannot be satisfied from a live stream because the audio has not arrived yet — the buffer drained to empty on a cycle no matter how deep it was, which was audible as constant clicking. The filter callback runs in lockstep with playback instead, so demand can never outrun real time.
- Incoming audio is resampled to the output rate by the mod rather than relying on Unity's mixer, so a device running at 44.1kHz or 48kHz sounds identical. Verified at every rate combination.
- A jitter buffer absorbs network timing, with a shortfall rendering what it has and padding the rest rather than muting and refilling, which pumped audibly when it happened often.
- `tools/stub_relay.py` is a development stub of the relay, enough to test the client end to end before the real backend exists. It speaks the same wire format and can generate a test tone, so no microphone or browser is needed.