mirror of
https://github.com/TheJordanDev/PEAK-JordanMod.git
synced 2026-10-17 08:05:25 +02:00
0.1.15 | BingBong live voice (in-game client)
Speak through BingBong with a live microphone, streamed from a relay rather than over PEAK's Photon servers. This is the in-game half only: the relay and the web app that captures the microphone are not built yet, so the feature does nothing on its own until they exist. Every BingBong gets a voice source as it spawns, positioned on the plush and inheriting the 3D setup of his own voice lines. His mouth animates to the live audio on every client, and hands back to the vanilla animation while one of his own lines plays. Those lines take priority: the live voice ducks for their duration and resumes at the live position. Playback uses OnAudioFilterRead rather than a streaming AudioClip. A clip created with stream:true fills Unity's decode buffer ahead of playback, and that read-ahead cannot be satisfied from a live stream because the audio has not arrived yet -- the buffer drained to empty on a cycle no matter how deep it was made, which was audible as constant clicking. The filter callback runs in lockstep with playback, so demand can never outrun real time. Includes tools/stub_relay.py, a development stub speaking the same wire format with steady, swept and speech-shaped test signals, so the client can be tested end to end without the real backend, a microphone or a browser.
This commit is contained in:
12
CHANGELOG.md
12
CHANGELOG.md
@@ -94,3 +94,15 @@
|
||||
- The audio bank sync no longer drives `UnityWebRequest` from a background thread, which is main-thread only, and no longer busy-waits on it while blocking a second thread-pool thread.
|
||||
- The song display no longer scans every loaded object looking for its font on every frame. It falls back to the default font immediately and retries for the nicer one on an interval, stopping once found.
|
||||
- The audio bank URL is normalised before use: a missing scheme is filled in (http for localhost, https otherwise) and a trailing slash trimmed. Without a scheme every request failed outright with "Invalid BugleSoundAPIURL in config", since the URL could not be parsed.
|
||||
|
||||
# v0.1.15 | BingBong live voice (in-game client)
|
||||
|
||||
- Speak through BingBong with a live microphone. Audio is streamed from a relay rather than over PEAK's Photon servers, so it adds no load there at all. This release is the **in-game half only** — the relay and the web app that captures the microphone are not built yet, so the feature does nothing on its own until they exist.
|
||||
- Every BingBong in the world gets a voice source as it spawns, positioned on the plush and inheriting the 3D setup of his own voice lines, so the live voice carries and attenuates like he does and obeys the game's volume sliders. Several BingBongs each speak from their own position.
|
||||
- His mouth animates to the live audio, on every client rather than just the speaker's, and hands control back to the vanilla animation while one of his own voice lines plays.
|
||||
- Canned voice lines take priority: the live voice ducks for the length of the line and then resumes at the live position, so the two never talk over each other. Detected from the RPC body, so the duck stays in step across the lobby without any networking of our own.
|
||||
- New "BingBong Voice" config section, all of it applied live. The relay URL is derived from `BugleSoundAPIURL` when left empty, since that is where the relay will live, and can be overridden to point anywhere — a bare host or `host:port` is completed automatically. Range defaults to 50m, further than his 30m voice lines carry.
|
||||
- Playback goes through `OnAudioFilterRead` rather than a streaming `AudioClip`. A clip created with `stream: true` fills Unity's decode buffer *ahead* of playback, and that read-ahead cannot be satisfied from a live stream because the audio has not arrived yet — the buffer drained to empty on a cycle no matter how deep it was, which was audible as constant clicking. The filter callback runs in lockstep with playback instead, so demand can never outrun real time.
|
||||
- Incoming audio is resampled to the output rate by the mod rather than relying on Unity's mixer, so a device running at 44.1kHz or 48kHz sounds identical. A jitter buffer absorbs network timing, rendering what it has and padding the rest on a shortfall rather than muting and refilling, which pumped audibly when it happened often.
|
||||
- Both Harmony patches are wrapped so that a failure can only cost the voice feature: the patch body is a try/catch around an isolated method, and repeated failures disable it for the session rather than breaking the item.
|
||||
- `tools/stub_relay.py` is a development stub of the relay, enough to test the client end to end before the real backend exists. It speaks the same wire format and generates steady, swept and speech-shaped test signals, so no microphone or browser is needed.
|
||||
|
||||
Reference in New Issue
Block a user