Compare commits

...

4 Commits

Author SHA1 Message Date
0d2350553a 0.1.15 | BingBong live voice (client groundwork)
Lets a live microphone be streamed through BingBong from a relay rather
than over PEAK's Photon servers. This is the in-game client only: it
connects over a WebSocket, decodes the audio and plays it. The relay and
the web app that captures the microphone are not built yet, so the
feature does nothing on its own.

Playback uses OnAudioFilterRead rather than a streaming AudioClip. A clip
created with stream:true fills Unity's decode buffer ahead of playback,
and that read-ahead cannot be satisfied from a live stream because the
audio has not arrived yet -- the buffer drained to empty on a cycle no
matter how deep it was made, which was audible as constant clicking. The
filter callback runs in lockstep with playback, so demand can never
outrun real time.

Includes tools/stub_relay.py, a development stub speaking the same wire
format, so the client can be tested end to end without the real backend,
a microphone or a browser.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-05 16:18:04 +02:00
50a7845345 0.1.14 | Audio fixes & optimisations 2026-09-05 16:15:01 +02:00
2802c71cd0 0.1.13 | Fix for PEAK update
Player.EmptySlot and PhotonNetwork.InstantiateItemRoom each gained an
optional parameter, which compiles fine but left the previous build
calling signatures that no longer exist. The InstantiateItemRoom throw
escaped the BagsForEveryone postfix and took down every
SingleItemSpawner in the level, which is why backpacks and BingBong
stopped spawning and campfires no longer advanced the biome.

Also hardens that postfix so a future API change costs the extra bags
and nothing else, registers them with the spawner's SpawnedItemTracker,
and stops the Open Backpack keybind from opening the wheel while the
backpack is held in hand.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 00:10:07 +02:00
96204e8a50 0.1.12 | Added: Open Passport keybind
Also removes a leftover debug keybind on the BingBong module that was
bound to P and would have fired alongside the new passport keybind.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 00:09:52 +02:00
17 changed files with 1615 additions and 148 deletions

View File

@@ -62,3 +62,44 @@
- Fixed the right-click confirmation always showing "No answer, not refreshing songs" even after successfully confirming, due to a timing check that ran before the confirmation window instead of after. - Fixed the right-click confirmation always showing "No answer, not refreshing songs" even after successfully confirming, due to a timing check that ran before the confirmation window instead of after.
- Fixed an error (NullReferenceException) every time a thrown item bonked something: `Bonkable` added at runtime has no serialized prefab data, so its sound-effect array was `null` instead of empty. It now defaults to an empty array so bonks still ragdoll/knock back the target, just silently. - Fixed an error (NullReferenceException) every time a thrown item bonked something: `Bonkable` added at runtime has no serialized prefab data, so its sound-effect array was `null` instead of empty. It now defaults to an empty array so bonks still ragdoll/knock back the target, just silently.
# v0.1.12 | Added customization keybinds
- Open the passport when pressing the "Open Passport" keybind (default: P)
- Removed a leftover debug keybind on the BingBong module that was also bound to P and would have fired alongside the new passport keybind.
# v0.1.13 | Fix for PEAK update
- Rebuilt against the September PEAK update. Two game methods gained an optional parameter, which compiles fine but leaves an older build calling a signature that no longer exists, so both threw a MissingMethodException at runtime:
- `Player.EmptySlot(slot)` became `EmptySlot(slot, bool andBroadcast = true)`. This killed the whole "Toggle Bugle" keybind (default: V) — both giving the Bugle and swapping it for the Megaphone.
- `PhotonNetwork.InstantiateItemRoom(name, position, rotation)` gained a fourth `bool warnIfNotHost = true`. This one threw inside the Bags for Everyone postfix on `SingleItemSpawner.TrySpawnItems`, and because a throwing postfix propagates out of the method it patched, it took down every `SingleItemSpawner` in the level rather than just the backpack one. Backpacks and BingBong stopped spawning, and lighting a campfire no longer advanced the biome: the spawn loop runs inside `MapHandler`'s transition coroutine, so the throw killed that coroutine and the campfire items, day/night blend, biome title and fog reveal all silently never happened.
- Removed the manual `RPCRemoveItemFromSlot` call the Bugle keybind made right after `EmptySlot`. `EmptySlot` already forwards the removal to the host itself, so the extra call only logged "Only Master Client can remove items!" on every press for anyone who wasn't hosting.
- The "Open Backpack" keybind (default: B) no longer opens the wheel while the backpack is held in your hands. The wheel pulls items out of the backpack's on-back visuals, which the game deactivates while the backpack is in hand, so anything taken out was spawned into a disabled rig and disappeared. Hold-interacting the held backpack still opens it the normal way.
- Hardened the Bags for Everyone patch. Its master-client and spawner-name checks now run in the postfix itself instead of inside the method that spawns the bags, and the spawn is wrapped in a try/catch. Because Mono resolves a method's call targets when it JITs that method, a changed game API used to throw before any of those checks could run, taking down every `SingleItemSpawner` in the level — which is what stopped campfires from advancing the biome in 0.1.12. A failure now costs the extra bags and nothing else.
- The extra backpacks are registered with the spawner's `SpawnedItemTracker`, so a quicksave records them instead of restoring only the vanilla one, and the patch no longer adds a fresh batch on top every time a run is loaded from a save.
# v0.1.14 | Audio fixes & optimisations
- Songs are now kept compressed in memory and decoded during playback instead of being held as raw PCM for the whole session. A 127-file, 110-minute bank measured ~82 MB on disk but around 0.9-1.8 GB once decompressed; it now costs roughly its size on disk, for a little CPU while a bugle is actually sounding.
- Fixed the audio bank sync never removing anything. The removal step ran on the sync's background thread and called `FindObjectsByType`/`Object.Destroy`, which are main-thread only, so it threw on the first song it tried to remove and aborted the whole sync — downloads included. Songs dropped from the bank had simply been reloading from disk on every launch ever since.
- Syncing now makes the folder match the bank exactly: anything the bank doesn't list is pulled out, including files added by hand and stale copies of a song left behind under a different extension. Nothing is deleted — files are moved to a `_removed` subfolder of the bank directory, which the loader ignores. BingBong voice lines are exempt and never touched.
- A sync that can't reach the bank, or gets an empty list back, now cancels instead of treating "no entries" as "nothing belongs here" and clearing out the whole folder.
- Fixed the game freezing the first time a sound is played in a session. `DownloadHandlerAudioClip` hands back a `DecompressOnLoad` clip whose PCM data is only decoded on first use, so that decode was landing inside the first `AudioSource.Play()` on the main thread. The clips are now decoded during loading instead, where the cost is expected and spread across frames.
- Fixed an empty sounds folder permanently breaking the audio system. With no songs loaded, picking a starting song indexed an empty list and threw, and because that ran from the load-complete callback it left the loading flag stuck on, silently killing every later sync and refresh for the rest of the session. The song index is also clamped now, since a refresh that removes songs could leave it past the end of the list.
- Fixed a Bugle picked up off the ground being completely silent. The patch that installs the custom sound only ran for items already in hand, but `ItemState.Ground` is the default and `Item.Start` doesn't set state, so anything spawned into the world never got it — while the vanilla Bugle sound was being muted regardless. The vanilla sound is now only muted where the replacement actually took over.
- Fixed BingBong subtitles never resolving: the localisation key was registered in lowercase, but `LocalizedText.GetText` upper-cases ids before looking them up. The entry also had a single string where one per language is required, which threw for every non-English player.
- Fixed `SFX_Instance` being created with `new`. It derives from `ScriptableObject`, which has to go through `ScriptableObject.CreateInstance` or it skips Unity's native-side initialisation.
- BingBong responses are now built once and shared instead of being rebuilt every time an item is enabled, which was allocating a `ScriptableObject` per response each time and never releasing the previous ones.
- Refreshing songs no longer scans the scene for audio sources once per song — a bank of N songs meant N full scene scans — and no longer forces a blocking garbage collection.
- File hashing is computed on demand on the sync thread rather than for every clip while loading, where it was reading each file end to end and running SHA256 on the main thread.
- The audio bank sync no longer drives `UnityWebRequest` from a background thread, which is main-thread only, and no longer busy-waits on it while blocking a second thread-pool thread.
- The song display no longer scans every loaded object looking for its font on every frame. It falls back to the default font immediately and retries for the nicer one on an interval, stopping once found.
- The audio bank URL is normalised before use: a missing scheme is filled in (http for localhost, https otherwise) and a trailing slash trimmed. Without a scheme every request failed outright with "Invalid BugleSoundAPIURL in config", since the URL could not be parsed.
# v0.1.15 | BingBong live voice (work in progress)
- Groundwork for speaking through BingBong with a live microphone, streamed from a relay rather than over PEAK's Photon servers. This release contains the in-game client only: it connects to a relay over a WebSocket, decodes the incoming audio and plays it. The relay itself and the web app that captures the microphone are not built yet, so the feature does nothing on its own.
- New "BingBong Voice" config section. The relay URL is derived from `BugleSoundAPIURL` when left empty, since that is where the relay will live, and can be overridden to point anywhere (a bare host or `host:port` is accepted and completed automatically).
- Playback goes through `OnAudioFilterRead` rather than a streaming `AudioClip`. A clip created with `stream: true` fills Unity's decode buffer *ahead* of playback, and that read-ahead cannot be satisfied from a live stream because the audio has not arrived yet — the buffer drained to empty on a cycle no matter how deep it was, which was audible as constant clicking. The filter callback runs in lockstep with playback instead, so demand can never outrun real time.
- Incoming audio is resampled to the output rate by the mod rather than relying on Unity's mixer, so a device running at 44.1kHz or 48kHz sounds identical. Verified at every rate combination.
- A jitter buffer absorbs network timing, with a shortfall rendering what it has and padding the rest rather than muting and refilling, which pumped audibly when it happened often.
- `tools/stub_relay.py` is a development stub of the relay, enough to test the client end to end before the real backend exists. It speaks the same wire format and can generate a test tone, so no microphone or browser is needed.

View File

@@ -13,6 +13,9 @@ public static class ConfigHandler
// Better Airport // Better Airport
public static ConfigEntry<float> ConveyorSpeedModifier { get; private set; } = null!; public static ConfigEntry<float> ConveyorSpeedModifier { get; private set; } = null!;
// Open Passport settings
public static ConfigEntry<KeyCode> OpenPassport { get; private set; } = null!;
// Stashed Bugle settings // Stashed Bugle settings
public static ConfigEntry<KeyCode> ToggleBugle { get; private set; } = null!; public static ConfigEntry<KeyCode> ToggleBugle { get; private set; } = null!;
@@ -26,6 +29,16 @@ public static class ConfigHandler
public static ConfigEntry<string> FavoriteSongsList { get; private set; } = null!; public static ConfigEntry<string> FavoriteSongsList { get; private set; } = null!;
public static ConfigEntry<KeyCode> FavoriteSongToggleKey { get; private set; } = null!; public static ConfigEntry<KeyCode> FavoriteSongToggleKey { get; private set; } = null!;
// BingBong Voice settings
public static ConfigEntry<bool> BingBongVoiceEnabled { get; private set; } = null!;
public static ConfigEntry<string> BingBongVoiceRelayURL { get; private set; } = null!;
public static ConfigEntry<string> BingBongVoiceDevToken { get; private set; } = null!;
public static ConfigEntry<float> BingBongVoiceVolume { get; private set; } = null!;
public static ConfigEntry<float> BingBongVoiceDuckVolume { get; private set; } = null!;
public static ConfigEntry<float> BingBongVoiceDuckLead { get; private set; } = null!;
public static ConfigEntry<float> BingBongVoiceJitterMs { get; private set; } = null!;
public static ConfigEntry<KeyCode> BingBongVoiceDebugKey { get; private set; } = null!;
public static void Initialize(ConfigFile configFile) public static void Initialize(ConfigFile configFile)
{ {
Config = configFile; Config = configFile;
@@ -49,6 +62,14 @@ public static class ConfigHandler
) )
); );
// Open Passport settings
OpenPassport = Config.Bind(
"Key Bindings",
"OpenPassport",
KeyCode.P,
new ConfigDescription("Open Passport UI")
);
// Stashed Bugle settings // Stashed Bugle settings
ToggleBugle = Config.Bind( ToggleBugle = Config.Bind(
"Control", "Control",
@@ -96,6 +117,63 @@ public static class ConfigHandler
new ConfigDescription("Key to toggle favorite status of current song") new ConfigDescription("Key to toggle favorite status of current song")
); );
// BingBong Voice settings
BingBongVoiceEnabled = Config.Bind(
"BingBong Voice",
"Enabled",
true,
new ConfigDescription("Let BingBong speak with a live voice streamed from the relay. When off, no patches are applied at all.")
);
BingBongVoiceRelayURL = Config.Bind(
"BingBong Voice",
"RelayURL",
"",
new ConfigDescription("Override for the voice relay WebSocket URL. Leave empty to derive it from BugleSoundAPIURL, which is where the relay lives by default (https://host -> wss://host/voice/ws).")
);
BingBongVoiceDevToken = Config.Bind(
"BingBong Voice",
"DevToken",
"listen:test",
new ConfigDescription("Temporary token appended to the relay URL when it carries none. The real join flow will fetch this from the audio bank instead.")
);
BingBongVoiceVolume = Config.Bind(
"BingBong Voice",
"Volume",
1.0f,
new ConfigDescription("Live voice volume", new AcceptableValueRange<float>(0f, 1f))
);
BingBongVoiceDuckVolume = Config.Bind(
"BingBong Voice",
"DuckVolume",
0.0f,
new ConfigDescription("Live voice volume while one of BingBong's own voice lines is playing", new AcceptableValueRange<float>(0f, 1f))
);
BingBongVoiceDuckLead = Config.Bind(
"BingBong Voice",
"DuckLeadSeconds",
1.0f,
new ConfigDescription("How long before a voice line's audio actually starts to begin ducking. AskRoutine waits about a second before it plays anything.", new AcceptableValueRange<float>(0f, 2f))
);
BingBongVoiceJitterMs = Config.Bind(
"BingBong Voice",
"JitterBufferMs",
80f,
new ConfigDescription("Audio buffered before playback starts. Lower is more responsive, higher survives worse connections.", new AcceptableValueRange<float>(40f, 400f))
);
BingBongVoiceDebugKey = Config.Bind(
"BingBong Voice",
"DebugToggleKey",
KeyCode.F7,
new ConfigDescription("Connect/disconnect the relay and play it through a non-positional debug source, for testing without a BingBong")
);
} }
} }

View File

@@ -8,7 +8,7 @@
<!-- This is the display name of your mod. Example: BepInEx Template --> <!-- This is the display name of your mod. Example: BepInEx Template -->
<AssemblyTitle>JordanMod</AssemblyTitle> <AssemblyTitle>JordanMod</AssemblyTitle>
<!-- This is the version number of your mod. --> <!-- This is the version number of your mod. -->
<Version>0.1.11</Version> <Version>0.1.15</Version>
</PropertyGroup> </PropertyGroup>
<PropertyGroup> <PropertyGroup>

View File

@@ -113,8 +113,10 @@ class BetterBugleModule : Module
List<string> scrollActionLocalizations = new(LocalizedText.LANGUAGE_COUNT); List<string> scrollActionLocalizations = new(LocalizedText.LANGUAGE_COUNT);
for (int i = 0; i < LocalizedText.LANGUAGE_COUNT; i++) scrollActionLocalizations.Add("Change Song"); for (int i = 0; i < LocalizedText.LANGUAGE_COUNT; i++) scrollActionLocalizations.Add("Change Song");
LocalizedText.mainTable.Add("SONG_LIST", secondaryActionLocalizations); // Indexer, not Add: Add throws if the key is already present, which happens if the game
LocalizedText.mainTable.Add("CHANGE_SONG", scrollActionLocalizations); // rebuilds its table (e.g. on a language change) and this runs again.
LocalizedText.mainTable["SONG_LIST"] = secondaryActionLocalizations;
LocalizedText.mainTable["CHANGE_SONG"] = scrollActionLocalizations;
} }
private void OnAllAudioClipsLoaded() private void OnAllAudioClipsLoaded()
@@ -128,8 +130,22 @@ class BetterBugleModule : Module
if (Song.Songs.ContainsKey(songKeyName) && !Song.FavoriteSongs.Contains(songKeyName)) if (Song.Songs.ContainsKey(songKeyName) && !Song.FavoriteSongs.Contains(songKeyName))
Song.FavoriteSongs.Add(songKeyName); Song.FavoriteSongs.Add(songKeyName);
if (!Song.Songs.ContainsKey(AudioSyncWorker.CurrentSongName)) // Indexing this unguarded threw ArgumentOutOfRangeException with an empty bank (a fresh
AudioSyncWorker.CurrentSongName = Song.GetSongNames_Alphabetically()[AudioSyncWorker.CurrentSongIndex]; // install, or a sounds folder that lost its files), and because this runs from
// OnAudioLoadComplete it used to leave IsLoading stuck true and kill sync/refresh for
// the rest of the session. CurrentSongIndex can also outrun the list after a refresh
// removes songs, so clamp rather than assume it is still in range.
List<string> songNames = Song.GetSongNames_Alphabetically();
if (songNames.Count == 0)
{
AudioSyncWorker.CurrentSongIndex = 0;
AudioSyncWorker.CurrentSongName = "None";
}
else if (!Song.Songs.ContainsKey(AudioSyncWorker.CurrentSongName))
{
AudioSyncWorker.CurrentSongIndex = Mathf.Clamp(AudioSyncWorker.CurrentSongIndex, 0, songNames.Count - 1);
AudioSyncWorker.CurrentSongName = songNames[AudioSyncWorker.CurrentSongIndex];
}
} }
} }
@@ -265,6 +281,10 @@ public class BetterBugleUI : MonoBehaviour
private bool fontLoaded = false; private bool fontLoaded = false;
private const float FontSearchInterval = 5f;
private float lastFontSearchTime = -999f;
private int offsetX = 0; private int offsetX = 0;
private int offsetY = 70; private int offsetY = 70;
@@ -294,28 +314,41 @@ public class BetterBugleUI : MonoBehaviour
private void OnGUI() private void OnGUI()
{ {
if (!fontLoaded) EnsureStyle();
{
Font[] array = Resources.FindObjectsOfTypeAll<Font>();
foreach (Font val in array)
{
if (val.name == "Tetsubin Gothic")
{
customStyle = new GUIStyle(GUI.skin.label);
customStyle.font = val;
customStyle.fontSize = fontSize;
customStyle.alignment = TextAnchor.LowerCenter;
customStyle.normal.textColor = Color.white;
fontLoaded = true;
break;
}
}
}
if (customStyle == null) return; if (customStyle == null) return;
RenderSoundDisplay(); RenderSoundDisplay();
RenderPlayingDisplay(); RenderPlayingDisplay();
} }
// This used to run Resources.FindObjectsOfTypeAll<Font>() on every OnGUI call until the font
// turned up -- a full scan of every loaded object, several times per frame, forever if the
// font was never present in the scene. Now we build a usable style immediately with the
// default font and only re-scan for the nicer one on an interval, stopping once found.
private void EnsureStyle()
{
if (fontLoaded) return;
if (customStyle != null && Time.unscaledTime - lastFontSearchTime < FontSearchInterval) return;
lastFontSearchTime = Time.unscaledTime;
Font? found = null;
foreach (Font val in Resources.FindObjectsOfTypeAll<Font>())
{
if (val.name == "Tetsubin Gothic") { found = val; break; }
}
customStyle = new GUIStyle(GUI.skin.label)
{
fontSize = fontSize,
alignment = TextAnchor.LowerCenter,
normal = { textColor = Color.white }
};
if (found != null)
{
customStyle.font = found;
fontLoaded = true;
}
}
private void RenderSoundDisplay() private void RenderSoundDisplay()
{ {
if (customStyle == null) return; if (customStyle == null) return;

View File

@@ -0,0 +1,236 @@
using System;
using System.Security.Authentication;
using System.Threading;
using WebSocketSharp;
namespace JordanMod.Modules.BingBongVoice;
/// One decoded stream. Swapped wholesale when the sender changes sample rate, so consumers
/// only ever hold a consistent (ring, rate) pair.
public sealed class VoiceStream
{
public VoiceRingBuffer Ring { get; }
public int SampleRate => Ring.SampleRate;
public VoiceStream(int sampleRate) => Ring = new VoiceRingBuffer(sampleRate);
}
/// <summary>
/// Holds the websocket to the voice relay and turns its binary frames into samples in a ring.
///
/// Everything here runs on websocket-sharp's own receive thread. It must never touch a Unity
/// API: hop through <see cref="Plugin.RunOnMainThread"/> for that, and never per packet — at
/// 50 frames a second that queue would balloon during any frame hitch.
/// </summary>
public sealed class BingBongVoiceClient
{
// Wire format, kept in step with the plan and the stub relay:
// 0 u8 magic | 1 u8 version | 2 u8 codec | 3 u8 rate code | 4 u32 seq | 8 u32 sampleTimestamp
private const int HeaderSize = 12;
private const byte Magic = 0xB1;
private const byte Version = 1;
private const byte CodecPcm16 = 0;
private static readonly int[] RateTable = [16000, 8000, 24000, 48000];
// A gap larger than this is treated as a restart rather than something to paper over.
private const int MaxGapSamples = 48000;
private WebSocket? _socket;
private VoiceStream? _stream;
private Thread? _reconnectThread;
private volatile bool _shouldRun;
private volatile bool _speakerActive;
private long _expectedTimestamp = -1;
private int _badFrames;
/// The live stream, or null before the first frame arrives. Read from the audio thread.
public VoiceStream? Stream => Volatile.Read(ref _stream);
/// True between speaker_joined and speaker_left control frames.
public bool SpeakerActive => _speakerActive;
public bool IsConnected => _socket is { ReadyState: WebSocketState.Open };
public string Url { get; private set; } = "";
public void Connect(string url)
{
if (_shouldRun) return;
if (string.IsNullOrWhiteSpace(url))
{
Debug.LogWarning("[BingBongVoice] No relay URL configured, not connecting.");
return;
}
// A malformed URL will never start working, so fail here rather than letting the
// reconnect loop retry it every minute for the rest of the session.
if (!Uri.TryCreate(url, UriKind.Absolute, out Uri? parsed) || (parsed.Scheme != "ws" && parsed.Scheme != "wss"))
{
Debug.LogError($"[BingBongVoice] '{url}' is not a usable WebSocket URL, not connecting.");
return;
}
Url = url;
_shouldRun = true;
_reconnectThread = new Thread(ConnectLoop) { IsBackground = true, Name = "BingBongVoice" };
_reconnectThread.Start();
}
public void Disconnect()
{
_shouldRun = false;
_speakerActive = false;
WebSocket? socket = _socket;
_socket = null;
if (socket == null) return;
try { socket.Close(CloseStatusCode.Normal, "disconnecting"); }
catch (Exception e) { Debug.LogWarning($"[BingBongVoice] Error closing socket: {e.Message}"); }
}
// Reconnect with exponential backoff. Runs on its own thread for the whole session.
private void ConnectLoop()
{
int attempt = 0;
System.Random jitter = new();
while (_shouldRun)
{
try
{
OpenSocket();
// Poll rather than block: websocket-sharp gives us no "wait until closed".
while (_shouldRun && _socket is { ReadyState: WebSocketState.Open }) Thread.Sleep(200);
if (!_shouldRun) return;
attempt++;
}
catch (Exception e)
{
attempt++;
// First failure and then every tenth, so an unreachable relay can't flood the log.
if (attempt == 1 || attempt % 10 == 0)
Debug.LogWarning($"[BingBongVoice] Connection attempt {attempt} failed: {e.Message}");
}
if (!_shouldRun) return;
int delayMs = (int)(Math.Min(60000, 2000 * Math.Pow(2, Math.Min(attempt - 1, 5))) * (0.8 + jitter.NextDouble() * 0.4));
for (int slept = 0; slept < delayMs && _shouldRun; slept += 100) Thread.Sleep(100);
}
}
private void OpenSocket()
{
WebSocket socket = new(Url);
// websocket-sharp writes its own "Fatal|WebSocket.connect|..." lines straight to the
// console, including a run of mojibake, every single reconnect attempt. We report
// connection state ourselves, so silence its logger entirely.
socket.Log.Output = (_, _) => { };
// This build of websocket-sharp defaults to a legacy protocol set, so wss:// to any
// modern TLS terminator fails outright without this.
if (Url.StartsWith("wss://", StringComparison.OrdinalIgnoreCase))
{
socket.SslConfiguration.EnabledSslProtocols =
SslProtocols.Tls12 | SslProtocols.Tls11 | SslProtocols.Tls;
}
socket.OnMessage += OnMessage;
socket.OnOpen += (_, _) =>
{
_expectedTimestamp = -1;
_badFrames = 0;
Debug.Log("[BingBongVoice] Connected to relay.");
};
socket.OnClose += (_, e) =>
{
_speakerActive = false;
Debug.Log($"[BingBongVoice] Relay closed ({e.Code} {e.Reason}).");
};
socket.OnError += (_, e) => Debug.LogWarning($"[BingBongVoice] Socket error: {e.Message}");
_socket = socket;
socket.Connect();
}
// websocket-sharp's receive thread. No Unity APIs, no allocation beyond what we're handed.
private void OnMessage(object? sender, MessageEventArgs e)
{
try
{
if (!e.IsBinary)
{
HandleControl(e.Data);
return;
}
byte[] d = e.RawData;
if (d.Length < HeaderSize) return;
if (d[0] != Magic || d[1] != Version)
{
if (++_badFrames <= 3) Debug.LogWarning($"[BingBongVoice] Bad frame header (magic {d[0]:X2}, version {d[1]}).");
return;
}
if (d[2] != CodecPcm16)
{
if (++_badFrames <= 3) Debug.LogWarning($"[BingBongVoice] Unsupported codec {d[2]}, frame dropped.");
return;
}
int rate = RateTable[d[3] & 3];
uint timestamp = (uint)(d[8] | (d[9] << 8) | (d[10] << 16) | (d[11] << 24));
VoiceStream stream = EnsureStream(rate);
// The sender's timestamp tells us exactly how much audio went missing, so a dropped
// frame costs its own duration instead of silently shifting everything after it.
if (_expectedTimestamp >= 0 && timestamp != _expectedTimestamp)
{
long gap = (long)timestamp - _expectedTimestamp;
if (gap > 0 && gap <= MaxGapSamples) stream.Ring.WriteSilence((int)gap);
// A negative or huge gap means the sender restarted; just carry on from here.
}
int payload = d.Length - HeaderSize;
stream.Ring.WritePcm16(d, HeaderSize, payload);
_expectedTimestamp = timestamp + (payload >> 1);
}
catch (Exception ex)
{
if (++_badFrames <= 3) Debug.LogError($"[BingBongVoice] Frame handling failed: {ex}");
}
}
private VoiceStream EnsureStream(int rate)
{
VoiceStream? current = Volatile.Read(ref _stream);
if (current != null && current.SampleRate == rate) return current;
VoiceStream replacement = new(rate);
Volatile.Write(ref _stream, replacement);
_expectedTimestamp = -1;
Debug.Log($"[BingBongVoice] Stream rate is {rate} Hz.");
return replacement;
}
private void HandleControl(string json)
{
// Deliberately not parsing JSON properly: the only fields that matter are these two
// markers, and a substring check can't throw on a malformed payload.
if (json.Contains("speaker_joined"))
{
_speakerActive = true;
Debug.Log("[BingBongVoice] A speaker joined.");
}
else if (json.Contains("speaker_left") || json.Contains("preempted"))
{
_speakerActive = false;
_expectedTimestamp = -1;
Debug.Log("[BingBongVoice] The speaker left.");
}
}
}

View File

@@ -0,0 +1,171 @@
using System;
using UnityEngine;
namespace JordanMod.Modules.BingBongVoice;
[Module(Enabled = true)]
class BingBongVoiceModule : Module
{
public static BingBongVoiceModule? Instance { get; private set; }
public override string ModuleName => "BingBong Voice Module";
public BingBongVoiceClient? Client { get; private set; }
private GameObject? _debugObject;
public override Type[] GetPatches()
{
// Config is bound before modules load (Plugin.Awake: ConfigHandler.Initialize ->
// SetupModules -> SetupPatches), so turning the feature off here means the Harmony
// patches are never applied at all and the mod has zero footprint.
if (!ConfigHandler.BingBongVoiceEnabled.Value) return [];
return [];
}
public override void Initialize()
{
if (Instance != null) return;
Instance = this;
base.Initialize();
}
public override void Update()
{
if (!ConfigHandler.BingBongVoiceEnabled.Value) return;
if (Input.GetKeyDown(ConfigHandler.BingBongVoiceDebugKey.Value)) ToggleDebugStream();
}
/// <summary>
/// Where the relay lives. The relay is served by the audio bank, so an unset override just
/// derives the WebSocket URL from BugleSoundAPIURL; setting it points somewhere else
/// entirely (a local stub, a separate host).
/// </summary>
public static string ResolveRelayUrl()
{
string overrideUrl = (ConfigHandler.BingBongVoiceRelayURL.Value ?? "").Trim();
string source;
string url;
if (overrideUrl.Length > 0)
{
// An override names the endpoint. Only fill in /voice/ws when it has no path at all,
// so "localhost:8787" works but "wss://host/custom/path" is left alone.
source = overrideUrl;
url = NormalizeOverride(overrideUrl);
}
else
{
// The audio bank URL names a service root, which may itself sit under a path
// (https://audiobank.thejordan.dev/api), so the endpoint always goes on the end.
source = (ConfigHandler.BugleSoundAPIURL.Value ?? "").Trim();
if (source.Length == 0)
{
Debug.LogWarning("[BingBongVoice] Neither RelayURL nor BugleSoundAPIURL is set, nowhere to connect.");
return "";
}
url = DeriveFromApiBase(source);
}
if (url.Length == 0)
{
Debug.LogError($"[BingBongVoice] Could not make a usable WebSocket URL out of '{source}'. Expected something like 'ws://localhost:8787/voice/ws' or just 'localhost:8787'.");
return "";
}
// Until the real /voice/join flow lands, the token comes from config rather than a
// lookup. A URL that already carries one is left alone.
if (url.IndexOf("token=", StringComparison.OrdinalIgnoreCase) < 0)
{
string token = (ConfigHandler.BingBongVoiceDevToken.Value ?? "").Trim();
if (token.Length > 0)
url += (url.IndexOf('?') >= 0 ? "&" : "?") + "token=" + Uri.EscapeDataString(token);
}
return url;
}
private static string NormalizeOverride(string raw)
{
string url = ToWebSocketScheme(raw);
// Split the query/fragment off first so appending a path can't mangle them.
int split = url.IndexOfAny(['?', '#']);
string tail = split >= 0 ? url.Substring(split) : "";
string head = split >= 0 ? url.Substring(0, split) : url;
if (!IsWebSocketUrl(head, out Uri? parsed)) return "";
// A bare host means the caller gave us the root, not the endpoint.
if (parsed.AbsolutePath.Length <= 1) head = head.TrimEnd('/') + "/voice/ws";
return head + tail;
}
private static string DeriveFromApiBase(string apiBase)
{
string url = ToWebSocketScheme(apiBase);
// The base is a service root, so anything after it is not ours to keep.
int split = url.IndexOfAny(['?', '#']);
if (split >= 0) url = url.Substring(0, split);
url = url.TrimEnd('/') + "/voice/ws";
return IsWebSocketUrl(url, out _) ? url : "";
}
private static bool IsWebSocketUrl(string url, out Uri parsed)
{
parsed = null!;
if (!Uri.TryCreate(url, UriKind.Absolute, out Uri? candidate)) return false;
if (candidate.Scheme != "ws" && candidate.Scheme != "wss") return false;
parsed = candidate;
return true;
}
private static string ToWebSocketScheme(string url)
{
if (url.StartsWith("ws://", StringComparison.OrdinalIgnoreCase)) return url;
if (url.StartsWith("wss://", StringComparison.OrdinalIgnoreCase)) return url;
if (url.StartsWith("https://", StringComparison.OrdinalIgnoreCase)) return "wss://" + url.Substring(8);
if (url.StartsWith("http://", StringComparison.OrdinalIgnoreCase)) return "ws://" + url.Substring(7);
// No scheme at all: same rule as everywhere else, loopback gets the insecure variant.
return Helper.EnsureScheme(url, "wss", "ws");
}
// Stage 1 harness: connect and play through a non-positional source, so the whole network
// and audio path can be tested with no BingBong, no lobby and no browser.
private void ToggleDebugStream()
{
if (Client != null)
{
Debug.Log("[BingBongVoice] Stopping debug stream.");
Client.Disconnect();
Client = null;
if (_debugObject != null) UnityEngine.Object.Destroy(_debugObject);
_debugObject = null;
return;
}
string url = ResolveRelayUrl();
if (url.Length == 0) return;
Debug.Log($"[BingBongVoice] Starting debug stream against {url}");
Client = new BingBongVoiceClient();
Client.Connect(url);
_debugObject = new GameObject("BingBongVoiceDebug");
UnityEngine.Object.DontDestroyOnLoad(_debugObject);
BingBongVoiceSource.Attach(_debugObject, null);
}
public override void Destroy()
{
Client?.Disconnect();
Client = null;
if (_debugObject != null) UnityEngine.Object.Destroy(_debugObject);
_debugObject = null;
base.Destroy();
}
}

View File

@@ -0,0 +1,296 @@
using System;
using UnityEngine;
namespace JordanMod.Modules.BingBongVoice;
/// <summary>
/// Plays the live voice stream through one AudioSource. One of these per BingBong (and one
/// non-positional debug instance), each with its own read cursor into the shared ring.
///
/// Output goes through <c>OnAudioFilterRead</c> rather than a procedural streaming AudioClip.
/// A clip created with <c>stream: true</c> fills Unity's own decode buffer ahead of playback --
/// that read-ahead is by design, and it cannot be satisfied from a live stream because the audio
/// simply has not arrived yet, so the ring drained to empty no matter how deep the jitter buffer
/// was made. OnAudioFilterRead is invoked exactly once per DSP block, in lockstep with playback,
/// so demand can never outrun real time. The AudioSource still needs *something* playing for the
/// filter to run, hence the silent carrier clip.
/// </summary>
public sealed class BingBongVoiceSource : MonoBehaviour
{
private enum State { Buffering, Playing }
private AudioSource? _source;
private AudioClip? _carrier;
private VoiceStream? _stream;
private int _outputRate;
private double _read; // fractional position into the ring, audio thread only
private State _state = State.Buffering;
private float _gain; // smoothed toward _targetGain, audio thread only
private volatile float _targetGain;
private volatile float _amplitude;
private volatile bool _dead;
private int _underruns;
private float _lastReportTime;
private int _strikes;
private volatile int _depthSamples;
private volatile int _minDepth = int.MaxValue;
private volatile int _streamRate = 1;
private volatile int _blockFrames; // frames per DSP block, as actually delivered
private static bool _reportedBlock; // once per session, not once per BingBong
private float _gainStep = 0.0008f;
private long _lastWritePos;
private float _lastRateTime;
/// Set by the duck patch. Live voice is attenuated until this time.
public float DuckUntil { get; set; }
public float Amplitude => _amplitude;
public static BingBongVoiceSource Attach(GameObject target, AudioSource? template)
{
if (target.TryGetComponent(out BingBongVoiceSource existing)) return existing;
BingBongVoiceSource voice = target.AddComponent<BingBongVoiceSource>();
voice.Configure(template);
return voice;
}
private void Configure(AudioSource? template)
{
_outputRate = AudioSettings.outputSampleRate;
_gainStep = 1f / (_outputRate * 0.015f); // ~15ms attack whatever the device runs at
// Silent carrier: the filter only runs while the source is playing something. Its
// contents are irrelevant, OnAudioFilterRead overwrites the buffer wholesale.
_carrier = AudioClip.Create($"BingBongVoiceCarrier_{GetInstanceID()}", 4096, 1, _outputRate, false);
_source = gameObject.AddComponent<AudioSource>();
_source.clip = _carrier;
_source.loop = true;
_source.playOnAwake = false;
if (template != null)
{
// Copy the vanilla voice-line source's 3D setup rather than guessing, so the live
// voice attenuates exactly like BingBong's own lines. Routing through its mixer
// group also means the game's volume sliders apply for free.
_source.spatialBlend = template.spatialBlend;
_source.minDistance = template.minDistance;
_source.maxDistance = template.maxDistance;
_source.rolloffMode = template.rolloffMode;
_source.dopplerLevel = template.dopplerLevel;
_source.spread = template.spread;
_source.outputAudioMixerGroup = template.outputAudioMixerGroup;
}
else
{
_source.spatialBlend = 0f; // debug instance: non-positional, always audible
}
_source.Play();
AudioSettings.OnAudioConfigurationChanged += OnAudioConfigChanged;
}
private void OnAudioConfigChanged(bool deviceChanged)
{
// Owning the resample ratio ourselves makes this a one-line update instead of a rebuild.
_outputRate = AudioSettings.outputSampleRate;
_gainStep = 1f / (_outputRate * 0.015f);
_state = State.Buffering;
}
private void Update()
{
_stream = BingBongVoiceModule.Instance?.Client?.Stream;
_targetGain = Time.time < DuckUntil
? ConfigHandler.BingBongVoiceDuckVolume.Value
: ConfigHandler.BingBongVoiceVolume.Value;
if (!_reportedBlock && _blockFrames > 0)
{
_reportedBlock = true;
AudioSettings.GetDSPBufferSize(out int dspLength, out int dspCount);
Debug.Log($"[BingBongVoice] DSP block is {_blockFrames} frames ({_blockFrames * 1000 / Math.Max(1, _outputRate)}ms at {_outputRate}Hz); buffer {dspLength}x{dspCount}.");
}
if (Time.time - _lastReportTime > 5f && _underruns > 0)
{
_lastReportTime = Time.time;
int depthMs = _depthSamples * 1000 / Math.Max(1, _streamRate);
int minMs = _minDepth == int.MaxValue ? -1 : _minDepth * 1000 / Math.Max(1, _streamRate);
int targetMs = TargetSamples(_streamRate) * 1000 / Math.Max(1, _streamRate);
long writeNow = _stream?.Ring.WritePosition ?? 0;
float elapsed = Time.time - _lastRateTime;
float producedRatio = elapsed > 0.01f && _lastWritePos > 0
? (writeNow - _lastWritePos) / (elapsed * Math.Max(1, _streamRate))
: 1f;
_lastWritePos = writeNow;
_lastRateTime = Time.time;
string hint = producedRatio < 0.95f
? $" - SENDER is only producing {producedRatio:P0} of real time"
: depthMs <= 1 ? " - CONSUMER is draining faster than real time" : "";
Debug.Log($"[BingBongVoice] buffer health: {_underruns} underruns, depth {depthMs}ms (min {minMs}ms) of {targetMs}ms{hint}");
_underruns = 0;
_minDepth = int.MaxValue;
}
}
// How much audio to hold before playing. One DSP block is consumed on every call, so the
// floor is (target - block); keeping two blocks plus the configured margin leaves that floor
// comfortably above the point where a read cannot be satisfied.
private int TargetSamples(int streamRate)
{
int blockStream = (int)(_blockFrames * (double)streamRate / Math.Max(1, _outputRate)) + 1;
return blockStream * 2 + (int)(streamRate * ConfigHandler.BingBongVoiceJitterMs.Value / 1000f);
}
// ---- audio thread from here down -------------------------------------------------
// Allocation-free, lock-free, no Unity API calls, no logging. An exception escaping onto
// the DSP thread is not recoverable, so everything is wrapped and falls back to silence.
private void OnAudioFilterRead(float[] data, int channels)
{
try
{
Fill(data, Math.Max(1, channels));
}
catch
{
Array.Clear(data, 0, data.Length);
_amplitude = 0f;
if (++_strikes >= 3) _dead = true;
}
}
private void Fill(float[] data, int channels)
{
int frames = data.Length / channels;
_blockFrames = frames;
VoiceStream? stream = _stream;
if (_dead || stream == null)
{
Silence(data);
return;
}
VoiceRingBuffer ring = stream.Ring;
long write = ring.WritePosition;
double ratio = (double)ring.SampleRate / _outputRate;
int targetSamples = TargetSamples(ring.SampleRate);
if (_state == State.Buffering)
{
if (write - (long)_read < targetSamples || write < targetSamples)
{
Silence(data);
return;
}
_read = write - targetSamples;
_state = State.Playing;
}
// Drifted too far behind (sender burst, or we were paused). Rejoin the live edge rather
// than playing further and further into the past.
if (write - (long)_read > targetSamples * 5)
{
_read = write - targetSamples;
}
double needed = frames * ratio;
long available = write - (long)_read;
_depthSamples = (int)Math.Min(int.MaxValue, available);
if (_depthSamples < _minDepth) _minDepth = _depthSamples;
_streamRate = ring.SampleRate;
int usableFrames = frames;
if (available < needed + 2)
{
usableFrames = (int)Math.Max(0, (available - 2) / ratio);
if (usableFrames > frames) usableFrames = frames;
_underruns++;
// Only rebuild the whole buffer when there was essentially nothing there. A brief
// shortfall should cost a few samples, not a fade plus a full buffer of silence.
if (usableFrames < frames / 4) _state = State.Buffering;
}
Render(data, ring, channels, usableFrames, ratio, fadeOut: usableFrames < frames);
// Pad any frames we could not fill.
for (int f = usableFrames; f < frames; f++)
for (int c = 0; c < channels; c++)
data[f * channels + c] = 0f;
}
private void Render(float[] data, VoiceRingBuffer ring, int channels, int frames, double ratio, bool fadeOut)
{
float peak = 0f;
float fadeStep = fadeOut && frames > 0 ? _gain / frames : 0f;
for (int f = 0; f < frames; f++)
{
long i0 = (long)_read;
float frac = (float)(_read - i0);
float sample = ring[i0] * (1f - frac) + ring[i0 + 1] * frac;
if (fadeOut) _gain = Math.Max(0f, _gain - fadeStep);
else _gain += (_targetGain - _gain) * _gainStep;
float outSample = sample * _gain;
int baseIndex = f * channels;
for (int c = 0; c < channels; c++) data[baseIndex + c] = outSample;
float abs = outSample < 0f ? -outSample : outSample;
if (abs > peak) peak = abs;
_read += ratio;
}
_amplitude = peak;
}
private void Silence(float[] data)
{
Array.Clear(data, 0, data.Length);
_gain = 0f;
_amplitude = 0f;
}
// ----------------------------------------------------------------------------------
private void OnDestroy()
{
// The audio thread can and will run the filter after Destroy happens on the main thread,
// so flag it dead first: that check is what makes the teardown safe.
_dead = true;
AudioSettings.OnAudioConfigurationChanged -= OnAudioConfigChanged;
if (_source != null)
{
_source.Stop();
_source.clip = null;
}
if (_carrier != null) Destroy(_carrier);
}
private void OnDisable()
{
if (_source != null) _source.Stop();
_state = State.Buffering;
}
private void OnEnable()
{
// Rejoin live rather than trying to resume where we left off.
_state = State.Buffering;
if (_source != null && _carrier != null) _source.Play();
}
}

View File

@@ -0,0 +1,71 @@
using System;
using System.Threading;
namespace JordanMod.Modules.BingBongVoice;
/// <summary>
/// Ring of mono float samples at the stream's sample rate. Single producer (the websocket
/// receive thread), multiple consumers (one <see cref="BingBongVoiceSource"/> per BingBong,
/// each reading from the audio thread with its own cursor).
///
/// Lock-free by construction: the producer publishes <see cref="WritePosition"/> only after the
/// samples are in place, and the ring is far deeper than the jitter target, so a consumer can
/// never observe a slot that is being overwritten under it. Consumers hold absolute positions
/// and resync if they ever fall further behind than the ring is long.
/// </summary>
public sealed class VoiceRingBuffer
{
private readonly float[] _buf;
private readonly int _mask;
private long _write;
public int SampleRate { get; }
public int Capacity => _buf.Length;
/// Total samples ever written. Monotonic; never reset, so consumer cursors stay meaningful.
public long WritePosition => Volatile.Read(ref _write);
public VoiceRingBuffer(int sampleRate, float seconds = 4f)
{
SampleRate = sampleRate;
int wanted = Math.Max(1024, (int)(sampleRate * seconds));
int size = 1;
while (size < wanted) size <<= 1; // power of two so the wrap is a mask, not a modulo
_buf = new float[size];
_mask = size - 1;
}
/// Producer thread only. Converts interleaved little-endian PCM16 straight out of the
/// websocket's byte[] with no intermediate allocation.
public void WritePcm16(byte[] src, int offset, int byteCount)
{
int samples = byteCount >> 1;
if (samples <= 0) return;
long w = _write; // safe unsynchronised read: this thread is the only writer
for (int i = 0; i < samples; i++)
{
int b = offset + (i << 1);
short s = (short)(src[b] | (src[b + 1] << 8));
_buf[(int)((w + i) & _mask)] = s * (1f / 32768f);
}
Volatile.Write(ref _write, w + samples); // publish only once the data is really there
}
/// Producer thread only. Used to reproduce an exact gap the sender told us about, so a
/// dropped frame costs its own duration rather than shifting everything after it.
public void WriteSilence(int samples)
{
if (samples <= 0) return;
if (samples > _buf.Length) samples = _buf.Length;
long w = _write;
for (int i = 0; i < samples; i++) _buf[(int)((w + i) & _mask)] = 0f;
Volatile.Write(ref _write, w + samples);
}
/// Consumer read. The caller is responsible for staying within
/// <see cref="Capacity"/> of <see cref="WritePosition"/>.
public float this[long index] => _buf[(int)(index & _mask)];
}

View File

@@ -1,5 +1,6 @@
using System; using System;
using UnityEngine; using UnityEngine;
using Zorro.Core;
namespace JordanMod.Modules.EasyBackpack; namespace JordanMod.Modules.EasyBackpack;
@@ -54,12 +55,26 @@ class EasyBackpackModule : Module
} }
else return; else return;
// The wheel we open is the one for the backpack worn on the target's back, and its
// on-back visuals are what BackpackWheel.Choose() pulls items out of. While the
// backpack is held in hand (its slot, ID 3, is the equipped one) CharacterBackpackHandler
// deactivates those visuals, so anything taken out of the wheel is spawned into a
// disabled rig and vanishes. Refuse to open in that state; the game's own
// hold-interact on the held backpack still works and opens the correct wheel.
if (IsBackpackInHands(targetCharacter)) return;
BackpackReference backpackRefs = BackpackReference.GetFromEquippedBackpack(targetCharacter); BackpackReference backpackRefs = BackpackReference.GetFromEquippedBackpack(targetCharacter);
int slotCount = (targetSlot.prefab as Backpack)?.slotCount ?? 4; int slotCount = (targetSlot.prefab as Backpack)?.slotCount ?? 4;
GUIManager.instance.OpenBackpackWheel(backpackRefs, slotCount, targetSlot.backpackType); GUIManager.instance.OpenBackpackWheel(backpackRefs, slotCount, targetSlot.backpackType);
_isBackpackOpen = true; _isBackpackOpen = true;
} }
private static bool IsBackpackInHands(Character character)
{
Optionable<byte> selectedSlot = character.refs.items.currentSelectedSlot;
return selectedSlot.IsSome && selectedSlot.Value == character.player.backpackSlot.itemSlotID;
}
private void CloseBackpack() private void CloseBackpack()
{ {
if (!Application.isFocused) return; if (!Application.isFocused) return;

View File

@@ -0,0 +1,28 @@
using System;
using UnityEngine;
namespace JordanMod.Modules.OpenPassport;
[Module(Enabled = true)]
class OpenPassportModule : Module
{
public override string ModuleName => "Open Passport Module";
public override Type[] GetPatches() { return []; }
public override void Update()
{
if (!Input.GetKeyDown(ConfigHandler.OpenPassport.Value)) return;
// PassportManager.instance is a plain static that survives a scene unload, so it can
// be a destroyed object here; the Unity == operator is what catches that. Its
// Initialize() also reads Character.localCharacter.characterName, so don't open it
// before we have a character.
PassportManager passportManager = PassportManager.instance;
if (passportManager == null) return;
if (Character.localCharacter == null) return;
if (passportManager.isOpen) return;
passportManager.ToggleOpen();
}
}

View File

@@ -12,6 +12,8 @@ class ReplaceBingBongModule : Module
{ {
public override string ModuleName => "Replace BingBong Module"; public override string ModuleName => "Replace BingBong Module";
public const string BingBongSubtitleID = "IDK_FUNNY";
public static bool HasReplacedSounds = false; public static bool HasReplacedSounds = false;
public static BingBongResponseData[] OriginalResponsesData = []; public static BingBongResponseData[] OriginalResponsesData = [];
@@ -23,31 +25,64 @@ class ReplaceBingBongModule : Module
public override void Initialize() public override void Initialize()
{ {
base.Initialize(); base.Initialize();
LocalizedText.mainTable.Add("idk_funny", ["Test subtitle!"]); // LocalizedText.GetText upper-cases the id before looking it up, so a lowercase key never
// resolved at all. It also indexes the list by CURRENT_LANGUAGE, so a single-entry list
// threw IndexOutOfRange (caught and logged by GetText) for every non-English player.
List<string> subtitleLocalizations = new(LocalizedText.LANGUAGE_COUNT);
for (int i = 0; i < LocalizedText.LANGUAGE_COUNT; i++) subtitleLocalizations.Add("Test subtitle!");
LocalizedText.mainTable[BingBongSubtitleID] = subtitleLocalizations;
AudioSyncWorker.OnAudioLoadComplete += OnAudioLoadComplete; AudioSyncWorker.OnAudioLoadComplete += OnAudioLoadComplete;
} }
public override void Update()
{
base.Update();
if (Input.GetKeyDown(KeyCode.P))
{
Helper.FindItemByName("BingBong_Prop Variant", out Item? item);
if (item == null) return;
Debug.Log($"Found item: {item.name} in scene {item.gameObject.scene.name}");
}
}
private static void OnAudioLoadComplete() private static void OnAudioLoadComplete()
{ {
if (!HasReplacedSounds) return; if (!HasReplacedSounds) return;
Action_AskBingBong[] allBingBongActions = UnityEngine.Object.FindObjectsByType<Action_AskBingBong>(FindObjectsSortMode.None); RebuildResponsesForAll();
foreach (Action_AskBingBong askBingBong in allBingBongActions) { }
ReplaceBingBongResponses(askBingBong);
// Build the new set and hand it to every live instance before destroying the old one, so no
// Action_AskBingBong is ever left pointing at a destroyed SFX_Instance.
private static void RebuildResponsesForAll()
{
Action_AskBingBong.BingBongResponse[]? previous = _cachedResponses;
_cachedResponses = BuildResponses();
foreach (Action_AskBingBong askBingBong in UnityEngine.Object.FindObjectsByType<Action_AskBingBong>(FindObjectsSortMode.None))
{
askBingBong.responses = _cachedResponses;
}
if (previous == null) return;
foreach (Action_AskBingBong.BingBongResponse response in previous)
{
if (response.sfx != null) UnityEngine.Object.Destroy(response.sfx);
} }
} }
// SFX_Instance derives from ScriptableObject, which cannot be constructed with `new` --
// doing so skips Unity's native-side init, so the object misbehaves and its overloaded
// == null check reports oddly. CreateInstance is the only correct way to make one.
public static SFX_Instance CreateSFX(string name, AudioClip[] clips)
{
SFX_Instance instance = ScriptableObject.CreateInstance<SFX_Instance>();
instance.name = name;
instance.clips = clips;
return instance;
}
// Every Action_AskBingBong ends up with the same set -- it is derived purely from static
// state (OriginalResponsesData plus the loaded voice lines). This used to be rebuilt on every
// OnEnable, allocating a fresh SFX_Instance per response each time and never destroying the
// previous ones, so BingBong items cycling in and out of scope leaked steadily.
public static void ReplaceBingBongResponses(Action_AskBingBong askBingBong) public static void ReplaceBingBongResponses(Action_AskBingBong askBingBong)
{
_cachedResponses ??= BuildResponses();
askBingBong.responses = _cachedResponses;
}
private static Action_AskBingBong.BingBongResponse[]? _cachedResponses;
private static Action_AskBingBong.BingBongResponse[] BuildResponses()
{ {
Action_AskBingBong.BingBongResponse[] currentResponses = new Action_AskBingBong.BingBongResponse[OriginalResponsesData.Length]; Action_AskBingBong.BingBongResponse[] currentResponses = new Action_AskBingBong.BingBongResponse[OriginalResponsesData.Length];
for (int index = 0; index < OriginalResponsesData.Length; index++) for (int index = 0; index < OriginalResponsesData.Length; index++)
@@ -55,8 +90,6 @@ class ReplaceBingBongModule : Module
currentResponses[index] = OriginalResponsesData[index].ToBingBongResponse(); currentResponses[index] = OriginalResponsesData[index].ToBingBongResponse();
} }
askBingBong.responses = [];
Dictionary<string, SFX_Instance> sfxDict = new(); Dictionary<string, SFX_Instance> sfxDict = new();
for (int i = 0; i < currentResponses.Length; i++) for (int i = 0; i < currentResponses.Length; i++)
{ {
@@ -79,14 +112,10 @@ class ReplaceBingBongModule : Module
bool isNew = !sfxDict.ContainsKey(voice.Name); bool isNew = !sfxDict.ContainsKey(voice.Name);
if (isNew) if (isNew)
{ {
SFX_Instance sFX_Instance = new()
{
clips = [clip]
};
Action_AskBingBong.BingBongResponse newResponse = new() Action_AskBingBong.BingBongResponse newResponse = new()
{ {
sfx = sFX_Instance, sfx = CreateSFX(voice.Name, [clip]),
subtitleID = "idk_funny", subtitleID = BingBongSubtitleID,
mouthCurve = null, mouthCurve = null,
mouthCurveTime = 1f mouthCurveTime = 1f
}; };
@@ -98,11 +127,7 @@ class ReplaceBingBongModule : Module
} }
} }
askBingBong.responses = new Action_AskBingBong.BingBongResponse[currentResponses.Length]; return currentResponses;
for (int i = 0; i < currentResponses.Length; i++)
{
askBingBong.responses[i] = currentResponses[i];
}
} }
@@ -122,11 +147,7 @@ public class BingBongResponseData
{ {
return new Action_AskBingBong.BingBongResponse return new Action_AskBingBong.BingBongResponse
{ {
sfx = new SFX_Instance sfx = ReplaceBingBongModule.CreateSFX(SfxName, (AudioClip[])Clips.Clone()),
{
name = SfxName,
clips = (AudioClip[])Clips.Clone()
},
subtitleID = SubtitleID, subtitleID = SubtitleID,
mouthCurve = MouthCurve, mouthCurve = MouthCurve,
mouthCurveTime = MouthCurveTime mouthCurveTime = MouthCurveTime

View File

@@ -31,17 +31,12 @@ class StashedBugleModule : Module
{ {
if (heldItem.UIData.itemName == _bugleItemName) if (heldItem.UIData.itemName == _bugleItemName)
{ {
localCharacter.refs.items.DestroyHeldItemRpc(); ClearHeldSlot(localCharacter);
localCharacter.player.EmptySlot(localCharacter.refs.items.currentSelectedSlot);
localCharacter.player.RPCRemoveItemFromSlot(localCharacter.refs.items.currentSelectedSlot.Value);
SpawnItemByDisplayName(localCharacter, _megaphoneItemName); SpawnItemByDisplayName(localCharacter, _megaphoneItemName);
} }
else if (heldItem.UIData.itemName == _megaphoneItemName) else if (heldItem.UIData.itemName == _megaphoneItemName)
{ {
localCharacter.refs.items.DestroyHeldItemRpc(); ClearHeldSlot(localCharacter);
localCharacter.player.EmptySlot(localCharacter.refs.items.currentSelectedSlot);
localCharacter.player.RPCRemoveItemFromSlot(localCharacter.refs.items.currentSelectedSlot.Value);
} }
} }
else if (heldItem == null) else if (heldItem == null)
@@ -59,6 +54,16 @@ class StashedBugleModule : Module
} }
} }
// Player.EmptySlot already tells the host to clear the slot (it RPCs
// RPCRemoveItemFromSlot to the master client when we aren't the host), so calling
// RPCRemoveItemFromSlot ourselves on top of it only logged "Only Master Client can
// remove items!" on every non-host press.
private static void ClearHeldSlot(Character character)
{
character.refs.items.DestroyHeldItemRpc();
character.player.EmptySlot(character.refs.items.currentSelectedSlot);
}
// SpawnItemInHand expects the item's Resources prefab name, which is not // SpawnItemInHand expects the item's Resources prefab name, which is not
// guaranteed to match its UIData.itemName display string, so resolve it // guaranteed to match its UIData.itemName display string, so resolve it
// via the ItemDatabase instead of assuming the two are identical. // via the ItemDatabase instead of assuming the two are identical.

View File

@@ -1,4 +1,7 @@
using System;
using System.Collections.Generic;
using HarmonyLib; using HarmonyLib;
using Peak;
using Photon.Pun; using Photon.Pun;
using UnityEngine; using UnityEngine;
@@ -7,34 +10,48 @@ namespace JordanMod.Modules.BagsForEveryone;
public class BagsForEveryonePatch public class BagsForEveryonePatch
{ {
private const string BackpackSpawnerName = "Backpack_Spawner";
private const string FirstBiomeName = "Biome_1";
[HarmonyPatch(typeof(SingleItemSpawner), "TrySpawnItems")] [HarmonyPatch(typeof(SingleItemSpawner), "TrySpawnItems")]
[HarmonyPostfix] [HarmonyPostfix]
static void Postfix(SingleItemSpawner __instance) static void Postfix(SingleItemSpawner __instance, List<PhotonView> __result)
{
ForEachPlayer(__instance);
}
private static void ForEachPlayer(SingleItemSpawner spawner)
{ {
if (!Helper.IsMasterClient()) return; if (!Helper.IsMasterClient()) return;
if (__instance.transform.name != BackpackSpawnerName) return;
bool isBackpackSPawner = spawner.transform.name == "Backpack_Spawner"; GameObject firstBiome = GameObject.Find(FirstBiomeName);
if (!isBackpackSPawner) return; if (firstBiome == null || !__instance.transform.IsChildOf(firstBiome.transform)) return;
if (__result != null && __result.Count > 1) return;
bool isFirstSpawner = spawner.transform.IsChildOf(GameObject.Find("Biome_1").transform); int extraBags = PhotonNetwork.PlayerList.Length - 1;
if (!isFirstSpawner) return; if (extraBags <= 0) return;
int playerCount = PhotonNetwork.PlayerList.Length - 1; // Already one bag try
if (playerCount <= 0) return; {
for (int i = 1; i <= playerCount; i++) SpawnExtraBags(__instance, extraBags);
}
catch (Exception e)
{
Plugin.Log.LogError($"Failed to spawn extra backpacks, continuing without them: {e}");
}
}
private static void SpawnExtraBags(SingleItemSpawner spawner, int count)
{
List<PhotonView> spawned = new(count);
for (int i = 1; i <= count; i++)
{ {
Vector3 spawnPosition = spawner.transform.position + Vector3.up * 0.1f + Vector3.right * i; Vector3 spawnPosition = spawner.transform.position + Vector3.up * 0.1f + Vector3.right * i;
PhotonView component = PhotonNetwork.InstantiateItemRoom(spawner.prefab.name, spawnPosition, spawner.transform.rotation).GetComponent<PhotonView>(); PhotonView view = PhotonNetwork.InstantiateItemRoom(spawner.prefab.name, spawnPosition, spawner.transform.rotation).GetComponent<PhotonView>();
if (spawner.isKinematic) if (spawner.isKinematic)
{ {
component.GetComponent<PhotonView>().RPC("SetKinematicRPC", RpcTarget.AllBuffered, true, component.transform.position, component.transform.rotation); view.RPC("SetKinematicRPC", RpcTarget.AllBuffered, true, view.transform.position, view.transform.rotation);
} }
spawned.Add(view);
} }
if (spawner.HasSpawnTracking(out SpawnedItemTracker tracker)) tracker.TrackSpawnedItems(spawned);
} }
} }

View File

@@ -10,16 +10,23 @@ namespace JordanMod.Modules.BetterBugle;
public class BetterBuglePatch public class BetterBuglePatch
{ {
private static readonly List<string> SupportedItemNames = ["Bugle", "Bugle_Magic", "Megaphone"];
[HarmonyPatch(typeof(Item), "Start")] [HarmonyPatch(typeof(Item), "Start")]
[HarmonyPostfix] [HarmonyPostfix]
static void ItemStartPostfix(Item __instance) static void ItemStartPostfix(Item __instance)
{ {
if (__instance.itemState != ItemState.Held) return;
if (__instance.UIData == null) return; if (__instance.UIData == null) return;
List<string> supportedItemNames = ["Bugle", "Bugle_Magic", "Megaphone"]; if (!SupportedItemNames.Contains(__instance.UIData.itemName)) return;
if (!supportedItemNames.Contains(__instance.UIData.itemName)) return; // This used to bail unless itemState was already Held. ItemState.Ground is the enum's
// zero value and Item.Start doesn't set state, so anything spawned into the world never
// got a BetterBugleSFX -- and since BugleSFXUpdatePostfix silences the vanilla BugleSFX,
// a Bugle picked up off the ground was completely silent. It only ever worked because
// SpawnItemInHand calls Interact() on the same frame it instantiates, landing
// SetState(Held) before Start runs.
if (__instance.TryGetComponent<BetterBugleSFX>(out _)) return;
{ {
Action secondaryAction = OnRightClick; Action secondaryAction = OnRightClick;
Action<float> scrollAction = OnScroll; Action<float> scrollAction = OnScroll;
@@ -120,6 +127,9 @@ public class BetterBuglePatch
[HarmonyPostfix] [HarmonyPostfix]
static void BugleSFXUpdatePostfix(BugleSFX __instance) static void BugleSFXUpdatePostfix(BugleSFX __instance)
{ {
// Only silence the vanilla bugle where our replacement actually took over. Muting
// unconditionally meant any bugle we failed to attach to played nothing at all.
if (!__instance.TryGetComponent<BetterBugleSFX>(out _)) return;
if (__instance.volume > 0f) __instance.volume = 0; if (__instance.volume > 0f) __instance.volume = 0;
} }

View File

@@ -3,6 +3,7 @@ using System.Collections;
using System.Collections.Generic; using System.Collections.Generic;
using System.IO; using System.IO;
using System.Linq; using System.Linq;
using System.Net;
using System.Security.Cryptography; using System.Security.Cryptography;
using System.Threading.Tasks; using System.Threading.Tasks;
using JordanMod.Modules.BetterBugle; using JordanMod.Modules.BetterBugle;
@@ -14,18 +15,33 @@ namespace JordanMod.Utils;
class AudioSyncService class AudioSyncService
{ {
public static string API_BASE_URL => ConfigHandler.BugleSoundAPIURL.Value; // Normalised once here rather than at each call site: the configured value is hand-typed, so
// it may be missing a scheme or carry a trailing slash, either of which broke the Uri parse.
public static string API_BASE_URL => Helper.EnsureScheme(ConfigHandler.BugleSoundAPIURL.Value, "https", "http").TrimEnd('/');
public async static Task<bool> DownloadAPIAudio(APIAudioFormat apiAudio, string SoundsDirectory, Song? existingSong = null) public static bool DownloadAPIAudio(APIAudioFormat apiAudio, string SoundsDirectory, Song? existingSong = null)
{ {
bool success = true; bool success = true;
try try
{ {
// Same audio under a different name in the bank: retire the local copy.
if (existingSong != null && apiAudio.Filename != existingSong.Name) if (existingSong != null && apiAudio.Filename != existingSong.Name)
{ {
File.Delete(Path.Combine(SoundsDirectory, $"{existingSong.Name}.{existingSong.Extension}")); string renamedFrom = Path.Combine(SoundsDirectory, $"{existingSong.Name}.{existingSong.Extension}");
if (File.Exists(renamedFrom)) AudioSyncWorker.TryQuarantine(renamedFrom);
} }
await apiAudio.DownloadToFolder(SoundsDirectory);
// Same name under a different extension -- a leftover .mp3 when the bank now serves
// .ogg, say. Both would be queued by the loader and fight over the same song name, so
// retire every variant that isn't the one we're about to write.
foreach (string ext in AudioSyncWorker.AudioTypes.Keys)
{
if (string.Equals(ext, apiAudio.Extension, StringComparison.OrdinalIgnoreCase)) continue;
string stale = Path.Combine(SoundsDirectory, $"{apiAudio.Filename}.{ext}");
if (File.Exists(stale)) AudioSyncWorker.TryQuarantine(stale);
}
apiAudio.DownloadToFolder(SoundsDirectory);
} }
catch (Exception ex) catch (Exception ex)
{ {
@@ -46,7 +62,7 @@ class AudioSyncService
return audioClips; return audioClips;
} }
using var client = new System.Net.WebClient(); using var client = new WebClient();
try try
{ {
string json = client.DownloadString(uri); string json = client.DownloadString(uri);
@@ -63,15 +79,19 @@ class AudioSyncService
public static void ClearAudioClips() public static void ClearAudioClips()
{ {
// One scene scan shared by every song. Song.Dispose() used to run its own
// FindObjectsByType<AudioSource>() call, so clearing a bank of N songs meant N full
// scans of the scene -- the bulk of the freeze on the Bugle's right-click refresh.
AudioSource[] audioSources = UnityEngine.Object.FindObjectsByType<AudioSource>(FindObjectsSortMode.None);
foreach (Song song in Song.Sounds.Values.ToList()) foreach (Song song in Song.Sounds.Values.ToList())
{ {
song.Dispose(); song.Dispose(audioSources);
} }
Song.Sounds.Clear(); Song.Sounds.Clear();
Song.SoundsByHash.Clear();
Song.Songs.Clear(); Song.Songs.Clear();
Song.BB_VoiceLines.Clear(); Song.BB_VoiceLines.Clear();
GC.Collect(); // No GC.Collect() here on purpose: the audio memory is native and is already released by
// Object.Destroy on each clip, so a blocking full collection only added a stall.
} }
public class APIAudioFormat public class APIAudioFormat
@@ -100,7 +120,10 @@ class AudioSyncService
[JsonProperty("owner")] [JsonProperty("owner")]
public string Owner { get; set; } = string.Empty; public string Owner { get; set; } = string.Empty;
public async Task DownloadToFolder(string folderPath) // Plain WebClient rather than UnityWebRequest: this runs on a background thread, and
// UnityWebRequest is main-thread only. The old version also spun on `await Task.Yield()`
// waiting for an operation that a pool thread never drives, burning a core.
public void DownloadToFolder(string folderPath)
{ {
if (string.IsNullOrEmpty(Filename) || string.IsNullOrEmpty(Extension)) if (string.IsNullOrEmpty(Filename) || string.IsNullOrEmpty(Extension))
{ {
@@ -117,15 +140,9 @@ class AudioSyncService
string url = $"{API_BASE_URL}/audio/{Id}/download?hash={Hash}"; string url = $"{API_BASE_URL}/audio/{Id}/download?hash={Hash}";
Debug.Log($"Downloading audio from URL: {url}"); Debug.Log($"Downloading audio from URL: {url}");
using UnityWebRequest www = UnityWebRequest.Get(url); using var client = new WebClient();
var operation = www.SendWebRequest(); byte[] data = client.DownloadData(url);
while (!operation.isDone) await Task.Yield(); File.WriteAllBytes(filePath, data);
if (www.result != UnityWebRequest.Result.Success)
{
Debug.LogError($"Failed to download API audio: {www.error}");
return;
}
File.WriteAllBytes(filePath, www.downloadHandler.data);
} }
} }
@@ -142,6 +159,11 @@ class AudioSyncWorker
} }
public static readonly string SoundsDirectory = Path.Combine(BepInEx.Paths.BepInExRootPath, "bugleSounds"); public static readonly string SoundsDirectory = Path.Combine(BepInEx.Paths.BepInExRootPath, "bugleSounds");
// Files the sync pulls out of the bank folder are moved here rather than deleted. It's a
// subfolder of the bank itself, which is safe because the loader's Directory.GetFiles calls
// are non-recursive and will never see it.
public static readonly string QuarantineDirectory = Path.Combine(SoundsDirectory, "_removed");
public static readonly Dictionary<string, AudioType> AudioTypes = new() public static readonly Dictionary<string, AudioType> AudioTypes = new()
{ {
{ "wav", AudioType.WAV }, { "wav", AudioType.WAV },
@@ -153,6 +175,11 @@ class AudioSyncWorker
public static bool IsLoading = false; public static bool IsLoading = false;
public static bool IsSyncing = false; public static bool IsSyncing = false;
// How many clips arrived from the download handler without their audio data decoded. Any
// number above zero means the first play of each would otherwise have stalled the main
// thread; reported once per load so the log can confirm it.
public static int ClipsNeedingPreload = 0;
public static int CurrentSongIndex = 0; public static int CurrentSongIndex = 0;
public static string CurrentSongName = "None"; public static string CurrentSongName = "None";
@@ -169,14 +196,66 @@ class AudioSyncWorker
public static void TrySyncAndLoadAudioClips() public static void TrySyncAndLoadAudioClips()
{ {
if (IsLoading || IsSyncing) return; if (IsLoading || IsSyncing) return;
Task.Run(() => Task.Run(SyncAndLoadAudioClips);
}
// Moves a file out of the bank folder instead of deleting it, so a mistaken sync is always
// recoverable. Pure file I/O, safe to call from the sync thread.
internal static bool TryQuarantine(string filePath)
{ {
SyncAndLoadAudioClipsCoroutine().GetAwaiter().GetResult(); try
}); {
Directory.CreateDirectory(QuarantineDirectory);
string target = Path.Combine(QuarantineDirectory, Path.GetFileName(filePath));
if (File.Exists(target))
{
string stem = Path.GetFileNameWithoutExtension(filePath);
string ext = Path.GetExtension(filePath);
target = Path.Combine(QuarantineDirectory, $"{stem}_{DateTime.Now:yyyyMMdd_HHmmss}{ext}");
if (File.Exists(target)) File.Delete(target);
}
File.Move(filePath, target);
Debug.Log($"Moved '{Path.GetFileName(filePath)}' out of the bank folder into _removed");
return true;
}
catch (Exception ex)
{
Debug.LogError($"Could not move '{filePath}' out of the sounds folder: {ex.Message}");
return false;
}
}
// Everything in the folder that the bank doesn't list gets pulled out, so every player ends
// up with an identical set. BingBong voice lines are exempt.
private static int QuarantineFilesNotInBank(HashSet<string> bankNames)
{
Dictionary<string, Song> loaded = new(Song.Sounds);
int moved = 0;
foreach (string ext in AudioTypes.Keys)
{
foreach (string file in Directory.GetFiles(SoundsDirectory, $"*.{ext}"))
{
string name = Path.GetFileNameWithoutExtension(file);
if (Song.IsBingBongVoiceLine(name)) continue;
if (bankNames.Contains(name)) continue;
if (!TryQuarantine(file)) continue;
moved++;
if (loaded.TryGetValue(name, out Song? song)) Plugin.RunOnMainThread(song.Dispose);
}
}
return moved;
}
// ShowActionbar reads Time.time and touches a MonoBehaviour, so it can't be called straight
// from the sync thread.
private static void ShowActionbarOnMainThread(string message)
{
Plugin.RunOnMainThread(() => BetterBugleUI.Instance?.ShowActionbar(message));
} }
private static IEnumerator LoadAllAudioClipsCoroutine(string directoryPath, string[]? forceReload = null) private static IEnumerator LoadAllAudioClipsCoroutine(string directoryPath, string[]? forceReload = null)
{ {
ClipsNeedingPreload = 0;
List<(string filePath, string ext, string name)> filesToLoad = new(); List<(string filePath, string ext, string name)> filesToLoad = new();
foreach (var ext in AudioTypes.Keys) foreach (var ext in AudioTypes.Keys)
@@ -212,13 +291,32 @@ class AudioSyncWorker
loadedCount += loadCoroutines.Count; loadedCount += loadCoroutines.Count;
BetterBugleUI.Instance?.ShowActionbar($"Loading audio clips... {loadedCount}/{filesToLoad.Count}"); BetterBugleUI.Instance?.ShowActionbar($"Loading audio clips... {loadedCount}/{filesToLoad.Count}");
} }
OnAudioLoadComplete?.Invoke();
if (ClipsNeedingPreload > 0)
Debug.Log($"Pre-decoded {ClipsNeedingPreload}/{filesToLoad.Count} clips that would have stalled the main thread on first play.");
// Clear the flag before notifying: a subscriber that throws must not leave IsLoading
// stuck true, which would wedge every sync and refresh for the rest of the session.
IsLoading = false; IsLoading = false;
try
{
OnAudioLoadComplete?.Invoke();
}
catch (Exception ex)
{
Debug.LogError($"An audio load handler threw: {ex}");
}
} }
private static IEnumerator LoadAudioClipCoroutine(string filePath, string ext, string name, bool forceReload = false) private static IEnumerator LoadAudioClipCoroutine(string filePath, string ext, string name, bool forceReload = false)
{ {
using UnityWebRequest www = UnityWebRequestMultimedia.GetAudioClip($"file://{filePath}", AudioTypes[ext]); using UnityWebRequest www = UnityWebRequestMultimedia.GetAudioClip($"file://{filePath}", AudioTypes[ext]);
// Keep the clip compressed in memory and let Unity decode it during playback. The default
// (DecompressOnLoad) holds raw PCM for the whole session -- roughly 10 MB per minute of
// 44.1kHz stereo, so a ~80 MB bank becomes ~1 GB resident. Compressed costs a little CPU
// per playing voice, which is nothing when one bugle is sounding, and avoids both the
// memory and the big upfront decode. Must be set before SendWebRequest.
if (www.downloadHandler is DownloadHandlerAudioClip audioHandler) audioHandler.compressed = true;
yield return www.SendWebRequest(); yield return www.SendWebRequest();
if (www.result != UnityWebRequest.Result.Success) yield break; if (www.result != UnityWebRequest.Result.Success) yield break;
@@ -236,11 +334,28 @@ class AudioSyncWorker
AudioClip audioClip = DownloadHandlerAudioClip.GetContent(www); AudioClip audioClip = DownloadHandlerAudioClip.GetContent(www);
if (audioClip == null) yield break; if (audioClip == null) yield break;
// GetContent hands back a DecompressOnLoad clip whose PCM data is decoded lazily, on
// first use. Left alone that decode happens inside the first AudioSource.Play(), on the
// main thread, freezing the game the first time anyone plays a song. Force it here
// instead: we're already on the loading path, where a stall is expected and we can yield
// through it. If the clip already arrived loaded this is a no-op.
if (audioClip.loadState != AudioDataLoadState.Loaded)
{
ClipsNeedingPreload++;
// loadInBackground is serialized on the asset and read-only at runtime, so this
// decodes synchronously. That's the point: the cost lands here rather than mid-game,
// and the batching in LoadAllAudioClipsCoroutine already yields between clips.
audioClip.LoadAudioData();
while (audioClip.loadState == AudioDataLoadState.Loading) yield return null;
}
Song song = new(name, ext, filePath, audioClip); Song song = new(name, ext, filePath, audioClip);
song.Register(); song.Register();
} }
private static async Task SyncAndLoadAudioClipsCoroutine() // Runs on a thread pool thread via Task.Run. Everything in here must stay off Unity APIs;
// hop back through Plugin.RunOnMainThread for anything that touches the engine.
private static void SyncAndLoadAudioClips()
{ {
if (IsLoading || IsSyncing) return; if (IsLoading || IsSyncing) return;
IsSyncing = true; IsSyncing = true;
@@ -248,37 +363,52 @@ class AudioSyncWorker
{ {
Dictionary<AudioSyncService.APIAudioFormat, Song?> toDownload = new(); Dictionary<AudioSyncService.APIAudioFormat, Song?> toDownload = new();
string[] existingSongNames = [.. Song.Sounds.Keys];
AudioSyncService.APIAudioFormat[] existingAPIFormats = [.. AudioSyncService.GetAudioClips()]; AudioSyncService.APIAudioFormat[] existingAPIFormats = [.. AudioSyncService.GetAudioClips()];
string[] apiExistingNames = [.. existingAPIFormats.Select(apiAudio => apiAudio.Filename)];
var songsToRemove = existingSongNames.Except(apiExistingNames).ToArray(); // A failed fetch (bad URL, offline, server error) also returns an empty list, and an
foreach (var songName in songsToRemove) // empty bank would mean "nothing belongs here" -- quarantining the entire folder. Only
// a bank that actually listed something is allowed to drive removals.
if (existingAPIFormats.Length == 0)
{ {
if (Song.Sounds.TryGetValue(songName, out var songToDispose)) Debug.LogWarning("Audio bank returned no entries; cancelling sync rather than emptying the folder.");
{ ShowActionbarOnMainThread("Audio bank unreachable or empty, sync cancelled.");
songToDispose.Dispose(); IsSyncing = false;
songToDispose.DeleteFile(); return;
}
} }
HashSet<string> bankNames = new(existingAPIFormats.Select(apiAudio => apiAudio.Filename), StringComparer.OrdinalIgnoreCase);
int quarantined = QuarantineFilesNotInBank(bankNames);
// Hashing reads the whole file off disk, so build the lookup here on the background
// thread rather than eagerly during loading where it stalled the main thread once
// per clip.
Dictionary<string, Song> soundsByHash = new();
foreach (Song song in Song.Sounds.Values.ToList())
{
// Skip anything just quarantined: its file has moved, so hashing would only fail
// and log noise. Its Dispose is already queued on the main thread.
if (!File.Exists(song.FilePath)) continue;
string hash = song.Hash;
if (!string.IsNullOrEmpty(hash)) soundsByHash[hash] = song;
}
foreach (AudioSyncService.APIAudioFormat apiAudio in existingAPIFormats) foreach (AudioSyncService.APIAudioFormat apiAudio in existingAPIFormats)
{ {
Song? existingSong = Song.SoundsByHash.GetValueOrDefault(apiAudio.Hash); Song? existingSong = soundsByHash.GetValueOrDefault(apiAudio.Hash);
if (existingSong == null || existingSong.Hash != apiAudio.Hash) if (existingSong == null || existingSong.Hash != apiAudio.Hash)
{ {
toDownload.Add(apiAudio, existingSong); toDownload.Add(apiAudio, existingSong);
} }
} }
BetterBugleUI.Instance?.ShowActionbar($"Syncing audio bank... {toDownload.Count} changed/new files found."); string quarantineNote = quarantined > 0 ? $", {quarantined} moved to _removed" : "";
ShowActionbarOnMainThread($"Syncing audio bank... {toDownload.Count} changed/new files found{quarantineNote}.");
string[] filesToOverload = []; string[] filesToOverload = [];
foreach (AudioSyncService.APIAudioFormat apiAudio in toDownload.Keys) foreach (AudioSyncService.APIAudioFormat apiAudio in toDownload.Keys)
{ {
bool success = await AudioSyncService.DownloadAPIAudio(apiAudio, SoundsDirectory, toDownload[apiAudio]); bool success = AudioSyncService.DownloadAPIAudio(apiAudio, SoundsDirectory, toDownload[apiAudio]);
if (success) if (success)
{ {
Debug.Log($"Successfully downloaded audio: {apiAudio.Filename}.{apiAudio.Extension}, adding to forceload"); Debug.Log($"Successfully downloaded audio: {apiAudio.Filename}.{apiAudio.Extension}, adding to forceload");
@@ -302,7 +432,6 @@ class AudioSyncWorker
public class Song : IDisposable public class Song : IDisposable
{ {
public static readonly Dictionary<string, Song> Sounds = new(); public static readonly Dictionary<string, Song> Sounds = new();
public static readonly Dictionary<string, Song> SoundsByHash = new();
public static readonly Dictionary<string, Song> Songs = new(); public static readonly Dictionary<string, Song> Songs = new();
public static readonly Dictionary<string, Song> BB_VoiceLines = new(); public static readonly Dictionary<string, Song> BB_VoiceLines = new();
@@ -316,6 +445,8 @@ public class Song : IDisposable
.ThenBy(name => name)]; .ThenBy(name => name)];
} }
public static bool IsBingBongVoiceLine(string name) => name.StartsWith(BingBongPrefix, StringComparison.Ordinal);
public static void UpdateRealIndices() public static void UpdateRealIndices()
{ {
var sortedNames = Songs.Keys.OrderBy(name => name).ToList(); var sortedNames = Songs.Keys.OrderBy(name => name).ToList();
@@ -328,65 +459,74 @@ public class Song : IDisposable
} }
} }
// Voice lines are personal, not part of the shared bank, so the sync never touches them.
public const string BingBongPrefix = "SFX_VO_BingBong_";
public string Name { get; set; } public string Name { get; set; }
public string Extension { get; set; } public string Extension { get; set; }
public string FilePath { get; set; } public string FilePath { get; set; }
public AudioClip AudioClip { get; } public AudioClip AudioClip { get; }
public string Hash { get; }
public int RealIndex { get; set; } public int RealIndex { get; set; }
private string? _hash;
// Computed on demand rather than in the constructor: hashing reads the entire file and runs
// SHA256 over it, and the constructor is called from the load coroutine on the main thread.
// The only consumer is the sync comparison, which already runs on a background thread.
public string Hash => _hash ??= GenerateHash(FilePath);
public Song(string name, string extension, string filePath, AudioClip audioClip) public Song(string name, string extension, string filePath, AudioClip audioClip)
{ {
Name = name; Name = name;
Extension = extension; Extension = extension;
FilePath = filePath; FilePath = filePath;
AudioClip = audioClip; AudioClip = audioClip;
Hash = GenerateHash(filePath);
} }
public void Register() public void Register()
{ {
Sounds[Name] = this; Sounds[Name] = this;
SoundsByHash[Hash] = this; if (IsBingBongVoiceLine(Name)) BB_VoiceLines[Name] = this;
if (Name.StartsWith("SFX_VO_BingBong_")) BB_VoiceLines[Name] = this;
else Songs[Name] = this; else Songs[Name] = this;
} }
public void Dispose() public void Dispose() => Dispose(null);
// knownAudioSources lets a bulk clear pass in a single scene scan instead of paying for one
// per song.
public void Dispose(AudioSource[]? knownAudioSources)
{ {
if (AudioClip == null) return; if (AudioClip == null) return;
var audioSources = UnityEngine.Object.FindObjectsByType<AudioSource>(FindObjectsSortMode.None); var audioSources = knownAudioSources ?? UnityEngine.Object.FindObjectsByType<AudioSource>(FindObjectsSortMode.None);
foreach (var audioSource in audioSources) foreach (var audioSource in audioSources)
{ {
if (audioSource.clip == AudioClip) if (audioSource != null && audioSource.clip == AudioClip)
{ {
audioSource.Stop(); audioSource.Stop();
audioSource.clip = null; audioSource.clip = null;
} }
} }
Sounds.Remove(Name); Sounds.Remove(Name);
SoundsByHash.Remove(Hash); if (IsBingBongVoiceLine(Name)) BB_VoiceLines.Remove(Name);
if (Name.StartsWith("SFX_VO_BingBong_")) BB_VoiceLines.Remove(Name);
else Songs.Remove(Name); else Songs.Remove(Name);
UnityEngine.Object.Destroy(AudioClip); UnityEngine.Object.Destroy(AudioClip);
} }
public void DeleteFile()
{
if (AudioClip == null) return;
var filePath = Path.Combine(AudioSyncWorker.SoundsDirectory, $"{Name}.{Extension}");
if (File.Exists(filePath))
{
File.Delete(filePath);
Debug.Log($"Deleted local file: {filePath}");
}
}
public string GenerateHash(string filePath) public string GenerateHash(string filePath)
{
try
{ {
using var hasher = SHA256.Create(); using var hasher = SHA256.Create();
var fileBytes = File.ReadAllBytes(filePath); var fileBytes = File.ReadAllBytes(filePath);
var hashBytes = hasher.ComputeHash(fileBytes); var hashBytes = hasher.ComputeHash(fileBytes);
return BitConverter.ToString(hashBytes).Replace("-", "").ToLowerInvariant(); return BitConverter.ToString(hashBytes).Replace("-", "").ToLowerInvariant();
} }
catch (Exception ex)
{
// A missing or locked file just means "no match", which makes the sync re-download it.
Debug.LogWarning($"Could not hash '{filePath}': {ex.Message}");
return string.Empty;
}
}
} }

View File

@@ -16,6 +16,25 @@ public static class Helper
return item; return item;
} }
/// <summary>
/// Config URLs get typed by hand, so a missing scheme is common. Adds one when there isn't
/// already a scheme, picking the insecure variant for loopback since a local dev server
/// almost never has TLS in front of it. Call as EnsureScheme(url, "https", "http") or
/// EnsureScheme(url, "wss", "ws").
/// </summary>
public static string EnsureScheme(string? url, string secureScheme, string insecureScheme)
{
string trimmed = (url ?? "").Trim();
if (trimmed.Length == 0) return "";
if (trimmed.Contains("://")) return trimmed;
bool loopback = trimmed.StartsWith("localhost", System.StringComparison.OrdinalIgnoreCase)
|| trimmed.StartsWith("127.0.0.1", System.StringComparison.Ordinal)
|| trimmed.StartsWith("[::1]", System.StringComparison.Ordinal);
return (loopback ? insecureScheme : secureScheme) + "://" + trimmed;
}
public static bool IsOnIsland() public static bool IsOnIsland()
{ {
return SceneManager.GetActiveScene().name.ToLower().StartsWith("level_") || SceneManager.GetActiveScene().name == "WilIsland"; return SceneManager.GetActiveScene().name.ToLower().StartsWith("level_") || SceneManager.GetActiveScene().name == "WilIsland";

286
tools/stub_relay.py Normal file
View File

@@ -0,0 +1,286 @@
"""Development stub relay for BingBong Live Voice.
Speaks the exact wire format the real relay (an endpoint on the AudioBank FastAPI service)
will use, so the mod can be built and tested before that backend exists. Deliberately minimal:
no auth, no session directory, no persistence. It is a test fixture, not a reference
implementation — build the real thing from the plan, not from this.
Requires `websockets` (pip install websockets).
python tools/stub_relay.py serve # relay on ws://localhost:8787/voice/ws
python tools/stub_relay.py tone # stream a 1 kHz sine into channel "test"
python tools/stub_relay.py tone --sweep # sweep 200-2000 Hz (resampling bugs are obvious)
python tools/stub_relay.py listen # dump received frames, to verify fan-out
Point the mod at it with, in BepInEx config:
[BingBong Voice]
RelayURL = ws://localhost:8787/voice/ws?token=listen:test
Leaving RelayURL empty instead derives the URL from BugleSoundAPIURL, which is where the
relay lives in production.
Connect as: ws://localhost:8787/voice/ws?token=listen:test
ws://localhost:8787/voice/ws?token=speak:test
Wire format — 12-byte little-endian header, then PCM16LE mono payload:
0 u8 magic 0xB1
1 u8 version 1
2 u8 codec 0=PCM16LE 1=mu-law
3 u8 rate code 0=16k 1=8k 2=24k 3=48k
4 u32 seq
8 u32 sampleTimestamp (samples since stream start)
Binary frame = audio. Text frame = JSON control.
"""
import argparse
import asyncio
import json
import math
import struct
import sys
from collections import defaultdict
from urllib.parse import urlparse, parse_qs
from websockets.asyncio.server import serve
from websockets.asyncio.client import connect
HOST = "localhost"
PORT = 8787
PATH = "/voice/ws"
MAGIC = 0xB1
VERSION = 1
CODEC_PCM16 = 0
RATE_CODES = {16000: 0, 8000: 1, 24000: 2, 48000: 3}
FRAME_MS = 20
HEADER = struct.Struct("<BBBBII")
def pack_header(seq: int, ts: int, rate: int = 16000, codec: int = CODEC_PCM16) -> bytes:
return HEADER.pack(MAGIC, VERSION, codec, RATE_CODES[rate], seq & 0xFFFFFFFF, ts & 0xFFFFFFFF)
# ---------------------------------------------------------------- relay
channels: dict[str, dict] = defaultdict(lambda: {"speaker": None, "listeners": set()})
def parse_token(raw_path: str):
"""token=<role>:<channel>. Returns (role, channel) or (None, None)."""
q = parse_qs(urlparse(raw_path).query)
token = (q.get("token") or [""])[0]
if ":" not in token:
return None, None
role, _, channel = token.partition(":")
if role not in ("listen", "speak") or not channel:
return None, None
return role, channel
async def handler(ws):
role, channel = parse_token(ws.request.path)
if role is None:
await ws.close(1008, "bad token")
return
ch = channels[channel]
if role == "speak":
old = ch["speaker"]
if old is not None:
try:
await old.send(json.dumps({"type": "preempted"}))
await old.close(1000, "preempted")
except Exception:
pass
ch["speaker"] = ws
print(f"[relay] speaker joined '{channel}' ({len(ch['listeners'])} listening)")
await broadcast_control(channel, {"type": "speaker_joined"})
try:
async for msg in ws:
if isinstance(msg, bytes):
await fanout(channel, msg)
except Exception:
pass
finally:
if ch["speaker"] is ws:
ch["speaker"] = None
print(f"[relay] speaker left '{channel}'")
await broadcast_control(channel, {"type": "speaker_left"})
return
ch["listeners"].add(ws)
print(f"[relay] listener joined '{channel}' -> {len(ch['listeners'])} listening")
try:
await ws.wait_closed()
finally:
ch["listeners"].discard(ws)
print(f"[relay] listener left '{channel}' -> {len(ch['listeners'])} listening")
async def fanout(channel: str, payload: bytes):
listeners = list(channels[channel]["listeners"])
if not listeners:
return
# Real relay uses a bounded per-listener queue and drops oldest; the stub just fires and
# forgets, which is fine locally and keeps this file short.
await asyncio.gather(*(safe_send(w, payload) for w in listeners), return_exceptions=True)
async def broadcast_control(channel: str, obj: dict):
msg = json.dumps(obj)
await asyncio.gather(
*(safe_send(w, msg) for w in list(channels[channel]["listeners"])),
return_exceptions=True,
)
async def safe_send(ws, payload):
try:
await ws.send(payload)
except Exception:
pass
async def cmd_serve(args):
async with serve(handler, HOST, args.port, max_size=None):
print(f"[relay] listening on ws://{HOST}:{args.port}{PATH}")
print(f"[relay] listen: ws://{HOST}:{args.port}{PATH}?token=listen:test")
print(f"[relay] speak : ws://{HOST}:{args.port}{PATH}?token=speak:test")
await asyncio.Future()
# ---------------------------------------------------------------- tone generator
try:
import numpy as _np
except ImportError:
_np = None
def generate_frame(phase, ts, n, rate, args):
"""Returns (new_phase, pcm16_bytes) for one frame, phase-continuous across frames."""
if _np is not None:
idx = _np.arange(n, dtype=_np.float64)
if args.sweep:
# 200-2000 Hz over 4 s: any resampling error shows up as an obvious warble
t = (ts + idx) / rate
freq = 200.0 + 1800.0 * (0.5 - 0.5 * _np.cos(2 * _np.pi * t / 4.0))
else:
freq = _np.full(n, float(args.freq))
ph = phase + _np.cumsum(2 * _np.pi * freq / rate)
samples = (_np.sin(ph) * 0.35 * 32767).astype("<i2")
return float(ph[-1] % (2 * _np.pi)), samples.tobytes()
# Fallback without numpy. Fine for a steady tone; sweep may not hold real time.
out = []
for i in range(n):
freq = args.freq
if args.sweep:
t = (ts + i) / rate
freq = 200.0 + 1800.0 * (0.5 - 0.5 * math.cos(2 * math.pi * t / 4.0))
phase = (phase + 2 * math.pi * freq / rate) % (2 * math.pi)
out.append(int(math.sin(phase) * 0.35 * 32767))
return phase, struct.pack(f"<{n}h", *out)
async def cmd_tone(args):
url = f"ws://{HOST}:{args.port}{PATH}?token=speak:{args.channel}"
rate = args.rate
n = int(rate * FRAME_MS / 1000)
print(f"[tone] {url} {rate} Hz, {n} samples/frame ({FRAME_MS} ms)")
loop = asyncio.get_running_loop()
async with connect(url, max_size=None) as ws:
seq = 0
ts = 0
phase = 0.0
# Deadline-based pacing. A plain `sleep(FRAME_MS)` per iteration sleeps for *at least*
# that long and then adds the generate+send time on top, so the sender silently drifts
# behind real time and starves the receiver's jitter buffer. Track an absolute deadline
# and sleep only for the remainder instead.
next_deadline = loop.time()
late_frames = 0
while True:
# Vectorised: generating 320 samples a frame in a Python loop costs more than the
# 20 ms frame budget once sweep mode adds a second transcendental per sample, and the
# sender then silently falls behind real time and starves the receiver.
phase, payload = generate_frame(phase, ts, n, rate, args)
await ws.send(pack_header(seq, ts, rate) + payload)
seq += 1
ts += n
next_deadline += FRAME_MS / 1000
delay = next_deadline - loop.time()
if delay > 0:
await asyncio.sleep(delay)
else:
# Generation itself overran the frame budget; we can't catch up by sleeping.
late_frames += 1
next_deadline = loop.time()
if seq % 50 == 0:
print(f"\r[tone] sent {seq} frames ({ts / rate:6.1f}s), {late_frames} late", end="", flush=True)
# ---------------------------------------------------------------- listener (verification)
async def cmd_listen(args):
url = f"ws://{HOST}:{args.port}{PATH}?token=listen:{args.channel}"
print(f"[listen] {url}")
async with connect(url, max_size=None) as ws:
count = 0
last_seq = None
gaps = 0
async for msg in ws:
if isinstance(msg, str):
print(f"\n[listen] control: {msg}")
continue
magic, ver, codec, rc, seq, ts = HEADER.unpack_from(msg, 0)
if magic != MAGIC or ver != VERSION:
print(f"\n[listen] BAD HEADER magic={magic:#x} ver={ver}")
continue
if last_seq is not None and seq != last_seq + 1:
gaps += 1
print(f"\n[listen] SEQ GAP {last_seq} -> {seq}")
last_seq = seq
count += 1
if count % 50 == 0:
print(
f"\r[listen] {count} frames, {len(msg) - 12} B payload, codec={codec} "
f"rate={rc} ts={ts} gaps={gaps}",
end="",
flush=True,
)
def main():
p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
p.add_argument("--port", type=int, default=PORT)
sub = p.add_subparsers(dest="cmd", required=True)
sub.add_parser("serve")
t = sub.add_parser("tone")
t.add_argument("--channel", default="test")
t.add_argument("--freq", type=float, default=1000.0)
t.add_argument("--rate", type=int, default=16000, choices=sorted(RATE_CODES))
t.add_argument("--sweep", action="store_true")
l = sub.add_parser("listen")
l.add_argument("--channel", default="test")
args = p.parse_args()
fn = {"serve": cmd_serve, "tone": cmd_tone, "listen": cmd_listen}[args.cmd]
try:
asyncio.run(fn(args))
except KeyboardInterrupt:
print("\nbye")
if __name__ == "__main__":
sys.exit(main())