Player: FluidSynth/Lite: Stuff to fix

[Ghabry]

Audio issues:

• [x] Investigate wrong CC command (Drums) Backport a fix?
• [ ] Stuck instruments when BGM_Stop -> BGM_Play or Exit Game -> Game browser -> Start game
• [ ] Further performance investigations

Better integration:

• [ ] Default search paths (see WildMidi/Timidity)
• [ ] Bundle SF2 with Player (Jenkins jobs)
• [x] Emscripten support (fetches easyrpg.soundfont)

Other:

• [x] Buildfix for Fluidsynth v1.1.11

[fdelapena]

Additional system soundfont search paths:

Fedora/CentOS/RHEL fluidsynth has a default soundfont concept, if there is a different soundfont not being Fluid R3:

/usr/share/soundfonts/default.sf2

Arch (soundfont-fluid), Fedora/CentOS/RHEL (fluid-soundfont-gm):

/usr/share/soundfonts/FluidR3_GM.sf2

Debian/Ubuntu, openSUSE/SLED/SLES (fluid-soundfont-gm):

/usr/share/sounds/sf2/FluidR3_GM.sf2

[fmatthew5876]

I just tried this by playing hh3 with a debug build in Windows 10 using fluidlite. I used scc1t2.sf2 as easyrpg.soundfont

I played a bit of the intro, then went to "end game" from the menu. Now Player is hung with a black screen upon returning to the title.

The intro is playing a midi, the title screen mp3.

    [External Code] 
>   Player.exe!WASAPI_WaitDevice(SDL_AudioDevice * this) Line 316   C
    Player.exe!SDL_RunAudio(void * devicep) Line 758    C
    Player.exe!SDL_RunThread(void * data) Line 289  C
    [Inline Frame] Player.exe!RunThread(void *) Line 90 C
    Player.exe!RunThreadViaCreateThread(void * data) Line 99    C
    [External Code] 

Looks like some kind of deadlock in the audio subsystem.

Unfortunately this isn't reproducable :(

[fmatthew5876]

Another issue,

When I play my midis in foobar2000, using BASSMIDI and scc1t2.sf2 they sound fine.

When I play with Player using fluidlite and the same soundfont, I hear a faint crackling sound, like you would from an vinyl record player.

[fmatthew5876]

This midi clearly shows the percussion issue in fluidlite:

https://github.com/fmatthew5876/hh3-rm2k/blob/master/Music/HH3%20Castle%20of%20Life.mid

The drums are replaced by some other low sounding instrument.

It also displays a high amount of the crackle effect described previously.

[fdelapena]

When I play with Player using fluidlite and the same soundfont, I hear a faint crackling sound, like you would from an vinyl record player.

Same here with stock FluidSynth 2.1.1 in Player (fails to build with 1.11.1 because uses 2.x API). It was happening already with the previous interpolation mode (7). @elsemieni's uploaded rendered tests were also affected.
I've tried the FluidSynth 2.1.1 CLI tool with same args (gain 0.6, sampling rate 44100, chorus off, reverb off) and it plays without noise. Same with 1.11.1 CLI.

Other stuff I've found:

• Apart of the stop/play not stopping notes, after the loopstart, some midis with active notes get stuck after returning to the loopstart position. Not sure how RPG_RT handles the note length, if it respects them or they makes an immediate note off for all of them. I didn't check if @tyrone-sudeium's #2302 improved this.

• The jumpy notes issue mentioned by @elsemieni still happens when the freq is not 44100 (the pitch fix didn't have effect on this) and it was happening already on Emscripten with fmmidi (same sequencer) when the host uses 48000 Hz.

By the way, there's a synth.midi-bank-select option to select how bank select commands affect percussion, but not sure if allows to workaround the fluidlite bug.

A note on a possible optimization for some low memory ports (but with decent I/O maybe), synth.dynamic-sample-loading might help.

[Ghabry]

• [x] make it build with fluidsynth 1.1 and test if there are the same drum bugs

[Ghabry]

When I compare it with fluidsynth or fluidsynth-sans-glib (yeah this one works) then the samples generated by FluidLite sound really bland. With fluidsynth(-sans-glib) I feel the intensity and the bass.

So for platforms that have std::thread (unused in our case but is a dependency) sans-glib can be used.

• Fluidsynth: Windows (_needs patch!!! pthread is hardcoded!!!_), Linux, macOS, Android, Switch
• FluidLite: 3DS, Wii, Vita, emscripten

Except if somebody wants to provide a patchset for the patchset that removes std::mutex and std::thread :P

[Ghabry]

BUG: This code

https://github.com/EasyRPG/Player/blob/master/src/audio_sdl_mixer.cpp#L430-L436

Is now handled by EP_MIDI_FREQ

[Ghabry]

When deleting these lines:

• FluidLite sounds less dull (but still worse)
• The audio glitches are gone (problem with our resampler? Though we never had such problems with other audio samples before . Otherwise you would hear audio glitches in almost all BGMs.)

[Ghabry]

Audio recording of both:
midi.zip

[fdelapena]

I feel both sound fine now without the noise or degraded quality :+1:.

Just two things:

• The first has broken percussion yet (recorded without applying the patch?).
• It's fairly hearable the jumpy uncoordinated notes issue here (on both, it's likely a midisequencer issue, it affected fmmidi too).

[Ghabry]

Is the "percussion" the drums? Guess I have to delete the channel to figure this out ;).

This is what I meant with "it sounds dull/worse". The "power" and the bass are missing

[Ghabry]

I found the bug in FluidLite. The Drums sound now beautiful. Basicly no difference to FluidSynth imo :+1:

Wait for my patch ;)