Open-ps2-loader: Sound Effects / Background Music - do we want it?

[KrahJohlito]

Had a go at porting the sound code from FMCB into OPL for sfx and possibly bgm, is this something people are interested in? Keep in mind it will expand the elf quite a bit depending on how many sounds are compiled in (could possibly compress later on somehow?).

OPL SND TEST.zip

This is still early stages it only has sfx mapped to up & down, has bgm compiled in already which is why the elf is 1.2mb but isn't called upon yet, requires code clean ups and such.

All credit goes to Neme & jimmikaelkael.

[Jay-Jay-OPL]

Perhaps if you provide the commit that shows these changes, it can benefit other developers to see the code changes you've made...

[KrahJohlito]

@Jay-Jay-OPL nothing has been committed yet.
We would like feedback from people first. @SP193 @elpatas1 @rickgaiser @TnA-Plastic

What are people's thoughts about this, is it OK to use code from FMCB or is this frowned upon? We don't want to step on any toes here.
Implementing it this way uses sjpcm v2 sound libs which from what I understand is quite old now. Perhaps this is not the best way?

I know showing the code changes would help understand what exactly has been changed but we don't want to commit anything until we know it isn't going to cause any trouble.

All feedback/comments welcome

[sp193]

Personally, I don't have a need for the sounds, but I know it can give users an impression that this is a polished product.

Don't worry too much about space. I mean, our users can afford storing Gigabytes of games, but cannot give even 1MB for their favourite game loader? But for the sake of those SMB users who might store OPL on the memory card, try to keep it below 3MB. You can use a lower-quality or shorter BGM to keep the footprint smaller.

The SPU2 only supports stereo, 48kHz (lower/higher rates will require up/down sampling by software).

If you are concerned about whether taking the code from the FMCB Installer is okay, I am pretty sure it should be fine. There was nothing that said that nobody could use the code from there.

Yes, sjpcm is ancient. You could also write your own, if you do not feel comfortable with copying code or wish to break away from legacy libraries like sjpcm. Audsrv makes sound generation quite easy to accomplish.

Beware of licensing restrictions of the audio files. Don't infringe copyright.
For our art assets, various graphic designers have willingly contributed art assets to OPL, which is how we got free art.

If you use an old sample/project as reference, please use the SDK freesd.irx instead of rom0:LIBSD. If you use the latter, we will lose compatibility with some of the less common PS2 types.

[ElPatas1]

Sorry, but i don't like this at all, i don't think OPL needs this type of things and i don't
woul like hear sounds and bgm, only if could be a feature compiled apart
like was before VMC, GSM, etc, or if is something that do not adds more things
to the code and works externally.

Best regards.

[barronwaffles]

Sounds like a reasonable user-enhancement feature provided it's configurable inside the OPL UI and doesn't have an impact on the functionality of the current codebase - I don't see how it could hurt.

[TnA-Plastic]

I think it is a nice addition, as long as it is made optional!

An in-OPL-setting to turn one or another (click by scrolling, or bgm, or...) on/off would be preferable, instead of a compile-time-function, IMO.

I see no problem regarding the use of FMCB-Source-components (in this). ;)

[gingerbeardman]

Optional, and with the sounds packaged externally?

[TnA-Plastic]

Yepp! It essentially just needs an if-clause to define, if the function to have sound is even available (Edit: depending on, if an audio-file is even present), thus stripping down the size of the ELF with support for sound (in the GUI) to the bare minimum (sound-lib + backend)...

This would also allow different sounds to be used and it could even be a different sound 'per theme', etc. + the ELF would be kept as small as possible (while still adding the feature).

[sp193]

By the way, the SPU2 uses ADPCM for voices. So your sound effects should be stored in the right format. I don't think ADPCM sound effects will take up much space.
Only the BGM could be large.

@ElPatas1 it is a GUI-side thing, so don't worry about it affecting game compatibility. I know that not everybody will like to hear the clicking sound when the D-pad buttons are pressed, so it should probably have an on/off option.

[KrahJohlito]

Thanks for the info & suggestions, have been able to set it as an option in opl settings on/off functions. unfortunately since sjpcm can only be called once per vsync period in the gskit it is incompatible with hires modes.
Going to need to change tack and try work with audsrv like @sp193 suggested so essentially back to square one.

SFX being less than 1 sec long are barely noticeable in size at around 11kb per file for an uncompressed wave (compressed mp3 is probably even smaller) so like you said bgm would be the noticeable increase.

[TnA-Plastic]

I tested it a bit and I have to say... The current implementation is not so good (no pun intended).

First off... I think it is better to call the playback of the click-sound, once a new item is marked and not on every 'button-press' (even simulated multiple presses, when holding the button).

This would eliminate some current bugs (like playing it, even when no item is shown or switched) and possibly is a better way of implementation, to expand upon later.

Another thing I've noticed... Since your implementation (or sjpcm) is dependent on the VSync, it will produce a lag during the time I press or hold the button... You can see the plasma stalling during that time. ;)

Anyway... It is still a good start for this feature and I very much would like it to be implemented, once the code/implementation is... erm... 'better'.

I think, changing the place when to call the playback (i.e. when a new item is marked), would already make quite a difference.

Edit: Btw.: I believe this deserves it's own 'settings-menu'! 'Sound-settings' or similar...

Then we could put various settings like BGM-Volume (0-100, 0=off), SFX-Volume (0-100, 0=off), 'override theme BGM' and various related stuff in there...

[BatRastard]

The only sound OPL should have IMO is during an animated boot screen. I recall an early version of GSM had something like an FMV boot screen of sorts, and I always wanted to see OPL have one. Of course, any music or sound effects during this boot screen can't be copyrighted material. It has to be given to us freely to use or we compose something ourselves.

As for actions within the GUI, sound isn't needed. Codebreaker didn't have any sound. Didn't need it.

[ElPatas1]

@sp193 , well if you say that can't break any game is good.

But i'm also very worried about the problems that can occur about speed, stability and size, because adding sound always leads to these problems and the resources that are available are limited.

I insist the best thing would be that it was put as a separate option compiled as the
VMC was before, if after a test time of several months we see that no
gives us no problem, then we can make the step of integrating it with
a sound option on/off in the GUI.

Please check the PS2RD, after the integration do not works fine.

Best regards.

[TnA-Plastic]

I don't see, how a compile-time-option would be any more preferable for this kind of feature, than an in-GUI-option...

It made sense for games and related options (VMC, GSM, PS2RD, etc.) and the special new Render-Mode (Hires), but I don't understand it for the GUI (Sound)... An in-GUI-option would work out the same, like a compile-time-function and it is rather beneficial to have as few of them (more different compile-functions = more OPL-Build-combinations and less are less in both cases) as possible, IMO.

If you insist on a compile-function,... Oh well... I do not have a problem with it, but I think it is neither necessary, nor does it impose many problems... However, I understand, that it could be a better choice, until this feature is implemented properly...

But MAYBE his new implementation already works so well, that it isn't needed, once he makes a PR. :)

@KrahJohlito: Btw.: Which part was dependent on the vsync? Was it the lib itself and why?
Does audsrv suffer from this limitation as well?

On another note: As for an idea which MIGHT affect performance... I am not sure, how channel-handling is done via these libs. I do not mean the channels like 'stereo' etc. but the 'channels' for different parallel audio-streams and stuff, like BGM and the click-sound... If the different sounds are mapped to the same channel, there might be some interleaving-'effect', so that the EE tries to push the needed part of the sound, once it is played... The interleaving should be done on the SPU2, rather than the EE, which should be possible if they are mapped to different channels. If that has to be done manually, instead of the lib handling it automatically, then I think this would be a quite easy change.

[carl0sjt]

I would love to listen to "Snake Eater" instrumental while scrolling through my game list <3

[KrahJohlito]

I haven't had a whole lot of time to work on this due to RL. It's currently working from a simple on/off switch in the audio settings menu and sound files are not compiled in they are called externally. However there are issues with implementation.

I changed the call location as @TnA-Plastic suggested and it works well but the plasma stall noticed previously was not a vsync issue (I didn't notice it as I was using a static BG image when testing).
It's due to multithreading not being implemented which is also why BGM is not BGM it's just playing a music track with no other processing happening while the sound function is called.
Multi threading is tough and I'm not sure I'll be able to implement it. For one a cross platform lib would need to first be compiled into OPL as afaik there isn't one in the SDK?

The vsync dependency was from the sjpcm lib itself so no audsrv does not have this issue and works fine with HiRes.

[TnA-Plastic]

I suppose you have to move your code from a thread/task where it is running 'in', to a separate thread. ;)

Edit: For what would another port ('cross platform lib') be needed? I don't get it... I think the issue is rather that the sound-function is not globally implemented, but as a function within a function or similar.

audsrv works well? Great! :)

Would you mind adding/attaching a new build?

Edit.2: Btw.: I mentioned something related in a related and still open issue #88! ;)

[TnA-Plastic]

Sorry, for double-posting. I edited my previous post various times and added a reference.

@KrahJohlito: Would you mind creating another branch?
I suppose you have the code of the sound-support itself 'within' the calling-function, rather than a global function which is called from the marking-function?!

On another note... BGM probably could be played 'while' something which is continously done is executed. Not sure tho'... I can rather make assumptions than good suggestions, because I can just assume the structure... (no source available). ;)

[KrahJohlito]

http://www.psx-place.com/threads/opl-snd-test.20370/#post-135568

discussion / suggestions / help should all happen here from now on to avoid annoying people.
Thanks

[TnA-Plastic]

Well, regarding my previous comment 'no source available'...

Now it is, on your Account (as a repo) and we are up to test3+ (almost test 4?)! ;)

So if anyone is interested, please take a look at the linked thread and on(to) @KrahJohlito 's account/repo and if you can, just try to contribute in some kind of way (i.e. via testing, code, etc.)!

[TnA-Plastic]

'Test 5'/The current commit, is the first I regard to be quite stable.

For those following this topic, please do some tests with the newest build!

[TnA-Plastic]

Oooooh, all those new changes to AUDSRV, THX to @SP193 and OPL THX to @KrahJohlito and @SP193...

BGM is on its way! That's great!

'Test 7' is going to be great! :)

[TnA-Plastic]

Well, it is not yet merged! Not sure if it was wise to close it beforehand. ;)