Gdevelop: Newide feature - a quick and tiny sound effects generator

[blurymind]

## Description
It would be nice if we had a quick and easy way to create sound effects directly inside gdevelop's new IDE itslef.

Solution suggested

The good news is that there are a few really nice js libraries that do exactly that
I have actually started playing with implementing this and trying it out via an npm library

The one I picked is jsfx
https://github.com/loov/jsfx

There is another one, but it drags angularjs dependencies which I dont want

Something like this:

I think of copying the sfxr theme, so as to make this more of a pleasant surprise to devs who already know it

But the theme can be anything. It could just as well copy gdevelop's theme

I can get it to create and load the sound effect wav file, however before being able to save the metadata, we wont be able to author the sound effect.

Another problem is that the resources editor does not show audio resources in use yet

I moved this out of https://github.com/4ian/GDevelop/issues/689

My implementation opens this in a reactjs dialog with the UI, once you are ready, if you click OK, it will ask you where you want to save the wav file and then save and apply it

Its still WIP - just sharing here for feedback

[4ian]

Interesting to see where this goes :)

Another problem is that the resources editor does not show audio resources in use yet

It should, even if some/most examples were done before adding this so the resource editor won't show audio resources.
(Audio resources are basically either included from the list in the resources editor or discovered on the fly when exporting/previewing the game).

[blurymind]

@4ian this is my progress so far:

It makes a sound on every change and the sound can already be saved locally. Just need to hook it up for the save operation and add back the play button.

I decided to go the piskel bridge way in the end- refactored some of piskel's code, so it can be used by other editors, such as this one.
This will pave the way for bassoon/other editors. Seems to work great so far!

I need to add the save and close operation, as well as the set path and this will be ready for a pull :)
Might be able to do a pull in the weekend some time.

I am thinking for jsfx, the user could be able to edit sound effects, but for now it will only be possible to save+assign new ones, since I can't store the metadata yet

[4ian]

You're becoming a pro of bridging external editors ;) Super great progress!

I decided to go the piskel bridge way in the end- refactored some of piskel's code, so it can be used by other editors, such as this one.

Yes, that what I wanted to do :) It's a cleaner approach in the long term and could pave the way as you said to other 3rd party editors. Typically at the end we should have a pretty good interface for editors that is clean to read, with similar functions to read files/write them back in resources.
I think we're on something really good there! Really excited by this 🔥

I am thinking for jsfx, the user could be able to edit sound effects, but for now it will only be possible to save+assign new ones, since I can't store the metadata yet

So the metadata is json too? (anyway as soon as it's a string we could store it).
So if I add a way to store a metadata along an audio resource, you should be able to load it back right?

[blurymind]

You're becoming a pro of bridging external editors ;) Super great progress!

Thank you 😊 means a lot from you. I learned a lot from you!

So the metadata is json too? (anyway as soon as it's a string we could store it).
So if I add a way to store a metadata along an audio resource, you should be able to load it back right?

The metadata is json yes - in the case of this and piskel, as well as yarn.
For bassoon I need to look more into it, but I think mod files are a combination of xml data and binary data for the sound samples. In that case I can store in the metadata a relative path to the mod file inside the game's project folder, as well as perhaps some user parameters for loading. I will need to tell gdevelop.js and the editor to include it as a resource in the game, so it can be loaded when called.

[4ian]

I've added setMetadata and getMetadata to gd.Resource. See an example of its usage in the tests of libGD.js: https://github.com/4ian/GDevelop.js/commit/0b4ad8067fd3b2e5f94a9d658cad88acb313198a#diff-d956e28fe78a99599bc19170f51f5704R655
(rest of the commit seems complicated but most files are autogenerated)

Metadata is simply a string that can contain anything, and will be persisted in the project file as it.
For storing JSON, you should use setMetadata(JSON.stringify(yourObject)). For reading back JSON, use JSON.parse but remember to use try {} catch(e) {} to catch any potential error during reading (for example, if someone wrongly modified the file - we still don't want to crash the editor):

let object = null;
const metadata = resource.getMetadata();
if (metadata) {
  try {
    object = JSON.parse(metadata);
  } catch(e) { console.error("Malformed metadata", e); }
}

// Use object here. It will be null if there is no metadata.

To use this, remove newIDE/app/public/libGD.js and restart npm start/yarn start so that latest version of libGD.js is downloaded.
Hope it's enough :) Also might be a good idea to store the metadata from the external editor in an object that is inside an object, with the key being the name of the editor:

{
  jsfx: {
    ...
  }
}

it's not mandatory and purely a convention, but might help latter in case we have lots of external editors that all have their own metadata.

[blurymind]

@4ian thank you. This is a perfect timing too! I can now fully implement this

[blurymind]

@4ian are you sure you updated it? For some reason the metadata methods are still missing,even after I delete newIDE/app/public/libGD.js and re-run npm start

The error states

LocalJsfxBridge.js:46 Uncaught TypeError: audioResource.setMetadata is not a function
    at EventEmitter.<anonymous> (LocalJsfxBridge.js:46)
    at emitThree (events.js:136)
    at EventEmitter.emit (events.js:217)

My code so far:

      const audioResource = new gd.AudioResource();
      const resourceName = path.relative(projectPath, newFilePath); // Still needed for onChangesSaved()
      audioResource.setFile(resourceName);
      audioResource.setName(resourceName);
      audioResource.setMetadata(JSON.stringify(fileMetadata))//setMetadata(JSON.stringify({ hello: 'world' }));
      resourcesManager.addResource(audioResource);
      audioResource.delete();

Do I need to update Gdevelop on my end? I've been working for this on an older version

Here is a teaser:

I made the path selector more generic/reusable. Also added a warning when a file already exists at that path

One thing that I am struggling to figure out is why I can't import scripts with relative paths in my jsfx-main.js ?
I've been trying to do this:
const pathEditor = require('../Utils/pathEditor')

But it fails to find it

[blurymind]

Pull here:
https://github.com/4ian/GDevelop/pull/695

It's feature complete and functional, but needs more work tomorrow to pass flow checks

[blurymind]

This has now been merged to gdevelop and will be available in the next release :)
Closing this