Ml5-library: Use a different crepe model for pitch detection

[Seanitzel]

Dear ml5 community,

I'm submitting a new issue. Please see the details below.

The pitch detection implementation currently uses the 'tiny' version of the crepe model.
I attempted to use a different version, but im not sure how the current version was created as i can't load my model successfully.

I tried the following:

1. Download the small crepe model version from the crepe repository.
2. Use tfjs to convert the keras model to a model that can be loaded by ml5.
3. Load the model to an example of the pitch detection ml5 and run it.

I noticed that the model.json file i generated has a different structure than the one currently used by ml5:

my model.json:

{
    "format": "layers-model",
    "generatedBy": null,
    "convertedBy": "TensorFlow.js Converter v1.7.4r1",
    "modelTopology": null,
    "weightsManifest": [...]
}

ml5 model.json:

{
    "modelTopology": {...},
    "weightsManifest": [...]
}

How was the model that is currently used by ml5 created?

Here's some helpful information about my setup...

• Web browser & version: Chrome latest
• Operating System: macOS
• ml5 version you're using: Latest

[joeyklee]

Hi @Seanitzel - Thanks for this question. Exciting to hear you've been working on another Crepe model.

This is a question that might be best answered either by @shiffman or @handav since I think they had worked on this back when it was first added. Let's wait to hear back from them.

[Seanitzel]

Great, i would also be more than happy to make a PR to add the other models to the library :)

[handav]

hi! I got the model from this part of the MARL CREPE repo: https://github.com/marl/crepe/tree/models - let me know if that helps?

[Seanitzel]

Not really, this is where i got the model from as well.

I later tried to convert it using tfjs, but got different results than the one used here...

[siva5kri]

Hi! I am also facing the same issue while using other versions of crepe models ( 'medium' version taken from https://github.com/marl/crepe/tree/models). can you please help me to resolve this.

I would also like to to know about time-step used in this pitchdetection method and whether the confidence threshold can be changed. At present the confidence threshold is at '0.5' i would like to decrease that to '0.25'.

[Seanitzel]

Any ideas or updates on this? 🙏🏻