Vexflow: Use with React / React-native?

[davidfloegel]

Hi there,

Is it possible to use vexflow with react (react-native) at all?
I've used this library before for a phonegap based app but switching to react and would love to keep using this one!

Cheers
david

[mturley]

+1! Would be very nice. Was about to look into this myself.

On Tue, May 3, 2016, 3:21 AM David Floegel [email protected] wrote:

Hi there,

Is it possible to use vexflow with react (react-native) at all?
I've used this library before for a phonegap based app but switching to
react and would love to keep using this one!

Cheers
david

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/0xfe/vexflow/issues/338

[marcoalkema]

Doesn't React also use a canvas? I've been using VexFlow with Pux, which is an interface to React for the language Purescript and I'm drawing my notation on a Canvas through that. Unless I'm not fully comprehending the workings behind Pux, I assume it should be possible.

[davidfloegel]

@marcoalkema Using the canvas is not the issue - it's more including the library and rendering things.. Do you have an example of how you're doing it?

[panarch]

If you use react-dom, there's no problem at all to use vexflow.
But if you consider react-native, it's totally different. Currently it is not possible because of a single issue which is about calculating text size.(only this is the thing, if vexflow removes this dom dependency, you can generate svg file and it means you can use vexflow with react-native directly)

If there's no other volunteers, I will may handle in the future. I'm currently considering to use this (https://github.com/nodebox/opentype.js & https://github.com/icons8/svg-path-bounding-box)

[mlennon3]

I am trying to use vexflow with react-native. It's a struggle, as I'm having to do it within a WebView component. I can get a 'hello-world' type example to render, but I don't know how I can do things more dynamically. Very interested in this discussion.

[mturley]

Can we figure out what we'd need to change in Vexflow to get it working
without the DOM? That sounds like a great option.

On Tue, May 3, 2016, 3:39 PM mlennon3 [email protected] wrote:

I am trying to use vexflow with react-native. It's a struggle, as I'm
having to do it within a WebView component. I can get a 'hello-world' type
example to render, but I don't know how I can do things more dynamically.
Very interested in this discussion.

—
You are receiving this because you commented.

Reply to this email directly or view it on GitHub
https://github.com/0xfe/vexflow/issues/338#issuecomment-216641940

[davidfloegel]

Oh wow, quite a few responses!

It would be great if we could look into it. I think it would be a great feature!

[panarch]

It looks many of you want this kind of standalone feature, so I changed my mind to do earlier.

I spent today and... I'm happy to share this, now vexflow can successfully generate svg files without DOM!
I used opentype.js to convert all text to svg path, and replaced getBBox to svg-path-bounding-box.
And then, for rendering svg strings I used react.
(I also considered jsdom rather than react but in my previous exp, it was too heavy and slow to use because it's purpose is covering all the dom functions, not performance.)

I added new context named SVGStringContext which extends SVGContext.

• Do you have any better name suggestion for the new context? It would be nice to use better naming but I don't have idea.

I'll clean up the codes and push to my fork and share with you tomorrow with details.

[0xfe]

That's great Taehoon. Thanks!

How about naming it TextSVGContext?

On Wed, May 4, 2016 at 2:24 PM, Taehoon Moon [email protected]
wrote:

It looks many of you want this kind of standalone feature, so I changed my
mind to do earlier.

I spent today and... I'm happy to share this, now vexflow can successfully
generate svg files without DOM!
I used opentype.js https://github.com/nodebox/opentype.js to convert
all text to svg path, and replaced getBBox to svg-path-bounding-box
https://github.com/icons8/svg-path-bounding-box.
And then, for rendering svg strings I used react.
(I also considered jsdom https://github.com/tmpvar/jsdom rather than
react but in my previous exp, it was too heavy and slow to use because it's
purpose is covering all the dom functions, not performance.)

I added new context named SVGStringContext which extends SVGContext.

• Do you have any better name suggestion for the new context? It would
  be nice to use better naming but I don't have idea.

I'll clean up the codes and push to my fork and share with you tomorrow
with details.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/0xfe/vexflow/issues/338#issuecomment-216956198

Mohit Muthanna [mohit (at) muthanna (uhuh) com]

[davidfloegel]

Wicked!! That was fast. Any chance you can provide a basic codesample as well??

[panarch]

I push to my fork with "standalone" branch.
https://github.com/panarch/vexflow/commits/standalone
https://github.com/panarch/vexflow/commit/c6727efe1c53f483240bf843ec82544c450974d1
https://github.com/panarch/vexflow/commit/89c4341d1e67bc2de77f5b42aacc5cb0223fcd83

grunt concat && npm run generate:current
will work.

But one more thing to fix.
I'm currently using NotoSans & NotoSerif fonts to render texts but they don't have some symbols for music notations below,

Vex.Flow.unicode = {
  // Unicode accidentals
  "sharp": String.fromCharCode(parseInt('266F', 16)),
  "flat" : String.fromCharCode(parseInt('266D', 16)),
  "natural": String.fromCharCode(parseInt('266E', 16)),
  // Major Chord
  "triangle": String.fromCharCode(parseInt('25B3', 16)),
  // half-diminished
  "o-with-slash": String.fromCharCode(parseInt('00F8', 16)),
   // Diminished
  "degrees": String.fromCharCode(parseInt('00B0', 16)),
  "circle": String.fromCharCode(parseInt('25CB', 16))
};

Because of that, I found that a single test case broken
"TextNote.TextNote_Superscript_and_Subscript"

The first one is blessed image and second thing is rendered by current fork.

For rendering musical chord symbols properly, we need special font for chord which should have sharp, flat, other symbols and latin characters.

Anyone knows any of license-free font for rendering music chord notations?

[panarch]

@0xfe I like TextSVGContext, thanks! I renamed :)

[mturley]

@Taehoon, that first image is considered correct? Even though that first
superscript has the +5 occluding with the other characters? Looks broken
to me

On Thu, May 5, 2016, 10:04 AM Taehoon Moon [email protected] wrote:

@0xfe https://github.com/0xfe I like TextSVGContext, thanks! I renamed
:)

—
You are receiving this because you commented.

Reply to this email directly or view it on GitHub
https://github.com/0xfe/vexflow/issues/338#issuecomment-217163170

[0xfe]

@mturley You're right, that first image is not correct, but it is blessed. It's probably an uncaught regression that originated before we had the visual regression tests. Given that the second image has the "+5" properly positioned, it seems like the bug related to measureText (as usual.)

[davidfloegel]

ah this is awesome! :)
I still don't get how to include and render something in react though.. Can someone give me a hint?

[mlennon3]

Yes this seems great, thanks for such fast response! A bit of example will really help to get started.

[infojunkie]

@panarch re open licensed font, how about Bravura? It implements SMuFL which includes chord symbols.

[panarch]

@infojunkie Yes it includes chord symbols but the issue is it does not have latin characters.
May be these are good examples,
http://www.myriad-online.com/resources/docs/melody/english/fonts.htm
https://www.myfonts.com/fonts/tijs-krammer/chord-symbols/

@davidfloegel Yeah current state may not be kind enough to refer if you are not familiar with vexflow test scripts because I just applied it to inside current test scripts. I will add code examples after fixing current chord notation font issue. After that you can easily figure out how to use.

[infojunkie]

@panarch re: fonts, it's my understanding that engraving systems typically use two fonts: one for music symbols and another for text, so it would make sense for VexFlow to follow suit. The advantage is that the engraver has the flexibility to choose any regular text font, without being limited to those that also contain music symbols. That's the idea of Bravura, Norfolk, etc.

[0xfe]

I have no problem with using multiple fonts, so long as they're public domain and license unencumbered. Feel free to add new glyphs into src/fonts/ if you need to. (I would love to have full support for Bravura in VexFlow.)

[infojunkie]

@0xfe yes, supporting multiple fonts and maybe even allowing app developers to specify their own - that don't need to be shipped with VF.

[infojunkie]

Regarding the creation of a React component for VexFlow, I think @mlennon3 and others are wondering, as I do, about how to design such a component to be _reusable_: the VF tests are of course trivial to wrap in a React component (the DOM/SVG issue above notwithstanding), but how would a reusable component look like? In other words, what kind of API would a generic VF React component expose to allow the app developer to dynamically specify the music to display?

To give examples of dynamic VexFlow generation:

• VexTab parses a text-based notation format to generate a VexFlow engraving
• VexFlow MusicXML parses a MusicXML file to do the same
• Varese has a React component to display chords using VF

A reusable React component would ideally expose all core VF functionality (e.g., via props) and conform to the React rendering workflow.

[panarch]

@infojunkie @0xfe I got it. My description was not enough for this. The issue is... for running vexflow without web browser, we cannot get web browser's pleasant font system support. So the alternative is opentype.js, it's really nice but currently it does not support using multiple fonts feature. So developers should manually handle fonts.
That's why I wanted to find the font which includes both latin characters and chord symbols.
It looks not too hard to contribute opentype.js to support multiple fonts but... not for now.

So I think this new TextSVGContext may not fit for many of light users because it requires users to do additional configurations. Users should manage fonts themselves, not convenient as CanvasContext or SVGContext which can use directly without any settings. TextSVGContext would be good choice for developers who wants to use vexflow in node.js or something like react-native, or better rendering performance (Because no more synchronous getBBox calling is required)

[panarch]

Possible hotfix is that I can bring chord symbols like sharp, flat, etc from BravuraText and merge into current existing fonts(NotoSans, NotoSerif). So, if there looks hard to get proper chord notation font for now, I will modify existing fonts to fit into currently broken test case; "TextNote.TextNote_Superscript_and_Subscript"

[SalahAdDin]

:+1:

[panarch]

@infojunkie And about React component, I'm not sure that I understand correctly. Although new TextSVGContext uses React library but it is not for providing users to make React component easily or somewhat related to React. It just need to use some kind of virtual-dom system to generate svg string and I thought ReactDOM was the best so I just used.
It's totally independent issue between TextSVGContext and reusable React component.

In case of making ideal React component which have all core VF functionality, I think it's hard for now.
For making that..., what we should do before is that I think it is making Builder API.
After we build a VexFlow builder api which is clearly divided between formatting and rendering.
Then we can try making proper VF React component which take a rendering part of the builder.

[infojunkie]

@panarch re: React component, yes I agree VF itself lacks a clear programming model. Your idea of Builder API sounds good. I will be looking into the examples I mentioned above to emulate or expand on their approaches. Re: fonts, I guess it's an ongoing area of discussion for VF...

Thanks for TextSVGContext, I'm looking forward to using it in my upcoming app project!

[dannyid]

@davidfloegel I've actually just yesterday starting playing around with getting VexFlow to dynamically update using React. My approach so far is to just destroy all the VexFlow created SVGs in willComponentReceiveProps() and redraw the entire thing based on Redux state. Have a look and let me know if you have any ideas.

https://github.com/dannyid/music-tutor/blob/master/js/components/Tutor.js

[davidfloegel]

Any progress on the use with react native? :)

[remon-nashid]

As per this post in Vexflow google group, @panarch has thankfully introduced an SVG rendering context (TextSVGContext) that works with react-native.

The context could be found in Taehoon's fork of Vexflow repo https://github.com/panarch/vexflow/commits/standalone

I tried to run the included tools/demo.js on android emulator, after making few unrelated changes. Unfortunately, I hit an error at this line:

import ReactDOMServer from 'react-dom/server';

Error message:
EventPluginRegistry: Cannot inject event plugin ordering more than once. You are likely trying to load more than one copy of React.
The error is not entirely clear to me, but I wonder how useful ReactDOM(Server) is in react-native. Same issue was reported here, and is still unresolved.

Could someone give that demo a try? Hope it works.

[SalahAdDin]

With the last ES2015 implementations i think is most easier use vexflow in react/react-native projects.

[davidfloegel]

No that doesn't work and won't work like it's implemented in the demo.

1. The demo uses react-dom - react-native has no DOM
2. The loading of the fonts doesn't work for me. If I remove the font loading then
3. Vex.Flow.Render.getTextSVGContent is not a function

:(

[SalahAdDin]

Oh yes, this is a big problem.

[panarch]

2,3 : I used node.js file api so, you should replace file api which react-native provides.

1 : hmm... it looks hard to solve this by adding react-dom lib to react-native project.
So, if referencing react of react-native from react-dom is not possible, how about bundling react + react-dom/server independently without using react-native?
If all looks hard to try, replacing TextSVGContext's react to another lib can be also option, it will be simple way to avoid conflict.

[davidfloegel]

@panarch hmmm I could try point 2. and 3 the way you mentioned...

No idea about the first part though... Anyone?

[remon-nashid]

I could take a stab at point 1. To make sure I fully understand the problem: are we proposing substituting ReactDOMServer with a library that could actually draw SVG in react native, for example react-native-svg?

[panarch]

@remon-georgy Yes, exactly.
react-native-svg you mention looks good to try.

[infojunkie]

react-native-svg looks great, but it lacks one important function that VexFlow renderers need: the ability to compute bounding boxes: https://github.com/react-native-community/react-native-svg/issues/89

[panarch]

@infojunkie It's ok, computing bounding box is the key that TextSVGContext provides,

opentype.js to convert all text to svg path, and replaced getBBox to svg-path-bounding-box.

[davidfloegel]

I've tried out a couple of things but I can't get it to work..
Anyone else here had any progress?

[SalahAdDin]

@davidfloegel I ask the same.

[jasonpang]

I found this demo: https://github.com/kirkness/react-vexflow-example. The direct source isn't on his GitHub repository, but if you download and run the project (don't forget to move it to the C:\ drive because the paths are hardcoded), you can use the provided source maps and view the original source code in Chrome.

He's got a pretty interesting setup e.g.:

render() {
    return (
        <div className='App'>

            <button onClick={this.shuffle.bind(this)}>

                Shuffle.
            </button>

            <VexFlow
                height={800}
                width={800}
                font={[ 'Arial', 10, '' ]}
                stave={this.stave}
                voice={this.voice}
            />
        </div>
    );
}

export default class VexFlow extends Component {

    constructor(props) {
        super(props);

        this._renderer = null;
        this._context = null;
    }

    componentDidMount() { this.handleProps() }
    componentDidUpdate() { this.handleProps() }

    handleProps() {

        this.clear();

        const {
            width = 500,
            height = 500,
            font = [ 'Arial', 10, '' ],
            fontColor = '#eed',
            stave,
            notes,
            voice,
        } = this.props;

        this._renderer = new VF.Renderer(
            this.refs.wrapper,
            VF.Renderer.Backends.SVG
        );

        this._renderer.resize(width, height);
        this._context = this._renderer.getContext();

        this._context
            .setFont(...font)
            .setBackgroundFillStyle(fontColor);

        stave.setContext(this._context).draw();

        voice.draw(this._context, stave);
    }

    clear() {
        this.refs.wrapper.innerHTML = '';
    }

    render() {

        return (
            <div ref={'wrapper'}></div>
        );
    }
}

[SalahAdDin]

@jasonpang I can not understand these example.

[matt-gardner]

I got a very simple example actually rendering using react-native. You can try it for yourself here: https://github.com/matt-gardner/react-native-vexflow-example. Just clone the repo, update submodules, and run react-native run-ios (haven't tried it on android, and I didn't update index.android.js - if you want to try on android, just copy the stuff I did from index.ios.js to index.android.js). It should work, assuming you've already set up react-native and have successfully run the demo app. I just very slightly modified the demo app, and the vexflow code, to get this working. Thanks to everyone here who posted stuff - I pieced together a lot of things from this thread to get this actually working.

There's one pretty big issue still: it appears either I don't know anything about how to render things properly with vexflow (which is pretty close to true), or there are some issues with placing things in the SVG. So while I got it to _render_, I could use some help with getting it to render _correctly_. I'm hoping that others here who are more familiar with vexflow and JS development can help out with that. Also, the modifications I made to vexflow are certainly not good javascript - I've mostly written scala and python for machine learning, not javascript or mobile apps. If someone wants to help package up the changes I made (along with @panarch's earlier changes) into a nice PR to vexflow so that we can support this in the main release, that'd be nice too.

I'm attaching a screenshot of the app running in the ios simulator, for proof that I got this running (and again, it's not pretty, but I finally got it to render, which I was pretty happy about).

[matt-gardner]

I spent some time today stepping through a bunch of the rendering code, trying to figure out what's going on. I made a tiny bit of progress (see screenshot below), finding a bit of user error in how I was calling the formatter, but there's still a long way to go. I have a couple of guesses about what could be going on here:

1. My first guess was that drawing Paths in the SVG is not working correctly. Except the rest is getting drawn, so maybe that's not it...

2. So my current best guess is that the offset computation isn't happening correctly.

I spent most of my time trying to trace all of the draw() calls to learn how things got rendered and see if I could find anything there, so I haven't yet looked at offset computation in detail. I'll try that next time I get time to look at this. It'd probably be a whole lot easier with some pointers from someone who's more familiar with the codebase, though...

[SalahAdDin]

Why use other vexflow version? Is not the official version.

[matt-gardner]

Because upstream does not have an SVG context that can be used outside of a browser, as it has calls to document. That was what all of the work earlier in this thread was about - getting vexflow to the point where it doesn't need a browser. I actually got it running in a react-native app, though I'm still having some rendering issues.

Once it's working properly, I intend to submit a pull request to upstream vexflow with the changes necessary to get it to work.

[matt-gardner]

I spent some time tonight going through all of the calculations done to get x and y positions and so on of all of the various elements. All of those computations look correct - the x and y positions of the Rect for the ledger lines match the x and y positions for the corresponding note heads, and they are next to each other in the G component of the SVG. Yet somehow they are rendered very far apart from each other. This is baffling to me, and it makes me wonder if there is something up with react-native-svg. I also have no idea where the clef and the time signature went off to. Their starting x values are off the screen to the right, but the positions specified in the Path are correct, so I'm not sure that's the problem.

Also, it looks like there's a different scaling for x than there is for y in the SVG, and that might be related somehow. The y position for the first ledger line is 90, while the x value is around 60, yet it is farther over than it is down. I'm not sure what's going on there, either.

I think my next step is to become more familiar with SVG rendering, and how react-native-svg works, as it seems like the problem is somewhere in the connection between vexflow and react-native-svg.

[davidfloegel]

@matt-gardner I used to play around with react-native-svg to build my own very small library for rendering scales + chords.

It seemed to me (like you say) react-native-svg is a bit dodgy in terms of rendering. I did all the calculations for distance + position etc on paper but if I put them in to the react-native app it didn't seem to work out.

If you want to work on it together feel free to ping me a message! Would be great to get this ready soon! :)

[matt-gardner]

Success! At least a bit. There are probably still some issues, but I found the main one. My current screenshot:

The problem is that the SVG in react-native-svg uses the x and y attributes in the element as an offset when reading numbers in a Path. So if x is 90, and Path.d has move commands that have numbers around 90, the Path object will actually get rendered near x=180. So I just set the x and y values to 0 whenever creating a Path element, and that magically fixed everything (well, all of the issues I know about; I haven't tried to do anything complicated with this yet). I updated the repo I pointed to above, so you can checkout the code and try this yourself. It appears to work.

[gristow]

Sounds like maybe react-native-svg isn't processing absolute vs relative move commands in paths correctly. (Absolute is capital M, relative is lower case m.)

[panarch]

Hi everyone, I have a good news.

Now VexFlow runs with React Native including font rendering.
You can use this, https://github.com/panarch/standalone-vexflow-context
and seed project url is here, https://github.com/panarch/react-native-vexflow-seed
I think it will work fine for any cases, but I didn't integrate VexFlow tests with this new context yet so it may have some bugs.

I wanted to participate since @matt-gardner reactivated but huh, I was having quite tough schedule on my 6 weeks trip so... now it's the last week of my long vacation and I made some work.
Thanks @matt-gardner for participation, I was able to avoid the absolute path of react-native-svg bug easily because of you.

At first, I thought making VexFlow run with React Native might be similar with Node.js, but it was very different issue.

What I did first was replacing my previous TextSVGContext render function to use react-native-svg rather than react server-side rendering function.

Then, other big issue was font.
Because we cannot use powerful font feature of web browsers in Node.js & React Native, I used some tricks for making VexFlow run on Node.js.(far above on this thread I mentioned)

It was easy to use file system in Node.js, but in React Native... it looked little bit messy to use file system directly for loading font files.
So, for making the issue simple, I just converted raw font files to base64 to solve.
(Similar work what https://github.com/mudcube/MIDI.js/ does for soundfont)

standalone-vexflow-context currently only supports react native for now.
I'll add Node.js context later.

[davidfloegel]

That is amazing!! Thank you so much :)

[SalahAdDin]

@panarch, thank you very much, why will you add Node.js context?

[panarch]

@SalahAdDin I just want to run VexFlow in my server without browser emulation. With Node.js context, we can render VexFlow without browser so it can make sense in may purposes.

[msand]

@panarch @davidfloegel @0xfe I've done a lot of work to improve the svg spec conformance and rendering accuracy of react-native-svg on android. You might want to try testing my fork, until it gets ported to ios and merged into master.

https://github.com/react-native-community/react-native-svg/pull/406

https://github.com/msand/react-native-svg/tree/5.3.0-conformance

[msand]

I'm currently implementing and improving
https://drafts.csswg.org/css-inline/#propdef-alignment-baseline and
https://svgwg.org/svg2-draft/text.html#AlignmentBaselineProperty

Therefore, I need access to the os2 and hhea baseline tables from all the glyphs in the used fonts.
https://www.microsoft.com/typography/otspec/base.htm
https://www.microsoft.com/typography/otspec/os2.htm
https://www.microsoft.com/typography/otspec/hhea.htm
https://developer.apple.com/fonts/TrueType-Reference-Manual/RM06/Chap6bsln.html

Which is probably highly relevant to your use cases as well. Related material:
http://bl.ocks.org/eweitnauer/7325338
http://apike.ca/prog_svg_text_style.html
http://christopheraue.net/2014/03/05/vertical-align/
http://vanseodesign.com/web-design/svg-text-baseline-alignment/
https://iamvdo.me/en/blog/css-font-metrics-line-height-and-vertical-align
https://stackoverflow.com/questions/19856094/computing-font-truetype-hhea-values-ascender-descender

I'm now in the choice of opentype.js and something like:
https://software.intel.com/en-us/articles/building-freetype-libraries-for-x86-devices-using-the-android-ndk

opentype.js would allow keeping all the logic in js, instead of both java + obj-c, but might have a performance impact. While building on and bundling freetype probably adds more to the app size and requires the NDK. Then again, the data only needs to be parsed once per configured font. Any suggestions/input?

[msand]

@matt-gardner you should probably get a mention about the fork/pull request as well

[msand]

I've forked the seed project, updated react, react-native, and, react-native-svg (using both master and my fork), ported the code to android, and updated the android build setup. If anyone is interested you can check it out here: https://github.com/msand/react-native-vexflow-seed

[msand]

[0xfe]

Wow, looks great Mike! Thanks for doing this. Would you mind adding some instructions to the Wiki?

[njwest]

Happy to find this thread! You guys have done some awesome work

I get the following error after I git clone, npm i, and react-native run-ios@msand's fork of https://github.com/msand/react-native-vexflow-seed

[msand]

@njwest Haven't touched this for quite some time, I guess there's newer/more changes in react-native-svg that I need to adapt the code to, or then I had monkey-patched it to work around that problem. Might look at it in the weekend.

[njwest]

@msand Many thanks for the response -- could you post the package versions your fork works with?

[ulisinn]

"dependencies": { "lodash": "^4.17.4", "midiutils": "0.0.4", "mobx": "^3.3.2", "mobx-react": "^4.3.5", "react": "16.0.0", "react-native": "0.50.4", "react-native-svg": "^6.0.1-rc.1", "standalone-vexflow-context": "^0.1.1", "tonal": "^1.1.0", "vexflow": "^1.2.84" }

[ulisinn]

https://github.com/panarch/standalone-vexflow-context

[SalahAdDin]

@ulisinn, hi man, it's your project a open source project?

[njwest]

Sweet project! I am also a trumpet player :P

[sschmidTU]

You can simply create a Vexflow component for React.
It works for OSMD, which has the same setup as Vexflow (handing it a div container).
Here's example code, you can pretty much just exchange the OpenSheetMusicDisplay.jsx in the lib folder:
https://github.com/opensheetmusicdisplay/react-opensheetmusicdisplay/tree/master/src

Maybe this will still help someone.