Maptool: Add HTML5 support through WebView

[Merudo]

Is your feature request related to a problem? Please describe.
Java Swing only supports HTML 3.2, which is antiquated to say the least. Being able to display HTML 5 would result in a much better looking MapTool.

Describe the solution you'd like
HTML 5 support through the JavaFX WebView class. The webview could be displayed in a manner similar to dialog() and frame, with something like

[frame5(frameName, properties): {
  HTML
}]

EDIT:

Implementation:

Currently the frame/dialog windows flow this way:

HTMLFrameFactory -> HTMLDialog/HTMLFrame (extends JDialog/DockableFrame) -> HTMLPanel (extends JPanel) -> HTMLPane (extends JEditorPane) -> HTMLPaneEditorKit (extends HTMLEditorKit)

with JEditorPane doing most of the heavy lifting related to the HTML content.

To accommodate HTML5, HTMLFrameFactory and HTMLDialog can be modified slightly to hold both html3.2 and html5 dialog/frame. New components are requires for the rest of the chain, which should look like

HTMLFrameFactory -> HTMLDialog/HTMLFrame (extends JDialog/DockableFrame) -> HTMLJFXPanel (extends JFXPanel) -> WebView

The JFXPanel is used to hold JavaFX components, and WebView deals with the HTML5 content.

Other factors to consider:

"link" tags (for CSS & onChangeSelection) : can be dealt after the html is loaded but before it is read, via the org.w3c.dom.Document class. Some will require callbacks.

"a" tags (for macroLinks): can be dealt by adding an addEventListener to the elements of org.w3c.dom.Document.

"input" tags: most of the input types are the same as in 3.2, although there might be some new properties to account for. Implementing the existing ones should be easy by using an eventListener on the Submit button and then creating the json from the input fields.

EDIT2: List of features implemented:

• src: external sources blocked
• src: external javascripts blocked before loading
• href: external sources blocked
• href: macroLinks executed
• href: external hyperlinks launched in browser
• title tag: window title changed
• meta tag: input, closebutton, width, height, temporary, working
• link tag: css sheets loaded from macros
• link tag: onChangeSelection and other events launched
• javascript: alert and confirm boxes displayed when called
• form: macro executed and data passed on form submission

[cwisniew]

I dont think this should be called webView as people will be expecting it to display web pages.

Also since form submittal works completely different in webview to the swing HTML frame I think this needs to be well thought out and documented before starting so its forwards compatible with MT Script 2. (or at least completely distinct so that it wont confuse matters)

[Azhrei]

I think this needs to be well thought out and documented before starting

Definitely!

Threading issues may occur as well.

The HTML viewer in JFX supports animation (of GIFs) which I assume use a separate thread for each animated image (just an assumption on my part; a quick Google search was inconclusive). If true, the MapTool stack size for threads could be prohibitive (although javafx.concurrent may allow setting the stack size on a per Executor basis, but I have no experience with that package).

I'm also not sure that the URLStreamHandler for asset:// is thread-safe, so there's a potential concurrency problem for assets used in JFX. (But the asset:// URI can only be used to read the asset, so I wouldn't expect any problems unless an asset is referenced that is only partially loaded and I _believe_ the existing ImageManager and AssetManager classes already handle that, so this is probably not an issue.)

[Merudo]

I dont think this should be called webView as people will be expecting it to display web pages.

True. Although webview (the class) can also be used to display actual web pages, so maybe that's something we could add as well.

Also since form submittal works completely different in webview to the swing HTML frame I think this needs to be well thought out and documented

I suggest we implement limited HTML 5 first, without support for form submission. Once a working model is implemented it will be easier to discuss how exactly implement form submission, which will indeed require careful thinking.

before starting so its forwards compatible with MT Script 2. (or at least completely distinct so that it wont confuse matters)

Where can I find the documentation for MT Script 2? I've looked on github and no one is assigned to work on the parser, and I couldn't find a discussion about it on the Developer Notes forum either.

[cwisniew]

I suggest we implement limited HTML 5 first, without support for form submission. Once a working model is implemented it will be easier to discuss how exactly implement form submission, which will indeed require careful thinking.

I would want to see the plan first before any work started as currently more and more is being hacked on without thought being put into how it works with other code.

[Merudo]

I would want to see the plan first before any work started as currently more and more is being hacked on without thought being put into how it works with other code.

I'm not sure what features or bug fixes you are referring to. I know you'd have preferred things done differently with json.path but I disagree that a lack of thought went into it. Many people gave their suggestions for the implementation, including @Azhrei, @JamzTheMan, @aliasmask, and @Phergus.

If you are unsatisfied with recent changes, I suggest you open new issues on github so they can be discussed by the community.

[cwisniew]

No I am suggesting a plan for this one that can be discussed before the changes are started. There are definitely a few people interested with ideas of how to do this so someone needs to document their thoughts first (any one) so it can be hashed out.

[Merudo]

No I am suggesting a plan for this one that can be discussed before the changes are started.

I agree that a plan is great and needed, and that the feature should be discussed thoroughly before any PR is merged.

I'm just not sure why concurrently working on a prototype is undesirable to you? Having a basic working prototype can be illuminating, and can help us make a better plan.

[cwisniew]

Because since many people have ideas of what they want then if you implement what you want and submit a PR and its not what others want then it gets rejected, and we are back to needing a plan to discuss how it is to be achieved. Not everyone will review the PR and asking them to do so is not practical.

I honestly dont think its too onerous for us to ask for a plan of how it is to be achieved/implemented for a major feature before hand so that it can be discussed.

I know you want to get your feature in, but on the other end there are other people who would like to get features in too and having a plan rather than having to go back and forth over a PR will take up less of their time as well

[Merudo]

I understand your concern regarding PRs. I don't intend to open a PR for the prototype, so there will not be any back and forth over PRs. At most I'll share a link to my fork.

Honestly the prototype is mostly for my own understanding, I don't feel I'd be able to make a good plan until I worked with the code.

[Phergus]

My two cents:

• definitely can't be called webView. So far everything I've come up is pretty terrible but some ideas: dialog5/frame5, dialogPlus, dialogH5, dialogJFX
• from the start it needs to be able to handle forms. Users will expect to use it for character sheets immediately so that needs to be dealt with.
• can WebView be stuck inside of dockable frame? Users will likely expect that.

[Merudo]

I like dialog5 / frame5.

WebView can be put into a jfxPanel which I believe can be put inside a dockable frame, but I'm not sure.

Also, eventually we might want to support FXML. That would be neat.

[Merudo]

I've added an outline of the WebView implementation in the first post.

[Azhrei]

I agree with Craig about outlining a plan, but I see no harm in doing as you say and coding up a proof of concept to see if/how it works, particularly in regards to the JIDE docking framework. Once you have your POC, you can recap here the things you've learned (and update your plan in the first post) and we can discuss some more.

My only other input would be to keep testing in mind. Lots of dependency injection so that mock objects can be used in testing. (Not necessarily for your POC, but for the resulting implementation.) Once this gets close to being merged, there's going to be a LOT of scrutiny because of how intrusive it will be for users; I'd hate to see you get too far into it only to find some killer objection late in the process. 🤔

[Merudo]

I've made a proof of concept here: https://github.com/Merudo/maptool/commit/38eb41188f1af9dba42589a63332d2aa1ac4909d

The new functions dialog5 and frame5 work quite similarly to dialog and frame but support HTML5.

The frame5 uses the same JIDE framework as frame and the compatibility appears good :).

I've tested it with my existing Call of Cthulhu framework, and it seems to work pretty well.

I've not tested the new HTML input types (email, range, search, etc) or javascript, though.

EDIT: on 125/175 HIDPI displays, the frame sometimes appear slanted.

[cwisniew]

I had a quick look, in AssetURLStreamHandler.java you shouldn't be creating a new key just because there is a sizing modifier to the passed in asset ID. If possible you should retain exactly the same behaviour as before with regards to asset IDs.

[cwisniew]

I've not types the new HTML input types (email, range, search, etc) or javascript, though.

JavaScript (and proper CSS) support will have to wait until we have somewhere to store JavaScript and CSS in the campaign structure (hint its not in token properties)...

[Azhrei]

I understand you to mean no CSS or JS from within MT, but there’s nothing to prevent CSS and JS being referenced using an external URL, is there? Assuming the HTTP handler isn’t being blocked...?

[Merudo]

JavaScript (and proper CSS) support will have to wait until we have somewhere to store JavaScript and CSS in the campaign structure (hint its not in token properties)...

I'm not sure what you mean by that. WebView should support JavaScript out of the box via the <script> tag.

And CSS can be accessed when it is in a macro via <link href>.

[cwisniew]

I understand you to mean no CSS or JS from within MT, but there’s nothing to prevent CSS and JS being referenced using an external URL, is there? Assuming the HTTP handler isn’t being blocked...?

There are certainly very good reasons to not allow that.
These will be used for character sheets, macro tools etc in frameworks people distribute and use. These things should not be reliant on a third party site being available to function.

[cwisniew]

JavaScript (and proper CSS) support will have to wait until we have somewhere to store JavaScript and CSS in the campaign structure (hint its not in token properties)...

I'm not sure what you mean by that. WebView should support CSS and JavaScript out of the box.

and where does the JS or CSS come from? it has to be stored somewhere, and as per my just added comment above, 3rd party website is not acceptable. (infact this should be specifically blocked by the implementation)

[Merudo]

and where does the JS or CSS come from? it has to be stored somewhere

JS is defined in the HTML page just like any HTML element with the <script> tag (try this), and CSS is already accessible from macros (check this)

[Azhrei]

I agree that in an ideal world, the campaign should be entirely self-contained. But we already have repos that rely on third party sites. IMO, it’s actually less of an issue to rely on a third party CDN for CSS and/or JS in terms of reliability.

However, I would support the enable/disable option that Jamz added for his REST functions being used to similarly control access to external resources for the web view component; I’m definitely concerned about DOS attacks and other attack vectors that allowing JS would open up.

(I’m also a bit concerned that the enable/disable option is in Preferences instead of campaign properties, but not _that_ concerned. Seems like maybe it should be in both places with a logical AND operation. But that’s a separate discussion.)

[Phergus]

Certainly if a webView frame/dialog was 100% compatible with the existing functions then a new set of macros wouldn't be needed.

To bring in the JFX HTML editor would be a whole new dialog and not actually part of the macro dialog/frame. Right? Would it need to be in a dockable frame?

To do any more with dialogs/frames than what we have now but bringing it up to HTML 5 would require significantly more work and support such as the filesystem internal to the campaign that @cwisniew mentioned.

[JamzTheMan]

Agreed. Figured I would spark some convo and ideas to make sure things are at least thought about. :smile:

The HTML editor would/could be its own dialog. Although, ideally it would be merged/used with the macro editor somehow. I assume most frame HTML dev would be while doing macro coding. Maybe that's a 2.0 thing then...

[Merudo]

The HTML viewer in JFX supports animation (of GIFs) which I assume use a separate thread for each animated image (just an assumption on my part; a quick Google search was inconclusive). If true, the MapTool stack size for threads could be prohibitive (although javafx.concurrent may allow setting the stack size on a per Executor basis, but I have no experience with that package).

It does support gif, and in fact I was able to easily display an animated gif from asset ID using the new dialog5/frame5 functions.

I would expect WebView to be relatively efficient and not use one thread per gif. But even if this is the case, I'm don't expect this to be an issue: I believe java can handle 6500+ threads, enough to display thousands of gifs at the same time.

[Merudo]

So Markdown support (via this new webview) would be nice (and can come later) but maybe also the WYSIWYG editor as well? FX has a built in HTML editor that could be used?

I'm all for Markdown support (it looks awesome) but I don't think it would be easy to do in webview - there is really no built-in support for it.

For Markdown we could use javafx.scene.web.HTMLEditor instead, or one of the many other libraries available.

Certainly if a webView frame/dialog was 100% compatible with the existing functions then a new set of macros wouldn't be needed.

I tested it with my framework and I couldn't find any compatibility issue - that is, the same code that worked with frame/dialog also worked with frame5/dialog5. If there is any incompatibility, I'd be glad to fix them.

[Azhrei]

I would expect WebView to be relatively efficient and not use one thread per gif. But even if this is the case, I'm don't expect this to be an issue: I believe java can handle 6500+ threads, enough to display thousands of gifs at the same time.

Heh, I don’t expect webview or any other components to be efficient at all — there’s less surprises that way. 😉

Let’s see, 6500 threads at 8M each is 5GB+. You still think that’s not a problem? (Reminder: physical RAM plus swap/paging is the limit of virtual memory for a system.)

Most OSes have limits well below your 6500 number, anyway. The point is not “what are the limits”, but “what are the ramifications”.

But none of this is important right now as you explore your proof of concept. 🙂