Go: proposal: golang.org/x/exp/shiny: an experimental GUI library.

[nigeltao]

Purpose.

First and foremost, this is an experiment. This is not 'the official Go GUI (Graphical User Interface) library' or the only blessed way to do things. Unlike a well-understood problem like decoding a JPEG image into an in-memory buffer, there are many possible approaches to GUIs, including existing code both under golang.org/x/mobile and elsewhere. This is simply an exploration. I've reached the point where I have an interesting if minimal demo, and I'd like to share and discuss the code, design and ideas.

Name.

There's nothing particularly deep about the "shiny" name. I just like shiny things.

The code would live at golang.org/x/exp/shiny. To emphasize what I've said above, this is an experiment, and this is _a_ GUI library, not _the_ GUI library, so it's not e.g. golang.org/x/gui.

It is under golang.org/x because I want to use the same code review process as everything else under golang.org/x, a process I use every working day.

Overview.

There are two layers: a lower-level window layer, and a higher-level widget layer.

For the lower layer, the primary concept is a Window. On the input side, a shiny.Window looks similar to what the golang.org/x/mobile/app package currently calls an App: you get a channel of events, such as key and mouse events.

The output side is where things diverge. x/mobile/app assumes that you paint pixels with OpenGL, either directly via the x/mobile/gl package, or indirectly via e.g. x/mobile/exp/sprite uses x/mobile/exp/gl/glutil uses x/mobile/gl. Instead, shiny has twin concepts of Buffers and Textures which map naturally to an OpenGL implementation, but also allow other implementations such as one that speaks a pure X11 protocol, without the need for cgo and OpenGL shared libraries. I assume that you could do somewhat similar things on Windows and Mac.

A Buffer is an in-memory, in-process bucket of bytes. A Texture is an opaque handle to something 'out-of-process'. A Buffer's contents can be Uploaded to a Texture, and Textures can be Drawn to the Window. As an optimization, it may be possible to Upload a Buffer directly to a Window.

For OpenGL, you can think of a Buffer as the result of a simple make([]byte, etc) call, and a Texture as an OpenGL texture. For X11, you can think of a Buffer as MIT-SHM shared memory (if applicable) and a Texture as a (server-side) Pixmap.

Uploads are 1:1 between source (Buffer) and destination (Texture) pixels. Draws can be subject to an arbitrary affine transformation (including simpler copies and scales) between source (Texture) and destination (Window) pixels. For OpenGL, affine transforms are simple shader programs. For X11, they are provided by the XRender extension.

For the higher layer, you construct a tree of widgets like buttons, text areas and containers. This shouldn't be overly surprising, except for the fact that the Go language doesn't have traditional object-oriented inheritance. The details of how that works is best explained with actual code, so I'll save it for a CL, if this proposal is approved.

The widget set is pure Go, like Swing instead of AWT or SWT. Sweating the details of a production-quality widget set is an enormous amount of work, and hitting that level of quality is not a short term goal. The primary goal is to explore what's possible in the GUI space with Go, and not necessarily to crank out a production-ready classic Win32 or Cocoa app any time soon.

A secondary goal is to drive development of the Freetype Go port - it has some basic functionality but also some long-standing known deficiencies. Writing a quality GUI text editor is again a substantial amount of work, and I don't intend to build one, but I intend that working on shiny will push the Freetype Go library to be good enough to let someone or some team do exactly that, if they had the time and inclination.

Relationship to golang.org/x/mobile.

There is a lot of overlap between shiny and x/mobile, and obviously I have contributed to both. The key difference, I think, is that the x/mobile code is rightly focused on mobile apps, where the dominant model is each app has only one (full-screen) window, whereas shiny starts more from a desktop perspective. To mix some metaphors, triangulating from different starting points (mobile vs desktop GUIs) can help avoid overfitting to local optima. In the medium term, the two should end up sharing a lot of code, but it's not obvious to me right now whether x/mobile should depend on shiny or vice versa, and it would be premature to pick a winner now. As I keep saying, there are many possible approaches, and I think it's valid to explore more than one of them concurrently.

There is also the operational concern that I would like the freedom for shiny to depend on third party libraries, such as github.com/golang/freetype and github.com/BurntSushi/xgb, but I'd also like to keep the golang.org/x/mobile tree free of such dependencies, at least for now.

Relationship to github.com/google/gxui.

I'm obviously aware of their work, but the two projects aren't strongly linked. As I've said, there are many possible approaches that are worth trying here. Longer term, we might discuss what we can take from each other's codebase, but it's premature to do that now.

Closing Remarks.

I'll repeat that this is experimental and exploratory. I just have a demo and I'd like to share it. I know that there could be a lot of excitement in the community for a 'Go GUI library', but please don't set your expectations too high at this stage, and the existance of this project should not invalidate any other Go GUI projects.

[robpike]

SGTM

[taruti]

What is the license policy of the x/ tree? Probably the FTL (or GPL) requirement is fine for most uses, but to avoid clients accidentally breaking the attribution requirement it should be clearly documented if there is a freetype dependency.

[ajstarks]

Interesting proposal. Can you sketch what a simple client program would look like?

[eaburns]

Looks very interesting. I also prototyped a UI library in Go. It shares many of the ideas that you propose.

It has two halves: the ui package (described below) and the gui package (not written) for simple widgets.

The ui package deals with Windows and Events. It's currently implemented on top of SDL2 using cgo. With ui, one can draw Buffers to Textures and Textures to Windows. However, I call Buffers image.Images and drawing them to Textures happens via draw.Draw. It also has an interesting (I think) solution to the annoying OpenGL thread context issue. In short, one calls Window.Draw with a drawing function that accepts a ui.Canvas. The Canvas has methods for actually drawing to the Window. Window.Draw ships the draw function and Canvas to the thread with the OpenGL context in order to perform all the drawing. This obviates the need for channel operations and context switching for each call to a draw primitive.

Perhaps you'd like to take a peek: http://godoc.org/github.com/eaburns/T/ui.

Also, I'd be interested in ditching my package in order to help out with Shiny. Particularly because the motivation for my ui package is to write a text editor, and you explicitly mention as a target use for Shiny.

[andrewchambers]

My vote for first demo app is an acme clone :)

[crawshaw]

SGTM

Note that x/mobile/app's OpenGL dependency will eventually be generalized to ES2, ES3, Metal, and Vulkan. But all of these share a similar GL style and shiny will need different machinery.

I would like to see the code in x/mobile/app converge in the medium term. There's a lot of subtly in cross-platform window and event management I would like to combine in one place.

[mattn]

Sounds good to me. Are you talking about mobile limited?

[aclements]

In the medium term, the two should end up sharing a lot of code, but it's not obvious to me right now whether x/mobile should depend on shiny or vice versa, and it would be premature to pick a winner now.

Rather than either depending on the other, would it make sense for them to share a common third dependency? I don't have a strong sense of exactly what would be shared between them, but, as an example, the OpenGL bindings seem like a common dependency that is neither mobile- nor GUI-specific, but is required by both.

Put another way, x/mobile is focused on mobile; it needs things like windows and OpenGL, but doesn't (itself) need a GUI toolkit. shiny is focused on GUIs; it needs things like windows and OpenGL, but isn't specific to mobile.

[owenthereal]

:+1: Is there any more info on what it is like?

[theGeekPirate]

I've reached the point where I have an interesting if minimal demo, and I'd like to share and discuss the code, design and ideas.

Mind sharing?

[crawshaw]

@aclements I believe that is what Nigel and I are both trying to say, there are just enough moving parts that's it is difficult to describe. Another attempt:

There should ultimately be one shared piece of code for creating windows and delivering events. It may have two separate interfaces, one for mobile and one for desktop, or maybe not. To some extent, it will have to know about the underlying drawing technology the user wishes to use: GL (probably all varieties can be collapsed at this level), Metal, Vulkan, or a *image.RGBA that is fed into the underlying system some other way (the X11 protocol). This is the work mostly being done in the x/mobile/app package right now, and needs to be generalized a little bit and live somewhere other than x/mobile. Both mobile and shiny will use it. In some ways this window+event package is like glfw with two differences: firstly that it's written in Go, and secondly that it doesn't necessarily require GL.

Then there are the packages for accessing the graphics system. Right now we have x/mobile/gl for GL ES 2. It has no mobile dependencies, and should live somewhere else, and can be optionally used by shiny. There are other GL packages out there that expose more features (and a fork of x/mobile/gl that works on GopherJS.) The x/mobile/app and its future generalizations will depend, probably in a "pluggable" way, on these graphics system packages.

[skelterjohn]

Consider treating the input/output (keyboard and mouse/pixels on the screen) as a completely separate issue from GUI widgets.

As an example, I created github.com/skelterjohn/go.wde to do the input/output, and github.com/skelterjohn/go.uik to do GUI widgets _using_ go.wde.

[phonkee]

Sounds interesting, is there anything we can help with?

[andlabs]

Oh boy, this is quite a long thread with a lot to talk about. here goes!

---

@nigeltao original post

Your design is in good company: this is pretty much how GTK+ works; the name of its lower layer is GDK.

What I learned from my own UI package attempt is that the way operating systems give you events and the way Go cahnnels work don't line up fully. With a Go channel, a transaction is considered complete once the message is received, and the code that produces the message can continue operating regardless of what the receiver does. But with a traditional GUI system, the system expeccts that you handle a message fully before asking for another one, and will generate input, painting, and timer events according to that design. In practice, things may seem to work at first, but unless you have an "okay, done with this one" channel for everything, you're eventually going to fall into the trap I fell in with the first version of my package ui: needing to introduce such a channel for painting events and everything deadlocking as a result. I also don't know how x/mobile handles it.

Of course, I could be wrong and this could be a non-issue if done right; only experimentation will tell. (Until then, I've decided to separate my own package ui from Go as much as possible, with the intention of avoiding conflicts between the Go runtime and what the C world expects.) There is definitely room for experimentation here (experimentation I am very interested in).

Next, here is a very very important question. Or rather, two questions whose importance is mutual. Using "shiny.low" to refer to the lower layer and "shiny.high" to refer to the higher layer:

1. Will a shiny.low.Window map solely to a top-level window (such as the one your web browser runs in) or to an arbitrary region of the screen that has its own independent event handling?
2. Will each shiny.high.Widget have its own shiny.low.Window?

If you answer "the latter" to 1 and "yes" to 2, the GTK+ project has done this in the past and found that it doesn't really work well on X11. The link is to a video that shows a window being resized in GTK+ 2 and the massive flicker that resulted as the widgets were redrawn. (Windows and OS X don't have this problem as much because of heavy optimization and API design tricks in the former's case and a similar design to what I'm about to say in the latter.) You may find it more performant to chop up a shiny.low.Window into each shiny.high.Widget's parts and mux out events that way. GTK+ 3 has switched to this model.

Another thing is that for the widget library, you would need more than just a text rendering library; you would also need a full vector graphics library. I know the old code.google.com freetype package had one built in, but I'm not sure how comprehensive it is for the purposes of a GUI package. That also leaves open the question of theming.

Threading is another issue. Windows and OS X (I don't know about the other Unix display servers) assume that all your UI operations exist on the same thread; OS X goes even further by saying that it must be the very first thread the OS creates (though there is an undocumented function to change it; I have yet to list the full side effects). I'm aware of runtime.LockOSThread() and that might be the solution you'll want to go for. (The future of my package ui will lie in having the GUI thread be a C thread that Go knows nothing about, and that uses other methods of communicating with other threads. In my early experiments this seems to work fine as well.)

That being said, I would be happy to contribute to this project if it did exist; I could even possiby (no promises, just a thought) prototype the low level part this weekend using code that I already have in my ui and libui projects. skelterjohn has also tried something like this with his go.wde (the low layer) and go.uik (the high layer) projects; possibly using some of that code might also help.

One big question that looms in my mind is how cgo, moving garbage collection, and arrays and slices will work, since it isn't very well defined (as far as I'm aware, anyway). At some point we're going to have to feed Go data blobs to C. I should have stated this in that one issue; I'm really not against moving garbage collectors so long as the ability to safely feed large amounts of data to the C world in a well-defined manner is provided (or the rules for doing so defined).

---

@crawshaw

See what I said above about how to map a low-level window to a high-level widget. I'm not sure how this will conflict with golang.org/x/mobile's needs... And in fact, wouldn't having a single end-all-be-all OpenGL package be the logical conclusion of this?

[rsc]

SGTM.

[ajstarks]

I agree with @andlabs on having a robust 2D vector API. In fact this is more important to me than standard "widgets" like buttons; I'd like to have the ability to build my own interaction objects without being constrained to one set. Would the vector stuff go in shiny.Widget or shiny.Window?

[nigeltao]

@taruti. The golang.org/x code is licensed the same as Go itself. For example, see https://go.googlesource.com/exp/+/master/LICENSE

IANAL, but I think you're right that shiny packages should document that an (implicit) Freetype dependency, or other third party dependency, could have license implications. Such a dependency would be in the high-level layer, not the low-level one.

[nigeltao]

@ajstarks, it looks like the consensus is to go ahead with this, so the best place to show example code will be in what we call a CL or change list (a git-codereview change).

[nigeltao]

@eaburns thanks for the link. I'll check it out.

[nigeltao]

@mattn, I'm sorry but I don't understand the question. What does "mobile limited" mean?

[nigeltao]

@aclements, yes, there might be a common third dependency in the future, and packages might move, but I don't know what the ideal package 'org chart' should be yet, and part of landing code now and iterating will be to learn what that should be.

[nigeltao]

@jingweno, @theGeekPirate, yes, I will start sending out some code, as it looks like the consensus is to go ahead with this. Code reviews will be CC'ed to the https://groups.google.com/forum/#!forum/golang-codereviews mailing list. Grep for "shiny".

[nigeltao]

@skelterjohn, yes, I think we're in agreement here. There will be separate shiny packages for the lower level (key/mouse/pixels) and higher level (widgets) as I originally described.

[nigeltao]

@phonkee, not anything specific right now, but constructive feedback is always welcome.

One general issue is that I'd like shiny to work on Windows, whether via OpenGL or otherwise, but I don't really run Windows any more, and certainly haven't written any Go code for Windows. So, after I submit what I've got so far, I'd appreciate any Windows expertise, from you or any other gopher.

[nigeltao]

@andlabs, the x/mobile code tries hard not deadlock around two-way channel communication. For example, see the pump function in https://go.googlesource.com/mobile/+/master/app/app.go.

A lowshiny.Window is always a top-level window, and highshiny.Widgets will not have their own 'window'. Client-side widgets, a la GTK+3.

Re vector graphics, that would indeed be nice-to-have, and the Go Freetype port does have a Bezier curve rasterizer, but I'm not convinced that it is a must-have yet. I know that @ajstarks is keen on SVG, but e.g. I'm not aware of a lot of web-app that use SVG for their GUIs, and to echo @andrewchambers' comment, we could possibly write an acme clone without the Go equivalent of a Java2D. I'll repeat that a Go2D would be nice-to-have, but I think it's a separate package than either lowshiny (which gives you an *image.RGBA that you can draw vectors on) or highshiny (which would be irrelevant; affine transformations of buttons and text areas are out of its scope).

Re threading, we already have this concern with golang.org/x/mobile/app and OpenGL, and I think the same approach can apply here.

Re cgo, blobs of bytes and a potentially moving GC, yes, this is an open question, but not one I think limited to shiny.

[andlabs]

Fair points. I'm not familiar with how x/mobile deals with threading and two-way communication; apart from pump(), where else can I look to learn what it's doing?

What exactly do you have ready now, since you seem to be implying that you do have something? (Not asking to upload anything; just wondering.)

[nigeltao]

I don't have a better suggestion than reading the x/mobile/app source code, although it's not the simplest Go package, as it has a lot of OS-specific parts.

As for what I have ready now, I'll start sending out code reviews shortly.

[mattn]

@nigeltao I expect to use this GUI library on Windows, OSX, Linux.

[adg]

SGTM

[nigeltao]

@mattn, so do I. :-)

[ngrilly]

As you wrote in the issue description, there are multiple possible approaches to GUIs. Do you intend to draw some inspiration from the "immediate mode" (versus "retained mode") recently popularized in some way by React and now React Native?

In my experience, this is really to not think anymore about the differences between the initial rendering and the subsequent updates.

References:
Immediate-Mode Graphical User Interfaces
Removing User Interface Complexity, or Why React is Awesome

[egonelbre]

@ngrilly both "Immediate" and "Retained" mode rely on stateful widget/layout representation (with the exception of games, which can allow updating screen at 30 times per second). Application UI-s have a lot of pixels that can carry over directly from the previous frame, hence it's beneficial to keep state and avoid redraw. Note, that I do agree that using "transient" representation in applications and let the GUI framework worry about the stateful part makes things much easier to manage.

[ngrilly]

@egonelbre I'm not sure to understand.

Immediate mode doesn't necessarily rely on a stateful widget/layout representation, according to the usual definition of "immediate mode". Wikipedia states that in immediate mode, in contrast to retained mode, lists of objects to be rendered are not saved by the API library. The whole UI is re-rendered for each update.

I agree that the underlying implementation can maintain data structures as a kind of cache for optimization purposes. And I agree that an immediate mode API can be built above a retained mode API (this is what React does), and reciprocally.

I was wondering if this new GUI toolkit for Go could be an opportunity to explore a more "transient" API and let the app manage the state, as you wrote at the end of your comment :-)

[egonelbre]

@ngrilly - sure... I simply meant to say that for complicated regular applications you need stateful part to keep the CPU usage low. Also, even if @nigeltao implements the stateful part, the immediate mode can be implemented on top of it - although code-wise it might not be as clear as pure immediate mode.

[anacrolix]

k

[anlhord]

this made my day. ACKed by important figures and deemed consistent with go philosophy (decision of go people that duck typing an interface{} is superior to everything) is all that's needed. when it's merged to the main go repo. I'll be laughing for 2 days at such a spectacular failure.

why would we build atop let's say wayland protocol, created by those who actually understand how a modern windowing system app should operate on lowest level when we can go Not Invented Here...

[andlabs]

Um, what exactly are you suggesting @anlhord? That the idea here is to write a window system? That some vague design decision of Wayland conflicts with this? Can you elaborate that? You should also look up GDK, because it works really similarly to this (a GdkWindow provides a window system object and you use cairo to draw to it instead of a pixel buffer (even though there are pixel buffers if you want them) but other than that), has been around for 20+ years, and is used in lots and lots and lots of applications, then try to argue that it failed :) And how does Wayland do things differently from Windows, X11, and the OS X window manager anyway? At the end of the day you have a window and a pixel buffer. Also, who are you suggesting is saying duck-typing with interface{} is a good idea? Russ? Rob?!

[anlhord]

1. type Buffer interface is useless. there should be a decent 2d slice in the language by now.
2. Release() in Buffer ? Really? I thought this was a GC'd language
3. type Window interface is too high level and mixes input stuff with drawing.
   there is nothing to review here really

yawn

perhaps somebody can review my proposed language change at gccgo in the meantime

https://groups.google.com/forum/#!topic/gofrontend-dev/VOtbIKclOPo

[andlabs]

And Wayland doesn't mix input and drawing? Because protip: every window system I am aware of does.

Where do you see Release() in Buffer? Do you mean Release() in Texture, the OS handle wrapper? Because that needs releasing... Or did Nigel upload his proposal and I don't see it?

I have no comment on 2D slices. image.RGBA gets along fine without it. I'm still not sure what Buffer does differently.

As for your proposal, have you described it in written form anywhere (not just a code sample)?

[tucnak]

Idea itself is really promising! Imo you should definitely think of QML and how it could be applied here. My experience with QML on Qt stack was really great and I think it can be fairly integrated with Golang.

[andlabs]

@tucnak you are aware of this, right?

[tucnak]

@andlabs I obv heard about it, but haven't got enough time to deeply look into. My colleagues reacted poorly about it though.

[andlabs]

Right; I haven't used it myself, but as it stands it's the most stable and well-supported GUI library for Go, so it must have some value :S I see you (and thus your colleagues) might have specialized needs, however, so I would ask them what the current version doesn't do.

[tucnak]

@andlabs Thanks for a clue, I gonna dive into the whole thing this weekend and give it a close look.

[niemeyer]

For the record, despite being the author of Go QML, I'm in full support of a native toolkit for Go that the community gets strongly behind. The reason Go QML exists is because it is a cheap way to go fast forward towards GUI applications in Go, leveraging a complete and modern toolkit. Conversely, implementing a full graphical toolkit in Go is a major undertaking, and to be honest pushing such a project naively will most likely do more damage to the ecosystem than otherwise, as it splits attention across several half baked offerings (x/mobile? google/xgui? x/shiny? etc). I wish someone that really cares for GUIs and knows how to organize a community could take the lead on this, and stick with it for a longer while.

[ajstarks]

@nigeltao just to clarify, I'm not expecting SVG, but as you put it "Go2D". If that can cleanly ride on shiny, cool. I'll restate that doing transforms on GUI elements is useful. OpenVG is an experiment with what is possible with a simple 2D API, but it lacks things like input support and animation that I would look to something like shiny to provide.

[ianremmler]

This looks very promising. I've considered trying my hand at making a simple GUI toolkit (if it even deserves to be called that), possibly even abstracted over x/mobile and GLFW or similar. It's way on the back burner at this point, but shiny seems to be in the same spirit.

There are a few things that I hope will be at least possible, hopefully easy, and perhaps even baked in.

• Non-rectangular widgets: The painted and event-receiving areas (which may or may not be identical) need not be the bounding rectangle of a widget. I have done this in a double buffered OpenGL application by drawing a "footprint" in the back buffer in a color coded to correspond to the widget, for example.
• Widget-local coordinate system: It sounds like the proposed affine transformations would be post-render. I would like a natural way for a widget to have it's own extents, origin, (maybe rotation and shear) that apply when rendering to the buffer, as well as the ability to convert to and from other widget, window, and screen coordinate systems.
• Aspect ratio constraints: A widget may have a fixed aspect ratio and weighted values specifying where to position the widget along the less constrained axis.

I'm really looking forward to see where this goes.

[anlhord]

@andlabs this is the shiny I found:

https://go-review.googlesource.com/#/c/12568/1/shiny/shiny.go

[nigeltao]

@ngrilly, @egonelbre, "immediate mode" is an interesting concept, and worth exploring, but the shiny widget library will take a traditional "retained mode" approach. As I said, there are many possible approaches, and I'm more than happy for somebody else to try an "immediate mode" substitute for the higher-level, that re-uses the lower-level. Or it could re-use the higher-level. I haven't used React myself, but IIUC, it re-uses the browser's DOM, and that DOM takes a traditional "retained mode" approach.

@anlhord, if you feel that there is nothing to review here, then you are free not to review it. As for your proposed language change, please file a separate proposal for that, and we can discuss it there.

@anlhord, @andlabs, as for Release, yes, some resources can be explicitly released. Just like the io.Closer interface exists for a reason, and we don't rely on garbage collection to close your database connection, io.Pipe, or gzip.Writer.

@andlabs, @anlhord's proposal was previously discussed at https://groups.google.com/forum/#!topic/golang-dev/_V7_Be-AA38, but this is off-topic here, and I have asked @anlhord to file a separate proposal.

@tucnak, by QML I presume you mean a declarative way to specify a widget hierarchy. It's an interesting idea, especially for people wanting to build GUIs visually, in IDEs, but we're a long way from there yet. In any case, it seems like something that could be a separate package that used (high-level) shiny, rather than necessarily being part of shiny.

@niemeyer, yes, I would like to repeat that shiny does not aim to be 'the official GUI package'. It is an experiment, although it will hopefully influence golang.org/x/mobile/app in the medium term.

@ianremmler, I would phrase it a potentially transparent widgets, instead of non-rectangular widgets, as I'd still like the concept of a bounding box (a rectangle). Each widget will also have its own relative coordinate system that is a translation of the window coordinate system, but not an arbitrary affine transformation (sorry @ajstarks), at least not in the near term. As for fixed aspect ratios, layout will indeed be an interesting problem.

[andlabs]

All right then. Argumentative posts aside, glad to see that a CL is up now; I'll look through it sometime in the next day or two and make my own comments as well :)

[egonelbre]

Here's a thought, let's gather all the knowledge and experience from using UI libraries. I'm sure everyone has their own pleasant and horror stories about some specific libraries. Or they know some specific library that did something exceptionally. I've setup a document with that in mind https://docs.google.com/document/d/1QtqrrSG1lbnmk8wPTOVNEjfby3zfymS8vHtM8cxy67I/edit everyone should be able to edit/write to it (note, if there are too many trolls I'll switch the global edit and manually moderate it, hopefully I don't have to do it). I currently added the things I knew from the top of my head - I believe, if everyone added just that it would be sufficient to get a quite good picture.

[nigeltao]

For anyone interested in the actual code, be aware that the "1" in
https://go-review.googlesource.com/#/c/12568/1/shiny/shiny.go
means "patch set number 1", and looking at that can be informative (due to the review comments), but the latest version of that change is always at
https://go-review.googlesource.com/#/c/12568/

That is, of course, only the first code review of many. As I said above, code reviews will be CC'ed to the https://groups.google.com/forum/#!forum/golang-codereviews mailing list. Grep for "shiny".

[andlabs]

Well I made a comment on Gerrit but it doesn't seem to have my name attached to it...? I went to the Contact Information preference and entered my name in the Google account window that popped up, but clicking Reload didn't actually load it :S What's going on?

[ianlancetaylor]

@andlabs What's the link?

[andlabs]

@ianlancetaylor the "Anonymous Coward" comment at the bottom of https://go-review.googlesource.com/#/c/12568/ (it starts with "So do I have the right idea here with the intended use of this package?") - what exactly did I forget to set up?

[ianlancetaylor]

@andlabs I don't know. If you go to https://go-review.googlesource.com/#/settings/, what do you see?

I see a little table:

Full Name Ian Lance Taylor
Email Address [email protected]
Registered Dec 8, 2014 10:14 AM
Account ID 5206

[andlabs]

I see a blank Full Name:

Full Name   
Email Address   [email protected]
Registered  Jul 24, 2015 2:31 AM
Account ID  9190

I thought the place to enter the full name is under Contact Information, but editing it and clicking Refresh doesn't bring anything up. Am I missing something?

[ianlancetaylor]

@andlabs Can you edit it at https://www.google.com/settings/name?pli=1 ?

[andlabs]

Yeah, it shows up fine there. I'm going to guess it'll take some time to update?

[andlabs]

Nope it's been 24 hours and it still won't grab my name from the Google Account data. I'll take this up with the Gerrit folks; thanks for helping in the meantime!

[niemeyer]

@niemeyer, yes, I would like to repeat that shiny does not aim to be 'the official GUI package'. It is an experiment, although it will hopefully influence golang.org/x/mobile/app in the medium term.

Being official or not matters little. The question is how much commitment exists behind it, both in technical terms and in community management terms, so that there's at least some small chance that it becomes more than a toy at some point.

[dmitshur]

Hi Nigel,

Overall, the proposal SGTM. I think it's an interesting area to experiment in. I think Go can be excellent for this, but as you said, there isn't yet one known best solution.

I just have some non-actionable thoughts and comments I'd like to share.

First, I like that you're looking at making it quite general and therefore possible for multiple backend implementations (OpenGL, X11, others). Just out of curiosity, I immediately started thinking how, if at all, this could have an HTML5 backend (via pure html, or perhaps canvas element, or maybe just WebGL). I don't think it's a priority, but it's interesting to see if that could fit in. I'll keep thinking about as I see shiny evolve.

Second, I just think it's really interesting that this proposal is being started despite there already being github.com/google/gxui, an experimental cross platform UI library written from scratch in pure Go. I mean it's interesting because it shows there's still room for experimentation and thinking about ways to solve the problem. Having multiple attempts shows that this is a more active area of interest and it's likely to have interesting results in the near future.

Finally, I've attempted to tackle creating a cross-platform GUI in Go about a year ago myself. The GUI was not my primary goal, it was just a dependency of a larger project I wanted to build, and no pure Go cross platform GUI (like gxui) existed at the time so I worked on my own. See https://github.com/shurcooL/Conception-go#screenshot for a sample screenshot. It was partially a port of the original undertaking in C++, and I used it as a platform to experiment with Go and see how I liked it.

It was a rather large undertaking and I only had a chance to spend a few months to work on it in my spare time, so I wasn't able to get it to completion. I'd estimate the code being maybe 15% of where I'd want it to be. But I had good ideas about how to make it much cleaner and better. As it is now, it's really incomplete and not usable by general public, but working on the prototype proved to me that Go is quite capable of handling such tasks, it just took some interesting ways of figuring out some things. And in the end I had a project that's very easy to distribute; you can literally run a single go-get command, wait 1 minute, and have a quite large project build successfully and run today.

In conclusion, I'll be keeping an eye on shiny as it's interesting to see how it'll go. :)

[rsc]

There seems to be general agreement to accept this proposal.

@adg, how do we do that? :-)

[adg]

I agree there is consensus to accept the proposal. It is hereby accepted.

[andlabs]

How do I clone exp for contributing without using go get? Just clone it from github?

[nigeltao]

@andlabs I don't understand the "without using go get" part. If you want to contribute, run "go get golang.org/x/exp/shiny", make your changes in that directory, and send them out with git-codereview, a la http://golang.org/doc/contribute.html

[andlabs]

Windows question time: What should we assume about the window canvas on a paint.Event? The Windows AlphaBlend function only supports the equivalent of draw.Over, not draw.Src. Should I just copy the RGB components of the pre-multiplied RGBA color onto the drawing surface in the case of draw.Src?

General question: how should we handle situations where only part of the window needs to be redrawn? All the painting models I'm familiar with, including X11 expose, work this way.

[nigeltao]

Yeah, just copy.

As for dirty rectangles, we could add a "Dirty image.Rectangle" field to the paint.Event struct, but I'm not sure if that performance optimization really needed these days. For example, on phones, all apps are the equivalent of full screen and don't overlap. For X11, the expose event also tracks whether there are more exposes coming (for non-rectangular exposes), and I intend to just drop all but the last one, and make that last one just trigger a full-window repaint. It seems to work fine in practice, and if it doesn't, then as I said, we can add a Dirty field later.

[tv42]

Probably worth looking more at Wayland than X11, these days. My understanding is it's all flip buffers there, with full repaints.

[tv42]

Actually, damage in Wayland seems to go the other way around: client tells the compositor what parts of its buffers it changed. Because of the compositing nature, no buffer is damaged by the compositor. http://wayland.freedesktop.org/architecture.html

[andlabs]

Good to know.

Last question before I upload Fill: do the other drivers send a size.Event before the first paint.Event? The basic example seems to imply so.

[nigeltao]

I would expect them to, but I don't think it's mandatory, and I think that any correct program should be able to handle seeing a paint.Event before any size.Event. The example/basic works here because filling zero-width zero-height rectangle is a perfectly fine no-op.

[andlabs]

All right; the Windows code will need to synthesize a fake size.Event in this case since Windows doesn't have an initial 0x0 state (and doesn't generate a sizing message for the initial size). I've done this already; just need to implement the draw.Src filling and then I can start the C->Go conversion. :)

[adg]

I think we can close this issue now that the proposed change is well underway.

[andlabs]

Do you have a minimum OS requirement in mind? If we're willing to drop anything before Vista SP2 with Platform Update, I'm wondering if using Direct2D instead of GDI/GDI+ would help, especially since it does support retaining window contents in the same way x11driver does. The only problem is recreating COM, though I think package syscall's Syscall* functions might help here...

[nigeltao]

My instinct is that I'd like shiny to work on whatever version of Windows that regular Go does, but I'm not overly familiar with Windows programming these days.

OTOH, if WinXP is no longer officially supported by Microsoft, then I suppose that assuming version >= Vista is fine, if it makes things significantly easier.

So really, I don't have a firm opinion, one way or the other. Whatever's easiest, I guess. I would have thought that wrestling with GDI+ would be easier than wrestling with COM, but I don't know for sure.

[minux]

Go still supports XP and 2003, so I'd expect shiny to support those
platforms as well.

[egonelbre]

I probably missed some discussion - what's the problem "retaining window contents"? Do you simply mean that you don't want to draw directly to the screen? In that case it's possible to implement the back-buffer yourself and blit it to the screen.

[nigeltao]

Yeah, we decided to always double-buffer, so the windriver (and x11driver) backends will have to implement the back-buffer themselves.

[bonly]

good work!

[sqweek]

I see I am very late to the party, but I wanted to add one thing:

@andlabs wrote:

you're eventually going to fall into the trap I fell in with the first version of my package ui:
needing to introduce such a channel for painting events and everything deadlocking as a
result.

@skelterjohn's go.wde package avoids this particular pitfall by taking care of the paint event itself, rather than passing it on to the application. Here "taking care of" means "blitting the most recent buffer the application has marked as a completed frame to the screen". From an application perspective I feel it works very nicely, allowing painting and input event handling to be handled independently.

[yalay]

SGTM TOO!

[ghost]

@nigeltao

MDL components
https://github.com/material-components

Google have put iOS, web, android MDL all together now.

If you be cool if shiny was at this point too.

[tildeleb]

Status update? It's been a year since the author commented (here) on this project and any major commits are months old. Is this dead?

[nigeltao]

It's moving slowly, as shiny is not the only thing that I'm working on.

Some recent progress might not be so obvious, as it involves some font work in golang.org/x/image/font, which will certainly be used by shiny, but that code is not under golang.org/x/exp/shiny per se.

[ghost]

@nigeltao
Thanks for the update.

It's definitely a tough / complex thing to get right. Also i saw the work on the vector graphics which looks like a really good base. i can see how you want to make the foundation right before letting loose on the easy stuff higher up.

If you dont mind i want to say a few things below and scratch my itch :) ; my intention is to highlight architectural opportunities, not increase pressure...

I have to ask though why not let others contribute in some form ? I stress "in some form". In the above thread allot of people for also itching to contribute, but i also see that the foundation was not ready and so there was too much flux in the direction to take.
If a game plan of the architecture tasks is done, i think you would have many contributors that would sign the CLA and use gerrit. Maybe some work can be done on non foundation areas; the MDL widgets can be done in parallel, whilst the layout engine catches up in the background.
Also the way the Flutter team solved the cascade layout / update issues is now well known where they gave very detailed video discussions on the matter.

Personally I played with different approaches and ended up resorting to use QT with golang to get what Shiny offers for now. It works very well and QT now has very decent MDL support. Their licensing is now open for Desktop and Mobile, but you pay for Embedded (automative, etc)
https://doc-snapshots.qt.io/qt5-5.8/qtquickcontrols2-material.html
But i would switch back to Shiny instantly if i had the chance because its far easier to innovate in pure golang rather than a QML, c++, golang environment like QT.

As we all know Ubuntu embraced it as their Official GUi engine in the end. https://developer.ubuntu.com/en/phone/apps/qml/tutorials/

Another later idea for Shiny is to use QML. I imagine about using it as the basis of golang code gen for both golang Web and Shiny. After all a QML file is at Design time a serialised form of an DAG, and at Runtime the object representation of that DAG in memory. So, golang code generation would work nicely at Build time to produce the code.

I have a rough golang based DAG for web GUI now using gopherjs, and plan to build a QML code gen approach with it. In terms of data binding, QML enforces very simple techniques (slots and signals) which is stupid simple but works. Its very early days though, and the next phase is to hook up the new MDL components from google.

Itch scratched .....

[andlabs]

What is this Flutter discussion (and what is Flutter)? And you can already contribute by signing the agreement and sending contributions in the same way you would to Go itself.

[nigeltao]

@andlabs Flutter is https://flutter.io/. I'm assuming that the discussions are the video'd talks such as https://www.youtube.com/watch?v=UUfXWzp0-DU about Flutter in general. I am not aware of any discussions of applying Flutter specifically to shiny.

@gedw99 as for contributions, yes, I'm aware that there are people itching to contribute, but I think that it's still a little early and the foundations are still not ready. Inviting people to write Material Design components isn't free from my point of view, as I'd still have to review code, answer questions, and write docs. Sure, I'd have to do that anyway in the long term, but my shiny time is already scarce enough in the short term. Yes, I know that progress is slow. Sorry.

[chenzhekl]

Is this project still alive? There seems to be few commits for a long time.

[nigeltao]

It's certainly not very active right now. I work part time, for uninteresting reasons, and shiny is unfortunately not my primary project.

I'd like to work on it again at some point, but I can't promise when that will happen.

[zacps]

@nigeltao I'd suggest handing this off to another maintainer. I'm sure there's someone around who'd have the time and experience to keep this going; it's obviously wanted.

[joeblew99]

Well to do that requires a roadmap first.

I doubt a person with the right skills to do this is available also. Would be better to delegate and supervise. But its really down to @nigeltao :)

i have noticed in the lat 6 mots there are about 30 projects using shiny now out here in github.
Its getting to the point where the framework is usable.
The packages of Node, widgets and flex helped allot to allow people to compose things.

For example:
https://github.com/hajimehoshi/ebiten

https://github.com/aarzilli/nucular

There are a few others that are written higher level layout, reflow and animation packages on top of shiny too. Reflow is a hard one.

Then there are people writing key native functionality for Mobile and Desktop too:

Virtual keyboard;

• E.G On mobile you are in an openGL TextBox, and the virtual keyboard should come up.

Cut and paste;

• E.G Keyboards based & Native OS overlay based.

Audio and Video capture and playback;

Intimate gesture control and keyboard mapping;

Printing via pdf templates;

• E.G in Form archetypes you will need to bind the data to a PDF template.
• Even printing via opengl even.

Sharing;

• E.G. You are in an app page and click “Share”, and the OS brings up all apps that handle that intent.

Notifications;

• E.G From within the app you wish to raise a native notification
• E.G You raised a notification from the Server via the mobile push gateways and the user has pressed on it in their Notifications bar, and it routes down into the right part of the App.

Linking, so that web links passed in route to the correct location within the app.

• E.G you send a http link to a friend and he has the app for it and so it opens the app, and navigates into the app to the right location. Google maps does this for instance.

---

There is no common GUI Material Design toolkit yet.
Because shiny is a simple framebuffer with immediate mode its not super hard to made these.
https://github.com/TheGrum/renderview

• Allows making image based controls, but whats needed is ones driven by shaders..

[nigeltao]

Handing this off to another maintainer requires there being a candidate to be that maintainer. For example, Rob Pike and Russ Cox are prominent gophers who have worked on the image/* packages in the Go standard library in the past, but these days, Rob is working on https://upspin.io/ and Russ is working on https://blog.golang.org/toward-go2 and they don't have much spare time for shiny. David Crawshaw has also worked on shiny in the past but he is similarly busy with other projects.

If somebody or bodies from the community wants to fork the source code and run with it, they have my blessing (but I won't have enough spare time to supervise). In hindsight, shiny would probably have been better as a vanilla github-hosted project instead of a golang.org/x project, although at the time, the github code review UI wasn't as good as it is now.

FWIW, there was some recent discussions of a new, multi-person Go GUI effort at https://groups.google.com/d/msg/golang-nuts/JyrN8cxWCrU/H6oZpv02BQAJ