Ebiten: Add a single-thread mode by introducing a new build tag

[jakecoffman]

This project is super cool so I finally started porting the cp-examples over to ebiten. With vsync off I noticed the ebiten tumble ran much slower than the cp-example tumble that builds the triangles every single frame.

The cp-example was getting ~1600 FPS and ebiten was at ~600 FPS.

So I did some profiling and noticed some channel related locks. From what I understand most game engines are single threaded, so I removed the threading and the ebiten version is now getting ~2100 FPS.

Is there a specific design decision that lead to threading in the GLFW code? If there's a good reason maybe add an option to disable threading to unlock peak performance?

[hajimehoshi]

Hi,

Thank you for the information!

OpenGL functions (and probably Metal functions) must be called on the specific OS thread. Without this syncing, the functions can be called on a wrong OS thread since goroutines can use arbitrary threads.

[hajimehoshi]

So probably we can disable the threading only when GOMAXPROCS=1. Let me think.

[jakecoffman]

I am aware of that restriction. I think if ebiten allows the user to disable threading and document doing this will mean that any goroutines they spawn will need to communicate back to the main thread then it's on the user to obey or get a crash.

[hajimehoshi]

My concern is that disabling threading for performance is counterintuitive and doesn't sound a way to go for optimization.

Just an idea: In the implementation, the exported functions in the internal/shareable package already processes with a mutex. Then, we can create a big main loop inside internal/shareable so that we can reduce bunch of internal/thread.Thread.Call.

[jakecoffman]

I tried this on a Mac and found that using threads the tumble demo used near 50% CPU, and without threads it used 15% CPU.

With threads

Without threads

[hajimehoshi]

Thanks.

In macOS's Activity Monitor, the CPU usage means the total of all the core usages, right? Then, this decreasing seems to make sense...

1358 might be related.

[hajimehoshi]

@jakecoffman Did you specify GOMAXPROCS=1?

[hajimehoshi]

Just an idea: In the implementation, the exported functions in the internal/shareable package already processes with a mutex. Then, we can create a big main loop inside internal/shareable so that we can reduce bunch of internal/thread.Thread.Call.

On second thought, making internal/graphiccommand have a thread (internal/thread.Thread) and use it whenever calling the APIs of internal/graphicscommand should be efficient by reducing the number of Thread.Call. It's because internal/shareable's API can be called very often, but internal/graphicscommand's API is not. I'll try this when I'm available.

[jakecoffman]

On Mac with GOMAXPROCS=1 the threaded version is at 30% now, a 20% improvement! The non-threaded is still at 15%.

On Windows it seems to have no effect on the FPS for either version.

[hajimehoshi]

@jakecoffman Could you try eed619ad0f51ed1e95b545da87c087ccb69f99a4? This reduces bunch of (*thread.Thread).Call.

Note that the version is already 2 and you'd need to update your code (https://ebiten.org/documents/to_v2.html)

[jakecoffman]

My updated code is here if you would like to try running it as well. The gh-pages branch is pointed to the single-threaded fork.

The tumble demo is at 1000 FPS on windows so a bit of an improvement but still ~2x slower than the single-threaded one.

On Mac CPU usage is at ~43% so slightly better.

[hajimehoshi]

Thanks! I'll take a look further later.

[hajimehoshi]

My intuition is that we can reduce (*thread.Thread).Call in internal/ui/glfw.

BTW, I'm now working on a bug around vsync initialization first https://github.com/hajimehoshi/ebiten/issues/1364

[hajimehoshi]

On my MacBook Pro 2020, the FPSs between the latest v2 fe6a2daef4507cc7a74b27ddf906198545720a6d and the single thread version were not different. Both are about 600, but unstable.

[jakecoffman]

Ah, that explains why I couldn't disable vsync.

I tested this too on my MBP 2019 and saw similar results: unstable FPS but spiking up to a similar level. I noticed that the v2 version used 100% CPU while the single thread version used 50%.

I did a profile of both and the single thread version spent most time in runtime.cgocall stemming from graphics related activity. The v2 spends most time in runtime.pthread_cond_wait and runtime.pthread_cond_signal which I assume channels use for syncing.

I'm guessing the thread backup is causing an active-wait situation with the main loop causing the 100% CPU usage. Not sure about unstable FPS though.

[jakecoffman]

Windows profiles looks similar, threaded version spending tons of time in code that looks like waiting on a lock (like runtime.semawakeup).

[jakecoffman]

Turns out closing Activity Monitor fixes most of the instability on Mac...

[hajimehoshi]

Thanks!

Ah, that explains why I couldn't disable vsync.

Now this should be fixed. Please try if you are available.

I did a profile of both and the single thread version spent most time in runtime.cgocall stemming from graphics related activity. The v2 spends most time in runtime.pthread_cond_wait and runtime.pthread_cond_signal which I assume channels use for syncing.

Thank you for profiling! As we've already said, using channels seems the bottleneck so far.

Turns out closing Activity Monitor fixes most of the instability on Mac...

Oh, that's good to know...

[hajimehoshi]

Note to myself: How to get the groutine ID: https://github.com/golang/net/blob/master/http2/gotrack.go#L51-L67

EDIT: Finally this is not used in Ebiten.

[hajimehoshi]

I've removed a lot of (*Thread).Call around the update loop.

[jakecoffman]

it got slower on windows? 500fps vs 2300fps

[hajimehoshi]

That's unexpected... What about mac?

[hajimehoshi]

Oh my Windows machine, the FPS was improved from 350 to 450 (v2.0.0-rc.1.0.20201015142520-09a6769fa70 to 974ec525fa9521cbf22bf191b982f79e07ab4a01)

EDIT: With the single mode, the FPS was about 500.

[hajimehoshi]

it got slower on windows? 500fps vs 2300fps

My guess is that you are replacing github.com/hajimehoshi/ebiten instead of github.com/hajimehoshi/ebiten/v2.

[jakecoffman]

Oh I was pulling from the 2.0 branch. Master has improved to ~1700FPS on windows. 🍾

Mac is still odd with the FPS equal but CPU usage high around 99%.

[Zyko0]

Differences are crazy, FPS of my project improved from 10xx to 26xx on Windows, really a good job ! 🚀

[hajimehoshi]

Mac is still odd with the FPS equal but CPU usage high around 99%.

I think higher CPU usage is expected, since when vsync is off, CPU should be used with less sleeping.

[jakecoffman]

I agree the CPU will be higher with vsync off but the same demo single-threaded with vsync off only uses ~60% CPU.

[jakecoffman]

I appreciate your hard work in reducing the calls through Thread.

I wanted to gently float the idea again that making ebiten single-threaded and removing all locks & channels will unleash maximum performance and simplify the code.

My concern is that disabling threading for performance is counterintuitive and doesn't sound a way to go for optimization.

It is the coordination via locks and channels that is currently in need of optimization by removing as many calls to them as possible. Seems intuitive to me to remove them all! 😄

[hajimehoshi]

Unfortunately making Ebiten single-thread breaks the backward compatibility. For example, some functions like SetWindowSize is concurrent safe but the internal process must be on the main thread, then a main thread loop must exist somewhere. There is no way to detect whether the current goroutine is the main thread's goroutine or not. Even if we could detect it and make a function like RunOnMainThread(f func() error) which runs a given function on the main thread synchronously, this doesn't work such case:

RunOnMainThread(func() error {
    ch := make(chan struct{})
    go func() {
        // Do Something
        RunOnMainThread(func() error {
            // This is never executed. Dead lock!
            close(ch)
        })
    }()
    <-ch
    return nil
})

[hajimehoshi]

As we discussed at the Slack channel, introducing a build tag like ebitensinglethread would be acceptable.

[hajimehoshi]

Though mutex is still remaining, but this should be nice to have. Let's close this. File a new issue when we work on removing mutex.