The way groups/group lists/blocks are displayed only works when they're displayed in a regular form. When you open one inside a global form, it gets rendered in the sidebar instead of inside the global form.
Add a global form that contains a group, or use useGlobalJsonForm on an existing json form that contains a group.
I'd expect the panel to open inside the modal.
What happened.

I鈥檝e just experienced this myself. It would be great if someone can fix it!
A fix is coming soon @timonweber. @spbyrne is doing a bit of work that will make a fix possible.
My previous attempt at fixing this used refs to measure the appropriate spot to render the panel. It ended up being overly complicated and prone to failure.
Nolan has set up a form portal that can be used to move any sub component up to the root form level, which will make this a lot easier. I'm currently working on this and should have it ready today.