Cxbx-reloaded: Milestone: v0.2 planning

[RadWolfie]

@Cxbx-Reloaded/developers You have permission to edit OP post for any updates or take assignment.
Contributors are welcome to send pull request to origin progress branch to get the job done quicker. 😉

Here's the plan for future v0.2 release.

• [ ] Improving our HLE kernel [LukeUsher]
• [x] Use open source API to move registry data into ini file. [RadWolfie]
  
  
  
  • See issue #1312
  
  
• [ ] HLE DirectSound, audio, improvements [RadWolfie]
  
  
  
  • See issue #1132 (Must be done first before proceed fix remaining bugs)
  
  
• [x] Create list of patches than search in symbol address list. [RadWolfie]
  
  
  
  • See progress in #1416 task
  
  
  • Then we do not need LLE flags in HLE cache file. (We will need to rename it to Symbol cache or the like.)
  
  
  • Plus start removing all FUNC_EXPORTS (except for kernel APIs?).
  
  
• [ ] HLE Selection renderer/audio support. (all feature may not be complete at release as they are more of a proof of concept what could be done in HLE)
  
  
  
  • [x] [G] DirectX9 feature [Graphic devs]
  
  
• [x] Log filtering support [ergo720]
  
  
  
  • Useful to only filter specific library sections to a log file plus some other miscs.
  
  
  • Log specific level, such as debug, info, warning, errors.
  
  
• [x] GUI log indication [RadWolfie]
• [x] CMake support (#1566)

What other tasks we should do?

Postpone:

• LLE implementation of Xbox hardware [v1.0 milestone]
  
  
  
  • [ ] CPU Emulation - (we will probably use the library lib86cpu for this.)
  
  
  • [ ] LLE GPU improvements [Graphic devs] - (Postpone, require virtualization support.)
  
  
  • [x] LLE USB feature [ergo720] - (Postpone, see https://github.com/Cxbx-Reloaded/Cxbx-Reloaded/issues/1318#issuecomment-479930558.)
  
  
  • [ ] LLE APU feature [Audio devs] - (Postponed due to complex coding requirement and most likely need virtualization support.)
  
  
• GUI v2.0 [v1.0 milestone] (by using Dear ImGui to support cross-platform)
  
  
  
  • Will be working with several various contributors.
  
  
  • See issue #677
  
  
  • HLE OpenGL feature
  
  
• Loading and running a REAL Xbox kernel/bios [LukeUsher]
• HLE OpenAL Soft feature
  
  
  
  • See issue #807
  
  

Scrap from plan:

• Split kernel and GUI codes into 2 executable (again)
  
  
  
  • By doing this method will allow kernel to load all settings without GUI set it up.
  
  
  • Once separation is done, we can finally start switch to GUI v2.0 development
  
  

[LukeUsher]

Split kernel and GUI into two executables is wrong :P What will most likely end up happening is the GUI and Emulator will remain one executable, but there won't be any process relaunching. The virtual xbox will be able to start/stop/pause at any time! (With CPU emulation & LLE we could even support save states, etc).

This will also mean multi-xbe titles won't cause weird issues: We'll just reboot the emulated Xbox with no process relaunch

CPU emulation also means a debugger can be part of the main emulator instead of a separate program: And with LLE it will be able to peek/poke hardware registers/state, as well as step through code execution one instruction at a time, view all registers, etc. The current debugger is pretty limited in this regard.

[LukeUsher]

Oh another thing:

We NEED a CMake build system, Cxbx-Reloaded should be able to be compiled with at least MSVC and GCC.

A single CMake project can generate MSVC solutions and GCC makefiles, alternatively, we could maintain the MSVC solution and a Makefile manually. Either way, we need to take steps towards working cross platform.

From this point forward, Windows specific code will not be allowed either, unless it is a bug fix or minor tweak to existing Windows specific code.

The effort to being cross-platform starts now.

HLE code can remain Windows specific for now but it will be migrated to cross platform over time. Even if we keep the existing DSOUND/D3D HLE, there must be cross platform alternatives made available.

LLE must be completely cross platform going forward, no dependencies on any specific operating system

[RadWolfie]

True, we do need a CMake build system. Though, I don't think we can completely remove Windows' functions before v0.2 release. Since it will not be able to build other platform within 6 months yet. Might be possible in v0.3 release plan.

It's my prediction theory, though. If we do get a lot of work done quickly, then sure.

EDIT:

Even if we keep the existing DSOUND/D3D HLE, there must be cross platform alternatives made available.

Updated the list for selection options in HLE.

[ghost]

Just a suggestion: if a release implements all the features of above, you could very well call it version 1.0 in my opinion

[LukeUsher]

That may well happen, v0.2 is just a tentative release version number: A more suitable verison number will be chosen closer to completion.

[RadWolfie]

@ergo720 True. Though for 6 months, not all of us will have tons of free time. So, it's possible only half or 3/4 of the planning will be ready in 6 months. It's a lot of stuff to work on.

Also, I consider full ability to support cross-platform as 1.0 with complete features support mainly LLE stuff. 😉

P.S. Sorry for my late input.

As for having CMAKE generate MSVC solution and GCC makefiles sound pretty good to maintenance projects/makefiles without need to manually update both sides.

[ghost]

Another question: from the above features I see that OpenAL is being mentioned, so has that proposal been resurrected? According to #807, I thought it was decided not to implement it. I'm just asking by the way, I have nothing against it.

[RadWolfie]

Is HLE DirectSound implement cross-platform compatible? Answer: No, OpenAL do. Beside, both options will be provide as proof of concept. DSP binary will not be support in HLE environment. Except environment sound effects will be supportive in OpenAL-soft.

To emulate audio better is to use LLE for support DSP binary. It will be pain in the butt. We'll get there eventually.

Although for LLE audio emulation, we might find a better solution than to use OpenAL which only need to stream direct to audio hardware (with decoder requirement for XADPCM).

Is the comment clear enough for you to understand @ergo720?

[x1nixmzeng]

@LukeUsher I have a question about the state of the debugger for this next milestone

At the moment it handles the Cxbx process through the Windows debugging API. Depending how the CPU is to be emulated a GDB stub could be written instead to expose Xbox memory - allowing other third-party debuggers - such as the C# version - which could be updated. The alternative is to stip it out completely.

What are your ideas?

[LukeUsher]

With the real Xbox kernel, you could attach directly to the Kernel debugger via en emulated serial port, or XBDM over the network. You could connect IDA Pro to this directly, or even use official XDK tools if you have them, to debug software running under Cxbx-R.

[LukeUsher]

Currently thinking I want a working/complete DirectX9 port for version 0.2 I also want it to be the default option.

The benefits for HLE are incredible:

1. Better driver support/more stability
2. Support for more texture and vertex data formats
3. The biggest one: Support for shader models > 1.1 allowing for complex vertex and pixel shaders that are impossible to convert for DX8 to be converted accurately.

This has potential to dramatically improve compatibility and rendering accuracy: Specifically games with missing effects, T-pose models, corrupted vertices, etc will likely improve dramatically.

CPU emulation is still happening, but it's a big project, likely too much for v0.1 but it's a bigger project, more work, and likely reason enough to do a major 1.0.

I suggest scrapping CPU emulation, GUI 2.0 off this list: Because they are more suited for a major release such as 1.0 than minor releases, and they will likely take several months to complete, i'd rather not have to delay 0.2 if everything is complete except for these tasks.

[RadWolfie]

I agree. I made an update to the OP to reflect your suggestion.

Any time a feature/improvement is postpone from milestone, move them into postpone section. Or if really going to scrap anything from current plan and the future. Put them in scrap section.

P.S. LLE APU will be taking awhile. I'm thinking about postpone HLE OpenAL-soft to v0.3 plan or later.

[CakeLancelot]

So would v1.0 be the more likely target for cross-platform/Cmake support or is it still an item on the list for v0.2?

[LukeUsher]

I'd say potentially further than that: cmake support could come in 1.0 but being fully cross platform requires the entirety of the emulator to be rebuilt, which is a long, gradual process.

Pretty soon (as in, within another release or two) there will be introduced a rule that windows specific code will no longer be accepted though: Any new development needs to be cross platform (eg: using std libraries instead of winapi)

[RadWolfie]

Correct, GUI system is still fixed for windows platform. Not all of std libraries has everything we need. There is more work require to bring cross-platform on the table.

[LukeUsher]

One thing I'd like to add to this: A vertex buffer cache is required for v0.2. It will improve performance in titles that use types that need conversion (quads, etc). This should be an enormous performance improvement for Metal Slug series, buffy the vampire slayer, and more.

[WSLUser]

I'd like to chime in and say some of the tools you'd like to use including debugging would probably be handled without wasting time porting to Windows by using WSL on Windows 10. You could use gdb easily then for example, not to mention straces. Plus you mention being x-plat. While GUI apps aren't officially supported by WSL, it is very much possible to run GUI on it. In fact, Wlinux would probably be your best bet of distro for automating most of what you'd need on the distro. Testing out games via WSL probably won't work but when you want to check if Linux could support, you could grab a Ubuntu 18.04 image from the Quick Create Gallery in Hyper-V to use for testing Linux. (There will likely be some tuning needed for rpm based distros, but we also have WSLFedoraRemix to help with that). Also, there's a ton more API surface (with additional features, bug fixes, etc.) and OSS components for Win 10 including the desktop (finally, this was the most recent OSS to come out from MS).

[RadWolfie]

Added CMake to the task list. I'm sure the loader project, which had been worked on by @PatrickvL, will be a success.

[LukeUsher]

I think we should remove the remove LLE GPU task: since we're reducing the amount of work going into XQEMU code at the moment, until we have a good way/process in-place to make sure our fixes make it upstream.

[ergo720]

@RadWolfie I found an issue in LLE USB which probably requires a proper thread scheduler to be solved, which won't happen in the near future, so you can remove that as well.

[AzurikRiseOfPerathia]

One should also add black screen resolution / black frame. because a lot of game are touching by getting problem

[LukeUsher]

@AzurikRiseOfPerathia that's not one problem to solve: It's 10s or Hundreds of problems with a different cause in different games, there is no 'fix everything' solution.

[AzurikRiseOfPerathia]

@LukeUsher I understand :)

[SkyHighGam3r]

It's fascinating to be able to read though these posts/threads, and I'm not one to comment usually. I have to say though, even though it's in the "postpone" section, I am absolutely THRILLED to see "_Loading and running a REAL Xbox kernel/bios_".

I'm consistently baffled by how grossly undervalued running a real BIOS is with people who use emulation. This is especially true for folks who purport to value emulation as a tool for preservation of the medium. Plus, if absolutely nothing else, the inevitable and subsequent support of the original boot sequence (of any console) is so 'key' to the experience of the hardware.

I'll be joining the Patreon as soon as I get paid again. Please keep up the great work everyone - you probably don't hear it enough, but Thank You for all you do.

[Hedda]

"HLE OpenGL feature" can maybe be replaced with "HLE Vulkan feature" if you've not already started?

As I understand it, OpenGL is really an outdated API compared to the Vulkan API, this as OpenGL does not support as many features as the newer Vulkan does, (newer Vulkan meaning Vulkan 1.1 and later).

I guess the main reason for wanting OpenGL support today would be to be able to run the emulator on older GPU hardware that does not fully support Vulkan. Vulkan is already supported on newer systems that support the very best GL implementation, making OpenGL rendering mostly moot by today standards.

Vulkan API also happens to be a popular choice among later game console emulators, though more among low-level emulators, renderers, and shaders. Now latest I read about work on the ParaLLEl-RDP (N64 RDP) emulator:

• https://www.libretro.com/index.php/reviving-and-rewriting-parallel-rdp-fast-and-accurate-low-level-n64-rdp-emulation/
• https://www.libretro.com/index.php/parallel-rdp-rewritten-from-scratch-available-in-parallel-n64-right-now-for-retroarch/
• https://github.com/Themaister/parallel-rdp

Nice to see a public milestone/roadmap planing by the way!

[ObiKKa]

Not so fast. OpenGL has a thing or set of features called "Compute Shaders" that is available since OpenGL 4.3. It may be used in the libretro and independent builds of the Dreamcast emulator, Flycast; and N64’s parallel (OpenGL version/backend, not the Vulkan backend).
It may be good enough for this Xbox emulator also.

It was implemented before or up to early May 2020 in this year in the libretro branch of the Saturn emulator, Kronos.

However, the OpenGL code of libretro Kronos was soon later rebased on Yabause (May 27, 2020) since the compute shaders in that other core has better rendering (The old OpenGL rework created issues, according to the libretro devs).

The new feature makes some complicated rendering cases (Like the VDP1 chip in Saturn) far more accurate and true to hardware specifications than older OpenGL could manage. The only known downside of this new renderer is that it will require a fairly good GPU!