Z88dk: Support for PRL, page relocatable executable for MP/M

[feilipu]

As part of a project to do a MP/M port I've found that it would be really useful to be able to handle the Microsoft REL format files.

There are some notes on the old wiki about the rel2z80 tool. Another interesting solution seems to be the Misosys UNREL program.

Question 1: has the z88dk rel2z80 tool been maintained recently such that it can still produce valid z80asm source files?

Rather than going back to source, it would be interesting to use original REL files as an input for the linker.

Question 2: Does / could the z88dk linker accept REL files as input, if the global symbols can be found within the REL?

[suborb]

It looks like rel2z80 outputs a very old .obj file version.

I think handling .rel files is on @pauloscustodio list of features for z80asm

[pauloscustodio]

Yes, it's in the backlog

[feilipu]

Ok thanks.
Happy to see z80asm being able to generate REL files at some stage. :+1:

On the linker question, I think in the short term that's probably more important. The REL files already exist for much of the MP/M operating system, and the source doesn't, so having an option to be able to link in the REL files seems pretty important.

Or, am I missing something?

[aralbrec]

It's probably a job for zobjcopy to convert between the two where possible.

[pauloscustodio]

As far as I could read in https://www.seasip.info/Cpm/rel.html, the rel format only supports three sections (program, data and common), and names are limited to 8 characters, so having z80asm generate rel files would restrict it too much.

I agree with @aralbrec, having zobjcopy transform the rel files to the current object file format is a better alternative.

[zx70]

I agree with @feilipu on the fact that the main interest would be to load external objects in the linking phase rather than generating them. shouldn't this make it possible to change z80asm, if preferred?
in any case I have no objections if we choose zobjcopy, especially if it is already able to output in z80asm format.

[aralbrec]

It'll just complicate z80asm if it has to support several library formats imo. The purpose of zobjcopy is to modify the object files, including converting older z80asm lib files to the latest version. I think expanding it in the direction of converting other formats to z80asm fits and rel might not be the only one we'd like to treat that way.

[aralbrec]

FB now has me trained to press ctrl+enter for next line :-1: so this didn't get added, sorry.

The other problem is how to map the three or four sections that rel format supports to the section names used in classic and newlib or defined by the user. zobjcopy can again do this, renaming the rel sections to whatever is needed.

[feilipu]

This MP/M project "onion" has a lot of layers. And, I really don't want to peel them all in one go.

I've found some useful sources for MP/M now 👍 , but of course they are in PL/M language 😢. Which means a diversion (another onion layer) into learning PL/M, or at least learning to compile PL/M and having a working PL/M compiler.

Avoiding PL/M and starting at the MP/M REL files and linking them using LINK-80 seems easier, and would work if z88dk could generate REL (through whichever tools needed) for the hardware specific XBIOS code.

Perhaps the most expedient option is just to use M-80 or RMAC to assemble the XBIOS, and "follow the book" of the MP/M System Implementor's Guide ❓

I guess the answer just depends on whether there is a "community value" in having z88dk being able to produce or consume REL files generally, and that I can't answer.

[aralbrec]

Going from z80asm to rel is harder because z80asm has no limit on number of sections (in newlib I think there might be 50). So you'd have to manually change all their names to map into the four or so sections that rel supports or maybe have a convention or algorithm that tried to do the mapping manually. Eg, the section names in newlib follow code_*, data_*, rodata_*, bss_* conventions.

Then there may be a mismatch in features. Maybe rel cannot patch the same things z80asm can so it's not possible to translate some z80asm object to rel. The reverse can be true too but at least we can do something about it.

We can try to put both directions into zobjcopy. I thought it would be preferable to work on the pc though by going rel->z80asm.

[feilipu]

Perhaps I can describe the MP/M use case, as I understand it today, to help with identifying the need.
MP/M comes as a number of REL files that need to be linked into specific origins, and then linked to the custom BIOS and XIOS assembly developed for the specific platform, to create the final system code.

Unlike CP/M 2.2, with one CCP and one BDOS origin, MP/M has a number of transient programs that are loaded below the BDOS as needed, and also it has a banked BDOS which comes as several pieces, that need to be linked into the right origins in the right banks, and a non-banked BDOS as another alternative.

I'm still learning about how much the MP/M system REL files are modified during the linking process. Perhaps just a jump table is completed, or perhaps all the jp xxxx instructions in the REL files need to be completed. To be determined.

That's why I'm still unsure of which option is the least amount of work in total.

1. Convert BIOS & XIOS z80asm into an object (REL) file that can be linked by LINK-80, or
2. Convert the MP/M system REL files into object files, that can be linked by z88dk linker, or
3. Convert existing BIOS in z80asm source format into M-80 source format.

I will do some more reading and experimenting, and then add further notes as I go.

[feilipu]

I've done some research into the process to get MP/M running from an existing CP/M system, and I'll try to unpack some of that reading here. But, the TL;DR is that there seems to be little value (high complexity / low return) in supporting generation of .REL or its cousins for MP/M development.

That said, there may be reasons to pursue support for .PRL page relocatable programs in appmake, for example, where this could be relevant also to the ZXNext and other platforms needing relocatable binaries. I'll edit the issue Title to reflect this refocus.

IMHO:

Assembler

The assembler needs to understand both 8080 Intel (for original DRI source) and Z80 Zilog (for the additional Z80 instruction) mnemonics. It needs to be able to produce both .REL for the linker and .SYM files for the symbolic debugger.

_Alternatives_
DRI RMAC - no Z80 mnemonics, crippled Z80 macro library only.
Microsoft M80 - no easy .SYM generation capability.
SLR Z80ASM - no 8080 mnemonics, and with odd command syntax, but feature rich.
SLR SLR180 - 64180 / z180 mnemonics, and with odd command syntax, but feature rich.

Linker

The linker needs to be able to produce .PRL page relocatable program executable files, as well as .SPR system page relocatable files for the system generation, and .RSP resident system procedure files.

_Alternatives_
DRI LINK-80 - supports what is needed, but is really slow.
Microsoft L80 + double assembling + patching Intel HEX + GENMOD - ugh... nope.

System Generation

The system generator needs to build a MPM.SYS file from multiple .SPR, .RSP, and other data and table files to enable the MPMLDR to load the operating system, and then initialise it.

_Alternatives_
DRI SYSGEN - the only tool available.
Patching the files by hand - not for beginners.

Debugger

The debugger should be able to understand symbols, insert breakpoints, and produce output in Zilog Z80 mnemonics, because that's what I read preferably.

_Alternatives_
DRI DDT - 8080 Intel only, doesn't do symbolic debugging
DRI SID - 8080 Intel only, but is symbolic
DRI ZSID - Z80 version of SID, that can read .SYM symbol tables.

What about these file types?

Out of the assembled Microsoft .REL files the linker has to generate a .PRL format executable for MP/M. The .PRL format is essentially a .COM file with some additional information to enable the program and its data to be relocated onto any page.

_What does a .PRL file look like?_ The first bytes are size of the program, followed by the program origin at 0x0100. Following the program, there is a bit-for-byte mask appended to allow the MP/M system to know which bytes in the program need to be changed when the program is relocated.

_How does the linker do that without disassembling the whole application?_ In advance the program is linked for two different origins 0x0100 and 0x0200, from the .REL objects. The linker trick is simply recognising which bytes in the two versions of the executable differ. These bytes are then recorded in the bit mask stored following the executable, and the final .PRL program is designed to run from 0x0100 plus its page offset.

The same trick is done for the .RSP and .SPR executable files, except that both these formats forego the offset, and run from 0x0000 plus their page offset. How they differentiate from each other, I'm still not sure.

[zx70]

werem't those files also supported by the gsx graphics engine by digital?

[feilipu]

This is related to #243, and possibly it is a "better" path to support the PRL page relocatable binary format rather than the REL format?

[zx70]

Recent existing linker written in C !

ld80 is a replacement of the good old M$ L80 linker for CP/M and other 8 bit operation systems on Z80/i8080 processor. ld80 incorporates almost all capabilities of almost L80 and has extra features. It is intended as part of your Z80 cross development toolbox.
ld80 accepts object (.REL) files produced with M80 or compatible assembler as well as library (.LIB) files made by LIB80.

ftp://gatling.ikk.sztaki.hu/pub/z80/

Slightly reworked version:
http://48k.ca/ld80.html

[zx70]

• ancient article, including the asm source of a simple linker:
  http://www.znode51.de/articles/tcj/hal/linkprl.txt