Z88dk: Implementing far memory

[obiwanjacobi]

More than two and a half years later I revisit the topic of far memory.

Now that I have basic software control of my MMU in place and working, I thought I would try my hand on implementing some sort of extended memory implementation.

The lp_* functions mentioned here (is this still up to date?) are what I am planning to implement. I think I have a reasonable understanding of how this is supposed to work, though there are some questions and I'm sure there will be more.

For working with data (structs, arrays strings) I can see how that is handled by the lp_x interface. There is even support for calling a mapped function. What I am unsure of is how one obtains the far pointer to begin with. I concluded so far that I will have to make a conversion method to make a far pointer from a regular one (I call it regular, because I also have a relative ptr - oh, great 3 ptr types! ;-). But to do that I will have to know where (in what 4k pages) things are located. Details on this are still in the dark for me...

Another question I have to answer is how I am going to load binaries into different banks. I think I will opt for generating multiple bin files using (memory map) sections and -if I remember correctly- orgs. But this can wait until I have far data working (implies lots of new code already).

Of the ehl registers that contains the far ptr, I will use the hl pair as a regular cpu (64k) memory ptr. From that ptr I can deduce what 4k page the code (or data) was targeted to. That current 4k page will be swapped out and replaced with the one that is indicated in the e register. The e register can exactly contain (what I call) a PageId that uniquely identifies one of the 4k pages in the 1mb total address space.
I will have to check not to swap out the 4k page I am currently executing code on. Is there a way to signal an error from the lp_x functions? Set the carry flag? Ideally this type of error is detected at 'compile' (or link or bin-building) time...

More info on my Memory Management system can be found here.

Please let me know if I am doing this wrong, because I have noticed that the docs are not always up to date to the latest changes and your knowledge about this specific topic is what I am looking for.

[suborb]

Nothing has changed with the far pointers in the past couple of years apart from the addition of far function pointers and the trampolined SHORTCALLs.

@feilipu is, I believe, supporting more than 64k in his project so he might have some acquired knowledge/thoughts

[feilipu]

I've only implemented a scheme to manage banks of memory in 64kB chunks for a Z180, rather than a more general solution. The Z180 memory banking scheme is pretty well documented, so I'll only refer to how I've set it up in my case rather than the more general solution.

Supporting 64kB chunks means that there are substantial simplifications in calculating the MMU address within the Z180 world. Basically only the BBR Register needs to be touched in this case, so bank swapping can be quite a few cycles faster.

Since I'm using a banked memory system the EHL registers are used a little bit special. The E register (upper byte) is a twos complement number (rather than being part of a 24 bit unsigned number). This means that applications can call a bank "above" or a bank "below" their current bank without any knowledge of where they are loaded in RAM.

As part of the logical solution for memory mapping, I've implemented bios calls that help applications to call functions in arbitrary memory _call_far(), and to copy arbitrary memory in the 1MB address space _memcpy_far(). Setting banks of memory _memset_far() was written then bumped out a few commits ago, due to space constraints in the Common Area.

RST short calls are used to drive the _call_far functionality, as well as a few other necessary system calls. The code is all present in the YAZ180 target in z88dk. But to make it easier for me to maintain the sections and code size, it is duplicated into a single file within the YAZ180 repository.

From my point of view the lp_* functions can be implemented as wrappers on the _memcpy_far() system function for the YAZ180 platform. It is unavoidably expensive to use the DMA hardware to copy one or two bytes. But, I'd contend that is would not be much cheaper to do the bank switching required to bring the required bytes into the right memory context.

Where am I at with this far memory support now? Well I've got all the pieces I need in place. I can write applications that call arbitrary functions anywhere in memory using short calls, and I can safely copy memory (transparently l->r & r->l) from anywhere to anywhere.

I've used these functions to implement a shell for loading applications into arbitrary banks, and a CP/M system that can be loaded from flash memory into any bank of RAM memory and uses the FATFS drives. Currently I'm working on a MP/M system that will use 7 banks of memory also using FATFS drives. Not much work has has happened on this in the past few months, but I hope to get back to MP/M and complete it shortly.

[obiwanjacobi]

Not sure what SHORTCALLs are (I guess you mean RST calls, but is this part of z88dk?) but I have got a beginning (untested yet).
I am finding writing assembly a bit of a struggle. Seems like every time I re-read a piece of asm code I wrote I find something wrong with it.
I also deal with a hardware glitch in the MMU I need to fix first, that is why the software hasn't progressed much. Almost there though, I think.

Not sure about the use of rst defb short calls yet. Seems like the compactness at the caller site is does not weigh against unraveling the stack in the function itself. I am not shooting for most compact code, I am shooting for fasted code.

Thanks for your story, feilipu. I do not have bin or hex loaders yet and certainly no drives or file systems. I am still at the start of this journey and it is certainly good to know there is someone here with some actual experience.

Keep you posted.

[obiwanjacobi]

I think I have a first working test.

The test uses the MemoryManager API to manually switch (push) page 20 into memory (at 0x8000) and write a byte (42) into that page. Then the memory layout is restored (pop) and the farptr to that byte is returned from a function and later on dereferenced and 42 is printed.

Next is writing data - I need another setup method for dealing with the exx regs etc...

Edit: just made sure and could see the lp_gchar call in the list asm.

[feilipu]

BTW, I only mention it because you mention wanting the fastest code, so doing a call my_function then ret in z80 assembly is unnecessary.

Just do jp my_function on the last function in the routine. You'll save 7 clocks using the jp (jp is 10, rather than 17 clocks), and then you'll save another 10 clocks on the redundant ret. That will soon add up.

:nerd_face:

[obiwanjacobi]

Yes I knew that trick and use in my C thunks when there is no post processing.

There are so many things to think about when writing asm, that I tend to focus on functionality. Also I usually do see the best solution at first sitting. I regularly revisit asm code I've written to see if I think differently now ;-)

Thanks anyway, I am interested in every trick in the book.

[feilipu]

Ok so far?