Z88dk: CP/M example ddir.c doesn't work

[nbrok]

The file DDIR.C used in the CP/M example doesn't work:
It is printing the first directory entry and keep this printing in an infinitive loop.
It's situated in directory: examples/osca/ddir.c

[zx70]

I Think it works but its compatibility is very low, it will run on a limited number of cp/m systems
Question: are you testing it on a generic cp/m emulator or on a booted cp/m system (emulated or real)? Which one? .. it is a map I always thought to create.

[feilipu]

Nick originally reported it here, and he reported finding the bug in z80pack, as well as his own z180 Scrumpel7 machine. I tested it on a RC2014 running CP/M-IDE, and found the same issue.

[zx70]

I noticed the problem as soon as I prepared this example.
What seems to happen is that many CP/M systems suffer a conflict on the 'setdma' status while the directory is being read, but I stopped investigating when I discovered that on some system it worked correctly. This is the output with "CPM Player":

C:\>cpm.exe ddir.cpm C*.C
-- Directory of volume #0 --
CAL.C          ........13312
CAT.C          .........2432
CKSUM.C        .........5120
CMP.C          .........2048
COMP.C         .........1536
CUT.C          .........8064
Total bytes: 32512.

..I've been scouting around this topic a number of times and I think that the DMA status gets corrupt by something inside the loop, I suspect the BDOS CFS function used by dir_get_entry_size().

A possible solution is to copy the directory in a buffer first, I never insisted on it because in any case the file size accuracy on CP/M is at block level and I thought there was little interest on it.

[feilipu]

From my point of view, the only interest is to fix a bug in my implementation of CP/M.
If the example should work in "real" CP/M, then my implementation should support it properly.
I can investigate why this it doesn't work. And the hint gives a great place to start looking.
Further hints are very welcome.

I can't speak for Nick...

[zx70]

Well.. the directory listing tools are known to be very machine specific, I wouldn't say it is due to a bug on cp/m. Similarly there is no universal way to save / restore the DMA status :)
This program fails on many real cp/m targets, probably for the way BDOS is checking the file size.

[zx70]

More about BDOS function 35, it looks a delicate one, e.g.

_When computing the size of a file, the DE register pair addresses an FCB in random mode format (bytes r0, r1, and r2 are present). The FCB contains an unambiguous filename that is used in the directory scan. Upon return, the random record bytes contain the virtual file size, (...)

Data can be appended to the end of an existing file by simply calling Function 35 to set the random record position to the end-of-file and then performing a sequence of random writes starting at the preset record address._

What I'm doing is to use a pointer to a portion of the directory buffer as file specifier. I wonder if that buffer is being corrupted somehow, I should have tried already to make it bigger (it is 132 bytes now), but the bug is quite predictable, I'm not sure it is due to a buffer overflow.

[nbrok]

As earlier said: in z80pack, the behavior is the same. It is more likely a bug in general because Scrumpel and z80pack have the same effect. I know that some CP/M versions in those days had some added features that where not common for standard CP/M. This could be the case in this file (ddir.c) too.

[feilipu]

Since my CP/M uses an almost (some z80 specifics used where obviously better) standard BDOS, I'd start looking where the C:\>cpm.exe program does something different to standard. I've tried to find reference to it on the 'net, but it seems obscure.

[zx70]

http://takeda-toshiya.my.coocan.jp/cpm/index.html
I agree with Nick,I think it makes little sense to try to correct your cp/m implementation.. you even risk to break the compatibility in some other way (that's the sense of my report about bdos function 35)

[feilipu]

Yes. I'll leave it be.

The cpm.exe source is there, but it is a full BDOS rewrite in C++.
There's no joy to be had with diff and friends there.

Better things to do.

[zx70]

What surprises me is that it was originally for cp/m already.
It impressed me for being very close to the msdos syntax but having been created before it.
Hope I remember correctly, the original work should be suffering the same limitations.
Your feedback is crucial to consider to enforce the directory functions, starting from scratch on the directory browsing after every entry should suffice.

[feilipu]

For reference, the issue is still evident in my RC2014 test following commit.

[zx70]

yeah, it is expected :(
I also tried to force a reset of the directory scan after every single entry being read but the result gets worse. The feeling is of something breakingbadly (e.g. a buffer overflow).

[zx70]

Varying from target to target I can't be sure I solved it, please repeat the tests and let me know.
During the code revision I think I found another bug in the optimized version of parsefcb (no DEVICES mode), happening only when a user number is put in the filespec; IX+37 needs probably to be changed to IX+41 or, better to be adjusted in order to avoid IX at all and be 8080 compatible ;)

[zx70]

also this needs to be aligned between the DEVICES mode and the asm optimized version:

if ( name != ptr && ptr == ':' ) { / uid has to end in colon */

CP '/' ;make sure number is terminated by a slash

..but keeping in mind that putting the 'colon' in the ASM version collides with the way the drivespec is detected..

INC DE  ;peek at 2nd char of filename
LD  A,(DE)
DEC DE
CP  ':' ;default disk byte value is 0
LD  a,0 ; (for currently logged disk)

[feilipu]

Better, but still not quite right...

B>dir

B: CPSKER   HEX : CPVGEN   HEX : AES256   C   : AES-DEMO COM
B: DEMO     C   : RANDOM   TXT : Z80DAA   COM : MAND180  HEX
B: STARTREK BAS : MANDEL80 HEX : STARTREK COM : MANDEL80 COM
B: MAND180  COM : 1BACK    PT3 : 2SANX    PT3 : 3CHILD   PT3
B: 4DAVOS   PT3 : 13SYNC   PT3 : ym2149-O bas : ym2149-D bas
B: ym2149-D BAS : TEST     COM : PASSWORD COM : ELIZA    COM
B: UMCHESS  COM : LADDER   COM : LADDER   DAT : LADCONF  COM
B: FLOAT    COM : RANDOM1  TXT : FTEST    COM : STAR32   COM
B: STAR32F  COM : 1M       TXT : HELLOWC  COM : HELLOWN  COM
B>a:xmodem ddir.com /r


File created
Receiving via CON with CRCsCCved 59 blocks
B>ddir

-- Directory of volume #1 --
CPSKER.HEX     ........70912
Total bytes: 70912.

B>

[zx70]

Ouch.. ok, I'll think at something more.
(this last 'commit' is not solving this specific problem, so avoid testing again)

[zx70]

Question: what happens if you pass a filespec not involving the first file (e.g. *.com) ?

[feilipu]

continuing...

B>ddir *.com

-- Directory of volume #1 --
Total bytes: 0.

B>ddir *.COM

-- Directory of volume #1 --
Total bytes: 0.

B>ddir *

-- Directory of volume #1 --
Total bytes: 0.

B>ddir

-- Directory of volume #1 --
CPSKER.HEX     ........70912
Total bytes: 70912.

B>

[zx70]

I might have found something:

SETDMA (function 12)
The next disc operation will read its data from (or write its data to) the address given in BC.

[zx70]

Even worse !
bdos(CPM_SDMA,fc_dirbuf); -> bios(CPM_SDMA,fc_dirbuf,0);

I'm reverting the workaround, let's try to keep things as clean as possible.

[zx70]

nope, I probably need to rest a bit :)

BDOS function 26 (F_DMAOFF) - Set DMA address
Supported by: All versions
Entered with C=1Ah, DE=address.

[zx70]

Possible test: this SETDMA function may be different in BIOS and BDOS.
Changing line 22 in 'fcntl/cpm/dir_move_first.asm' from BDOS to:

bios(11,fc_dirbuf,0);       // BIOS FN #11, Set DMA address

..should be the same thing but makes cpmplayer react differently.

[feilipu]

It isn't helping. But, it does react differently.
Which is surprising, since the code is very straightforward on the CP/M BDOS side.

> cpm sys.cpm user.cpm
Opening "sys.cpm" at LBA 104146
Opening "user.cpm" at LBA 136914
Initialised CP/M

A>b:

B>a:xmodem ddir.com /r

File created
Receiving via CON with CRCsCCC
B>ddir

-- Directory of volume #1 --
               ............0
Total bytes: 0.

B>ddir *.txt

-- Directory of volume #1 --
Total bytes: 0.

B>ddir *.com

-- Directory of volume #1 --
Total bytes: 0.

B>ddir *.*

-- Directory of volume #1 --
               ............0
Total bytes: 0.

B>ddir

-- Directory of volume #1 --
               ............0
Total bytes: 0.

B>

For reference, the CP/M implementation looks pretty simple.

;
;   Function to set the dma address.
;
PUTDMA:
    EX    DE,HL
    LD    (USERDMA),HL    ;save in our space and then get to
    JP    DEFDMA        ;the bios with this also.

then later...

;
;   Do a write to the directory of the current disk.
;
DIRWRITE:
    CALL    SETDIR        ;set checksum byte.
    CALL    DIRDMA        ;set directory dma address.
    LD    C,1        ;tell the bios to actually write.
    CALL    DOWRITE        ;then do the write.
    JP    DEFDMA
;
;   Read from the directory.
;
DIRREAD:
    CALL    DIRDMA        ;set the directory dma address.
    CALL    DOREAD        ;and read it.
;
;   Routine to set the dma address to the users choice.
;
DEFDMA:
    LD    HL,USERDMA    ;reset the default dma address and return.
    JP    DIRDMA1
;
;   Routine to set the dma address for directory work.
;
DIRDMA:
    LD    HL,DIRBUF
;
;   Set the dma address. On entry, (HL) points to
;   word containing the desired dma address.
;
DIRDMA1:
    LD    C,(HL)
    INC    HL
    LD    B,(HL)        ;setup (BC) and go to the bios to set it.
    JP    SETDMA        ;this is BIOS Fn 11.
;
;   Move the directory buffer into user's dma space.
;
MOVEDIR:
    LD    HL,(DIRBUF)   ;buffer is located here, and
    LD    DE,(USERDMA)  ;put it here.
    LD    BC,128        ;this is its length.
    LDIR                ;move it now and return.
    RET

[zx70]

Thankyou, it is interesting.
Don't overestimate this last experiment, though. I can have done something wrong.

[zx70]

It can be a beginner's error!
Could you please try changing in dir_move_first and dir_move_next and remove the ampersand everywhere?
I'd leave the stabilization trick in, other programs may need it.

[feilipu]

1am. :sleepy: Tomorrow.

[zx70]

Thanks for the lots of support. I checked it myself and it looks worse (WC.c fails).. nevertheless I'd like a second opinion

[zx70]

I Think I found a clue.
BDOS functions 17 and 18 pass successful return codes which can be between 0 and 3, but I was assuming 0.
in ddir.c a possible workaround could be to change:
while (x == 0) {
in:
while (x !=255) {

if it works, then we have the fix for the library. .

[feilipu]

Sorry. No :game_die: :game_die:

B>a:xmodem ddir.com /r
File exists. Overwrite (Y/N)?y
File created
Receiving via CON with CRCsCCC
B>ddir

-- Directory of volume #1 --
Total bytes: 0.

B>

But I have a counter clue...
https://www.seasip.info/Cpm/bdos.html
BDOS function 18 - CPM_FNXT - search for next
Entered with C=12h, (DE=address of FCB)?. Returns error codes in BA and HL.
This function should only be executed immediately after function 17 or another invocation of function 18. No other disc access functions should have been used.

[zx70]

that one should be solved already (it was my initial guess), but now you are listing only one element.
thank you for having it confirmed, though

[zx70]

I think it's better to warn I'm on holidays, I'll do it in fragments but for the next week I won't be able to test a thing.
My suggestion was incomplete you should also change:

if ((x=dir_move_first())!=0) return(x);

but guessing without trying myself is .. stupid.

I'll do some more experiment when possible and will get back to you, sorry for this last miss :(

[zx70]

That hint couldn't work because of the way we deal with the BDOS return codes.
Hope it is the only 'bug' left

[zx70]

I really hope I now found all the problem preventing the directory functions from working on all (most?) the CP/M machines. Wishing to get the cleanest code I was tempted by wrong assumptions :)
The updated one is not a beauty but should work better.

[feilipu]

> cpm sys.cpm user.cpm
Opening "sys.cpm" at LBA 104146
Opening "user.cpm" at LBA 136914
Initialised CP/M

A>b:
B>a:xmodem ddir.com /r
File created
Receiving via CON with CRCsCCC
B>
B>ddir

-- Directory of volume #1 --
CPSKER.HEX     ........70912
CPVGEN.HEX     .........2816
AES256.C       ........13056
AES-DEMO.COM   .........9216
DEMO.C         .........2304
RANDOM.TXT     ..........256
Z80DAA.COM     .........1792
MAND180.HEX    .........6016
STARTREK.BAS   ........20992
MANDEL80.HEX   .........6400
STARTREK.COM   ........39936
MANDEL80.COM   .........2304
MAND180.COM    .........2176
1BACK.PT3      .........1280
2SANX.PT3      ........12544
3CHILD.PT3     .........2560
4DAVOS.PT3     .........5248
13SYNC.PT3     ........11264
ym2149-O.bas   ..........256
ym2149-D.bas   ..........384
ym2149-D.BAS   ..........384
TEST.COM       .........3456
PASSWORD.COM   .........3456
ELIZA.COM      .........8192
UMCHESS.COM    .........7936
LADDER.COM     ........39936
LADDER.DAT     ..........512
LADCONF.COM    ........22528
FLOAT.COM      .........4608
RANDOM1.TXT    ......1048576
FTEST.COM      ........10496
STAR32.COM     ........39552
STAR32F.COM    ........40192
1M.TXT         ......1048576
HELLOWC.COM    .........6272
HELLOWN.COM    .........3456
DDIR.COM       .........7552
Total bytes: 2507392.

B>ddir *.com

-- Directory of volume #1 --
AES-DEMO.COM   .........9216
Z80DAA.COM     .........1792
STARTREK.COM   ........39936
MANDEL80.COM   .........2304
MAND180.COM    .........2176
TEST.COM       .........3456
PASSWORD.COM   .........3456
ELIZA.COM      .........8192
UMCHESS.COM    .........7936
LADDER.COM     ........39936
LADCONF.COM    ........22528
FLOAT.COM      .........4608
FTEST.COM      ........10496
STAR32.COM     ........39552
STAR32F.COM    ........40192
HELLOWC.COM    .........6272
HELLOWN.COM    .........3456
DDIR.COM       .........7552
Total bytes: 253056.

B>

Sure looks right to me. :smiley:

[zx70]

Wonderful !
It doesn't matter much in this particular case, but you can even redirect the output to a file, e.g. " ddir *.c* >file.txt"

I have few sources of similar tools which I'm arranging just for fun, if you think they can be of any use in the CP/M world, I can send the results (sources and compiled code).

[nbrok]

Where can I find the corrected ddir.c?

[feilipu]

You have to pull the latest changes to the z88dk libraries.
And then rebuild z88dk and make ddir.com again.

No change to ddir.c code at all. Just how the fcntl/cpm library works.

__EDIT__. Just to note, tested on the YAZ180 CP/M too.
Another very vanilla CP/M implementation, on a z180 processor.
It works as advertised. :tada:

[zx70]

Let me point out that the "nightly builds" are not yet ready, you need to wait 'till the.. next morning (roughly, it depends on your time zone !)

[nbrok]

I have managed to download the latest z88dk and did insert the two mods of today. Now the output is as expected. DDIR gives a list of all files in the current drive as expected. But one issue is stlll left: ddir driveletter:*.* doesn't work. It can only list the directory on the current drive.

[zx70]

Have you tried the wildcards without specifying a drive letter?

[nbrok]

Without specifying drive letter, wildcards works.

[zx70]

This is expected. DDIR is designed to list the content of the current volume.
The output redirection could work on a different drive and even user but it should be checked.
Another interesting thing to do is to build ddir with support for devices and test redirection to LST:
The drivespec could be added, but it is different on OSCA (numbers) and cp/m (letters).. this tool was meant to be universal

[zx70]

To add support for devices (e.g. output redirection on a printer) is should suffice to add -lcpmdevice.