CPCWiki forum

General Category => NC100, NC200, PCW, PDA600 - the rest of the Family! => Topic started by: JonB on 12:02, 15 January 17

Title: Can't find the directory track on CPM Plus
Post by: JonB on 12:02, 15 January 17
Hi


Where should I expect to find the directory track on a CP/M disk? In CP/M 2.2 it is the first track immediately after the system tracks (on a floppy it's track 2 - third track). I can't find it on a CP/M Plus disk.


The issue is with the IDE driver I have written. It seems to work, except when I copy a file to the IDE device, it appears four times in the directory. So I wanted to read the sectors of the directory track and see what the BDOS had written. Only thing is, I can't find it. I wrote a small BASIC program to dump the content of any sector, but entering the expected sector number results in a non populated directory track.


My IDE DPB has 1 reserved track, so the first data track is track 1.


When reading a non executable file such as a text file, it works fine, despite there being four directory entries for the file. However, attempting to load a .COM file (executable) results in a system lock up. When writing a text file to the IDE device, it appears once in the directory, but after a reboot it is there four times.
Title: Re: Can't find the directory track on CPM Plus
Post by: GeoffB17 on 13:54, 15 January 17
Jon,


What drive are you using?   C:?   What format have you defined for C:?


Basically, the directory should be wherever you define it to be.   CP/M doesn't know, except for what you tell it.   There could, for example, be as many System Tracks as you might need.


If you're trying to use the data in the first sector to prime the XDPB, then that may give some indication.  But that might not work for HD, or might be conflicting?


You say a file appears once, but then 4 times.   How large is the file.   If you do a DIR then it should still appear once ONLY, but if you're looking at sector data, then the file could well show umpteen times, depending on the size of the file.   Check up about 'extents', and note that each extent can handle only 8 or 16 allocations blocks of the file dep on using 1 or 2 bytes for each alloc block.   If a file is showing 4 times, are the alloc numbers different in each entry?   If it's a small file only, needing 1 extent only, or if the block info is repeated, then YES, something is wrong.


I'm not sure you'll be able to use the normal DPB/XDPB processes, as the HD capacity could be far more than that system can cope with?


More info please.


Geoff
Title: Re: Can't find the directory track on CPM Plus
Post by: GeoffB17 on 14:29, 15 January 17
Jon,


What size of HD are you trying to emulate?


Within JE's Joyce emulator, the max size of emulated HD is 8Mb, although Joyce allows you to have a number of them.   That 8Mb may be the max that a 'normal' DPB can cope with (before one or more of the values required to define the parameters overflows).


The XDPB is separate, the numbers in the first sector info do not cover all the values required, and therefore limit further the options available and pretty much limit you to 800k or thereabouts.


If you're going to exceed the normal limits, then your code/drivers etc will have to create their own DPB system, and I assume disregard the XDPB process.   Are you trying to use the 'normal' DPBs within CP/M at present?


Geoff
Title: Re: Can't find the directory track on CPM Plus
Post by: JonB on 19:53, 15 January 17
Emulate?


It's real hardware on a real machine. There is no XDPB. That's Spectrum only according to JE. I provided a standard CP/M Plus DPB for an 8 meg HDD. All very ordinary.


The offset in the DPB is 1 (1 system track) so I expect the directory to be on track 1, per the CP/M Plus system guide.


Now, my DPB says there are 128 logical sectors per track, and 512 tracks. But since my driver uses LBA, I have to take a disk number, track number and sector number and bit bang them into LBA 0/1/2 values. Also, the FID interface deals in 512 byte sectors, so when it asks for a sector it is asking for a 512 byte sector. CPM 3 does the deblocking.


Anyway long story short: I expect the directory for C: (which is drive 0 in IDE terms) to start at LBA 32, because sectors 0-31 are the reserved track 0. But when I examine that sector after formatting the disk and copying a file to it, all I see is an unused directory data segment. Yet the file is there, can be read, even though it shows up 4 times when I do DIR.


Conclusion: CP/M put the directory somewhere else, but I cannot find it. Hence my OP...


I suppose I should ask "Does the Amstrad CPM put the directory somewhere unexpected?
Title: Re: Can't find the directory track on CPM Plus
Post by: GeoffB17 on 20:11, 15 January 17
Jon,


I've looked at enough PCW disks via ANADISK to be sure that the DIR sectors are always exactly where they're supposed to be as per the format on that disk (and I've used a number of different formats).


If the DIR is finding it, then I'd assume it IS the right place.   DIR will just go by the 'rules'?   But why the extra copies?


But your BASIC routine is NOT finding the data?


The only suggestion I can make is to get it to read LOTS more sectors, maybe specifically look for the one(s) that contain the correct data?   And report which TR/Sector that is.


An aside - the PCW does use the XDPB - that's what the data in the first sector of the floppy is.


I assume there's some reason why you're using LBA, and needing to translate sector/track ##.   Maybe connected with the difficulty of finding small HDs nowadays??


Geoff


Oh, unless this is part of your LBA translation...


Track numbers go from 0 to ??


Sector numbers go from 1 to ??
Title: Re: Can't find the directory track on CPM Plus
Post by: JonB on 21:22, 15 January 17
Ok.


I'll have to have a think. I had looked elsewhere on the disk, but it's not anywhere obvious.


Yes, the BASIC program could have a bug in it. It is terribly simple though. I have one that generates LBA 0/1/2 given disk/track/sector that runs on the PC, and one on the 9512 that takes the result and retrieves the sector off the disk and dumps it in hex. If you want to know why they're separate, it's because Mallard BASIC on the PCW doesn't support BIN$ (but has HEX$ and OCT$!).


The LBA translation code has been tested. It works as far as I can see.


One uses LBA because it frees you from concerns of what the physical disk geometry is.


I should also say that this isn't my first time round this particular block. See http://www.vcfed.org/forum/showthread.php?55487-TRS-80-Model-II-Lifeboat-CP-M-LoTech-IDE-Adapter-Extended-Driver-v1-9 which supports up to 32 8MB partitions on two physical drives, and has mapping tools so you can choose which of the 32 partitions are allocated to the 16 possible CP/M drive designations (A: to P:).


Although that is for CP/M 2.2, the disk access routines are the same as those in my PCW driver. Except that CP/M Plus does the deblocking for me.


Fascinating stuff...
Title: Re: Can't find the directory track on CPM Plus
Post by: GeoffB17 on 22:22, 15 January 17
Jon,


It'll be brilliant when it all works!


If DIR can find the directory, then it must be when it's supposed to be.   Must be something wrong with your BASIC prog.


If you can find the sector with the first bit of directory in, could you read out the data and write to a binary file and send it to me.   Interesting to see.


You never answered the question about the size of your test file, and if it might/ought to be using extents.


Another possibility MIGHT be that you've got 4 sectors each with one item in.   Not sure just how directory works, i.e. does it keep reading dir sectors until all done, or does it give up when it comes to an empty line.


Are you leaving CP/M to create the dir entry, or is your LBA stuff involved.   If CPM is confused about the blocking/size of file, it could be doing something wrong.


Geoff
Title: Re: Can't find the directory track on CPM Plus
Post by: JonB on 22:54, 15 January 17
CPM creates the directory. All it asks of my FID is to save a 512 byte sector. It does blocking/deblocking automatically. The extant point is a good one, but the file size doesn't change the behaviour.


CPM reads all of the directory when the disk is logged in. It builds an allocation bitmap called the ALV buffer. This is added to when you write a file, on the fly.


I observe that writing a file after logging the disk in with an empty directory shows one instance of the file as normal, but rebooting then logging the disk in again with DIR shows 4 instances. I conclude that the on-disk directory entry is corrupted on write, so when it tries to recreate the ALV it gets it wrong.


Hope that makes sense...
Title: Re: Can't find the directory track on CPM Plus
Post by: JonB on 23:14, 15 January 17
Oh, one more thought before I turn in.


The "extended DPB" sounds like some stuff that is put on a floppy by the Amstrad floppy driver. There is a nice essay about it here http://johnrhudson.me.uk/cpm/CPM_Disc_formats.pdf .


I reckon it reads the format type off this and chooses the correct DPB so that it can interpret the disk content. Further, the floppy driver is doing this. I've not seen any FID example doing the same (probably because all I have seen so far is for a fixed disk like my IDE drive). So, I know what the format of the drive is when the FID is loaded, because I formatted it (using the xdriver formatter, unchanged from my TRS-80 driver suite, because my PCW IDE prototype has the same I/O base address as the TRS-80 card.)


That's how I know this thing works. But on CP/M 2.2 only! I need to discover what is different about CP/M 3... which is at the core of my original question.


I know about timestamps and passwords, etc, but none of this is enabled on my drive (I haven't run INITDIR.COM). So it should work just like CP/M 2.2.


...should... :)


Tell me, are you interested in using this device on your PCW? I think the adapter board can be fabricated for less than a tenner and that the full cost would be in the £30 range. It's a way off, but I can do it (meaning by that the PCB design; I already have a design / schematic of sorts).


You never know, it may even work with Locoscript (which supports FID files, I believe).
Title: Re: Can't find the directory track on CPM Plus
Post by: GeoffB17 on 23:17, 15 January 17
Just about, but I wasn't aware that CP/M (or just CP/M+ ?) buffered the directory in any way.


I understand the use of AL0 and AL1 as format parameters, they are doing a bitmap of blocks used.   I assume the ALV bitmap notes which dir entries contain a file?   If it's a bitmap, it cannot be doing much more?


Still interested to see the data for the dir sector.


Regarding the BIN$ - didn't you feel inclined to write your own?   But, it cannot be trivial.   Checking my manual for QBASIC for the PC, I note that is just like Mallard BASIC, HEX$, OCT$, but no BIN$.


I think my MS C 7 can do it, but might have to use sprintf.   Clumsy, but who cares these days?


Your BASIC prog reminds me of long ago.   My first computer was an Epson HX-20, and after a year I got the add-on floppy drive.   The Disk BASIC included a couple of functions to read and write to the disk sector by sector, which allowed reading all the system/dir data directly.   The disks used were CP/M 2.2.  Very interesting functions!


May not reply further for a few days.   Off to hospital tomorrow for some surgery.


Keep up the good work.


Geoff
Title: Re: Can't find the directory track on CPM Plus
Post by: GeoffB17 on 23:35, 15 January 17
Yes, I could well be interested in adding a HD - I have at least one smaller IDE drive I might use.


Looking into the ALV thing, I see (at least for CP/M 2.2) that it's a long way from what I was thinking.   Seems to be used only to mark used blocks, i.e. free blocks (not used).   Does not take any note of file entries as such.   As I understand what it's doing, it would have no bearing on your duplicated entry problem.


I'll dig some more regarding CP/M+.


Still leaves the question, do you have one directory sector with 4 identical entries in, or do you have 4 sectors each with one?


Geoff
Title: Re: Can't find the directory track on CPM Plus
Post by: JonB on 07:53, 16 January 17
That's correct, Geoff. In CPM 2 the ALV uses one bit per disk block, but on CPM 3 it's 2 bits per block and hence needs to be twice as big. That's why my FID was crashing on access earlier on - I'd used the buffer settings for CPM 2 unchanged.


Incidentally, it means "Allocation Vector", and strictly speaking, it refers to the word in the DPH that contains the start address of the allocation bitmap buffer.


No need for a small capacity IDE device. Just use a CF card or DOM. Smaller, quicker and can be powered directly off the interface. If you look at the bottom of the README file for the TRS80 xdriver, there is a link to a German firm called Pollin, who can supply 128MB DOMs for pennies.
Title: Re: Can't find the directory track on CPM Plus
Post by: JonB on 08:38, 16 January 17
Geoff


Sorry, didn't notice your post about going to hospital. Hope it goes well!


Meanwhile.. I located the directory track eventually. I couldn't find it because the drive was not accepting my sector addresses in LBA mode, so it thought I meant "track/head/sector". Most odd, I thought LBA mode was the default. Mutter mutter. Oh well.


So anyway, what I see is that every logical 128 byte sector is being written four times (that is, one 512 byte disk sector contains 4 copies of the 128 byte sector that CP/M wanted to write). I see this in the files, too and I think this explains why executables won't run but text files can be read. All very odd.


First thing to do is work out why the BASIC program isn't working in LBA mode! If the FID is doing likewise then there will be missing sectors, because in non-LBA mode there is no sector 0.
Title: Re: Can't find the directory track on CPM Plus
Post by: JonB on 08:53, 16 January 17
Ah-ha! Found a defect in the BASIC program. I had the IDE Head register's I/O address wrong, so the drive never got addressed in LBA mode. (When you use LBA mode, you set bit 6 of the head register to indicate LBA addressing. Duhh..)


The FID was correct, though.
Title: Re: Can't find the directory track on CPM Plus
Post by: JonB on 14:05, 16 January 17
And - Ah-ha! again - I found out why there are duplicate entries appearing in the directory track. Basically a bug in the FID write routine. So now I am copying loads of CP/M utilities to the IDE device with PIP, but there is still a problem. Most of the programs will run from C: but BASIC.COM doesn't. Moreover, it won't run the DIR.COM either (which is small); it just says Illegal option or modifier" twice and locks up. I spotted some directory corruption, too (at the end of the list), so I'm not out of the woods yet. This may have been written by VDE as a temp file, which I had just loaded. More testing needed.


On the plus side, 2in1 does run from C: and it is much faster.
Title: Re: Can't find the directory track on CPM Plus
Post by: FloppySoftware on 15:02, 16 January 17
Jon, double check your DPB & DPH structures with the CP/M 3 manual.


Don't check anything related to CP/M 2, because they differ a lot.


CP/M 3 BIOS functions must to work with 'real' sector sizes - ie 512 bytes.


I quoted 'real' because you could build these structures with any accepted value - the hard work would be in your BIOS functions.



Title: Re: Can't find the directory track on CPM Plus
Post by: JonB on 18:22, 16 January 17
@FloppySoftware (http://www.cpcwiki.eu/forum/index.php?action=profile;u=1162): I've done that several times.. Can't see any problems. The CPM Plus driver on the PCW 9512 lives in a FID file and has a simplified interface, becuse this CP/M has loadable device drivers. The FID implements a routine called fidems that tells CP/M about the IDE drive (by way of a standard DPB and an allocation vector buffer size) and driver jump table.


The DBP (for an 8MB IDE drive, with ALV len = 512 bytes) is:



         ; Block size = 4096 bytes
dpb:      dw 128  ;SPT - sectors per track
      db 5    ;BSH - block shift factor
      db 31   ;BLM - block mask
      db 1    ;EXM - Extent mask
      dw 2043 ;DSM - Storage size (blocks - 1)
      dw 511  ;DRM - Number of directory entries - 1
      db 240  ;AL0 - 1 bit set per directory block
      db 0    ;AL1 -            "
      dw 8000h;CKS - DIR check vector size 8000=CP/M 3 fixed disk
      dw 1    ;OFF - Reserved tracks (for warm booting)
      db 2   ;PSH - Phys sector shift (2 => 512 byte sectors)
      db 3   ;PSM - Phys sector mask (3 => 512 byte sectors)


The jump table is:


; Disk device jumpblock
ddjump:   jp   logon      ; log in a disk
   jp   read      ; read sector
   jp   write      ; write sector
   jp   flush      ; flush write buffer
   jp   message      ; expand error messages


..so it is quite straightforward on the face of it. The interface between these calls and CP/M is documented.

I've done a bit more testing, and I can see all executables from the boot disk are loading and running from the IDE drive, with the exception of BASIC.COM and DATE.COM. (DIR.COM started working). If I copy BASIC.COM and DIR.COM back onto the IDE drive, they work. So the pattern is, neither will work the first time they are copied to the drive, but on a second copy they do work.
Title: Re: Can't find the directory track on CPM Plus
Post by: JonB on 19:38, 16 January 17
Alright, some more test results.
This points to something related to the initialisation of the ALV buffer on first logging the IDE disk in after a reboot. The directory track looks correct though (BASIC is 28k long on disk, and I can see it using 7 contiguous 4096 byte blocks in its directory entry).
Title: Re: Can't find the directory track on CPM Plus
Post by: FloppySoftware on 22:25, 16 January 17
Jon,


CKS must be ZERO for HDs (non removable media).


And where is the DPH?


Title: Re: Can't find the directory track on CPM Plus
Post by: JonB on 23:02, 16 January 17

Fixed disk CKS = 0000h for CP/M 2.2
Fixed disk CKS = 8000h for CP/M 3.1 ("Plus")


See P.41 of http://www.cpm.z80.de/manuals/cpm3-sys.pdf; (http://www.cpm.z80.de/manuals/cpm3-sys.pdf;) specifically:


Quote from: DRI CP/M 3 System Manual
CKS:  The size of the directory check vector, @DRM/4)+l. Set bit 15 of CKS to 1 if the drive is permanently mounted. Set CKS to 8000H to indicate that the drive is permanently mounted and directory checksumming is not required.


No DPH is needed, Amstrad CP/M 3 creates one for you via the FID_D_HOOK SVC call - http://www.seasip.info/Cpm/amsfid.html#svc_d_hook


I might try with a zero CKS anyway, it wouldn't take long.


[Edit: No, didn't make any difference]
Title: Re: Can't find the directory track on CPM Plus
Post by: JonB on 18:40, 19 January 17
OK, it's been a long haul but I may have it nailed.

The interface ignores /MI (the interrupt signal) and as a result, interrupt processing might be getting in the way of the timing of the input / output instructions in my driver. To test, I wrapped all IN and OUT instructions in DI / EI instructions, so as to ensure an interrupt never happens when accessing any of the IDE registers.

It seems to have worked, but I have a bit more testing to do; and I read the information about disabling interrupts and the effect on the PCW system clock. Too bad the PCW doesn't have the +3's SVC_CATCHUP routine..

Does this sound like a probable cause? As I understand it, a simple IDE interface would use /MI in its chip enable decoding (along with /RD, /WR, /IOREQ), but mine doesn't, because it's really a TRS-80 adapter, and that machine has additional decoding inside (buffered I/O too) whereas on the PCW it is more or less connected directly to the Z80 pins.

So I propose to incorporate /MI in the IDE enable logic so as to inhibit the enable signal if an interrupt is in progress, and remove the DI/EI instructions; thereafter it ought to work properly. Sounds reasonable?
Title: Re: Can't find the directory track on CPM Plus
Post by: GeoffB17 on 22:27, 19 January 17
Hello Jon,


I'm back home now, although minus a bit of colon!   Hopefully fine now.


I think you suggest a bit of your problem is still there.  There is one point that you still need to look at.


When CP/M+ initialises the disk it reads data into memory, and creates ALV.


Then, it reads writes files as approp, and I think it also updates ALV, so ALV is correct while you're using the machine.


BUT, when you shut down, the ALV info (updated) is NOT written back.   It is discarded.


Next time you turn on, the ALV is created afresh from the disk.


This seems consistent with what you decribe.   There seems to be still something going wrong with your use of alloc blocks ON the disk.


I'll read through all your other messages in more detail tomorrow.


Geoff
Title: Re: Can't find the directory track on CPM Plus
Post by: JonB on 09:44, 20 January 17

Hi Geoff


Good to hear from you again!


As I understand it, the ALV buffer is created when the FID is loaded (the buffer size being a parameter to the FID_D_HOOK SVC), then initialised when you first read the disk (log the disk in). Thereafter, the ALV is kept up to date with every write to the disk's directory. If the ALV and directory become out of synch, then a reboot will show the errors.


One thing I noticed was that on reboot, executing BASIC from C: didn't always have the same result. It might lock up, return to the C: prompt, or print the BASIC sign-on message and give me a BASIC prompt, but not respond to any commands. I also noted my test program that reads individual sectors directly off the disk (not via the driver) would sometimes drop the first byte of the sector, but at the time I thought it was a bug in the program.


I took a copy of BASIC.COM from the floppy drive and another from the IDE drive, and transferred them to a PC. I have only recently been able to do this as I have replaced the defective 3.5" drive in the PCW. On comparison of the files, it is clear that some bytes have been skipped in the IDE copy, whereas others are incorrect. These differences are scattered throughout the file, and there is a pattern of sorts.


So I reckoned something else was going on.. and recalled a conversation I'd had with 1024MAK about the interface, where he said the /MI line should be gated with the IDE enable signal to prevent interrupts from affecting I/O. During that exchange, I said there were no interrupts going on, but of course, I was wrong about that.


Since I have used a TRS-80 IDE adapter as my test bed and it doesn't gate /MI, I decided to put DI / EI around every IN / OUT instruction as a sort of "software equivalent". That's when the whole thing started working properly, but I haven't done much testing.


Since then, I started a code tidy as the FID's source was littered with test code. And now, it's broken again, but at least I have a lead on what the problem might be.


Wishing you a speedy recovery!


Cheers
JonB
Powered by SMFPacks Menu Editor Mod