News:

Printed Amstrad Addict magazine announced, check it out here!

Main Menu
avatar_OffseT

UniDOS, the new multi-device AMSDOS replacement

Started by OffseT, 15:51, 24 January 21

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

OffseT

Quote from: m_dr_m on 12:22, 08 April 21
Are there some routines that allow to seek/read a file block in a guaranteed amount of time?
Use-case: having file versions of track-load demos (with e.g. music or 3d animations while loading).
Yes, there are some routines to seek/read or seek/write into files.
The amount of time will obviously depend on the size of the block and on the node on which the file is opened.
For instance, seek/read on Albireo node is much faster than on FatFs/X-Mass node.
Anyway, it could be considered that any "modern" node will be faster than floppies.


Please also note that seek is not possible on all medias. Tape or serial don't allow seek operation (you can know about the current drive capabilities with the DEVICE INFO routine).

OffseT

Also, for your information, all nodes that were created for now (both released one and upcoming ones) are operating with interrupts enabled.

m_dr_m

Wait, what? Even FDC based nodes? How does it work?
For the other devices, the transfert can be interrupted/resumed? Is there a max time allowed to fetch the data?


Another question: Can `|COPY` use a directory as source? The doc contradicts itself on this point.


OffseT

Quote from: m_dr_m on 10:02, 09 April 21
Wait, what? Even FDC based nodes? How does it work?
I was speaking about actual nodes for new hardwares only (Albireo, X-Mass, M4, ...).


In Fact, there is no FDC based node for now (legacy floppy disc drives access is done through AMSDOS ROM, which is obviously disabling interrupts).
But sure, when one will me created (to handle HxC in direct mode for instance), it will have to disable interrupt at some point.

I am thinking about providing information related to interrupt handling in the DEVICE INFO status; so that a program could know if interrupts will be disturbed or not by drive accesses.

Quote from: m_dr_m on 10:02, 09 April 21For the other devices, the transfert can be interrupted/resumed? Is there a max time allowed to fetch the data?

Not for now.

Quote from: m_dr_m on 10:02, 09 April 21Another question: Can `|COPY` use a directory as source? The doc contradicts itself on this point.

|COPY is limited to single file copy for now. It will be expanded to a full featured copy later.
In the meantime, a dual-lister graphical copier should be available too (DOpus / FileMaster style).

GUNHED

Well, IMHO it would be better to use ParaDOS instead of Amsdos, bigger floppy formats are a prerequisite for efficient work.


BTW: For FDC data transfer every 25 us a byte needs to be written/read to/from FDC.
http://futureos.de --> Get the revolutionary FutureOS (Update: 2023.11.30)
http://futureos.cpc-live.com/files/LambdaSpeak_RSX_by_TFM.zip --> Get the RSX-ROM for LambdaSpeak :-) (Updated: 2021.12.26)

OffseT

Quote from: GUNHED on 18:13, 10 April 21
Well, IMHO it would be better to use ParaDOS instead of Amsdos, bigger floppy formats are a prerequisite for efficient work.
As stated on the web site, both AMSDOS and ParaDOS are already supported by UniDOS (and a fixed ParaDOS version is available which is able to work in any ROM slot, not only 0, 6 or 7).

GUNHED

http://futureos.de --> Get the revolutionary FutureOS (Update: 2023.11.30)
http://futureos.cpc-live.com/files/LambdaSpeak_RSX_by_TFM.zip --> Get the RSX-ROM for LambdaSpeak :-) (Updated: 2021.12.26)

OffseT

#57
UniDOS 1.10 is now available! 8)


This new version features:

       
  • New FatFs DOS node with X-Mass and compatible interfaces support (more FAT devices will follow).
  • New built-in TAPE: drive where applicable.
  • Optimized Albireo DOS node.
  • New |NODE RSX to list all installed nodes and detected devices.
  • New |FORMAT RSX to prepare new discs.
  • Improved memory layout and developpers documentation.
All information can be found here:
UniDOS (english Google translate)
https://unidos.cpcscene.net (français)

Targhan

Wow! You did it very quickly! I don't have Xmass though to test the latest node, too bad :).
(small typo found in the doc: "qu'elle être omise une fois le disque formaté.". Not very french :)).
Targhan/Arkos

Arkos Tracker 2.0.1 now released! - Follow the news on Twitter!
Disark - A cross-platform Z80 disassembler/source converter
FDC Tool 1.1 - Read Amsdos files without the system

Imperial Mahjong
Orion Prime

OffseT

Quote from: Targhan on 09:48, 19 April 21
Wow! You did it very quickly! I don't have Xmass though to test the latest node, too bad :) .
(small typo found in the doc: "qu'elle être omise une fois le disque formaté.". Not very french :) ).
In fact it was finished about one month ago already, but I wanted to test it correctly before a release. ;D


It was quickly done because the code is based on FatFs which was converted to Z80 bizarre assembler using sdcc, and then to regular assembler thanks to a tool from No Recess.
The hard part was to make all this ugly code to fit in ROMs, and to handle the insane stack usage of FatFs... and also that I do not have a X-Mass.  :P
(that's the reason why I had to create a X-Mass emulation plugin for ACE in the first place).


Anyway, the typo is fixed. Thx.

zhulien

The sdcard device,  is that for m4 cards or another sdcard reader?


Also is the usb device compatible with mini booster and HxC native?

zhulien

I long for the day the following are all supported so all can be used at the same time...


- floppy a and b (supported)
- SD on M4 (?)
- xmas 128mb and 512mb (supported?)
- HxC native mode for FDA and FDB
- USB sticks (supported?)


OffseT

#62
Quote from: zhulien on 21:01, 20 April 21
The sdcard device,  is that for m4 cards or another sdcard reader?

Also is the usb device compatible with mini booster and HxC native?
Supported devices are listed on download page of https://unidos.cpcscene.net

Basically...

Already supported:

       
  • Floppy disc drive A & B (both AMSDOS and ParaDOS).
  • Tape.
  • USB Mass Storage through Albireo USB (any kind of device apart of USB floppies and CD-ROM).
  • SD card on Albireo.
  • Master drive of X-Mass or Symbiface 2 (any FAT, any size up to 128GB).
Being worked:

       
  • Nova NVRAM and RTC support.
  • CPC built-in parallel port.
  • Fast serial port of Albireo.
  • SD card on M4.
Nice to have:

       
  • Slow serial port of Mini-Booster.
  • Slave drive of X-Mass or Symbiface 2.
  • HxC native mode on drive A and B.
  • CPC464 and CPC664 firmware compatibility.
That said, do not expect myself to code all the DOS nodes for all the devices. First I do not own all the existing devices and second I have other projects on the way.

Anyway, SDK for DOS nodes available and fully documented, and I am always available to provide support to anyone willing to code new DOS nodes. To make it short: we need your help to support as much hardware as possible.

The goal remains of course to have one single AMSDOS-compatible DOS to handle all the existing hardware, so that software coders won't have to care about which hardware is being used.

zhulien

What is the current expected behaviour if using an M4 (with the M4 ROM) and UniDOS?

OffseT

Quote from: zhulien on 21:08, 22 April 21
What is the current expected behaviour if using an M4 (with the M4 ROM) and UniDOS?
No idea. But I guess it won't fit very good.

GUNHED

Quote from: OffseT on 08:15, 23 April 21
No idea. But I guess it won't fit very good.
The M4 board is probably by far the most often sold expansion for the CPC during the last 30 years (more than 1000 units). Also the M4 provides LBA sector reading / writing (all you need for FAT32).
IMHO you can't name your DOS with the label UniDOS without M4 support.  ;) :)
http://futureos.de --> Get the revolutionary FutureOS (Update: 2023.11.30)
http://futureos.cpc-live.com/files/LambdaSpeak_RSX_by_TFM.zip --> Get the RSX-ROM for LambdaSpeak :-) (Updated: 2021.12.26)

OffseT

#66
Quote from: GUNHED on 20:03, 23 April 21
The M4 board is probably by far the most often sold expansion for the CPC during the last 30 years (more than 1000 units). Also the M4 provides LBA sector reading / writing (all you need for FAT32).
IMHO you can't name your DOS with the label UniDOS without M4 support.  ;) :)
M4 support will come with a neat M4 DOS Node. There is no reason to mess everything up by trying to make it work with the M4 ROM which is not designed for multi-devices at all.
Also, there is no need to deal with FAT32, AFAIK M4 have direct file support just like Albireo.

zhulien



Below logic so far to get an understanding of how a node logic could be, does it look right?




var nvram_opened = false;
var nvram_offset = 0;
var nvram_start = 0;
var nvram_size = 128;


// Initializes the DOS node
//
// Enter - D = -1
// E = -1
// Exit - If Carry = 1 then the node has been initialized
// If Z then a non-volatile memory manager is available in this node (2K mini)
// Only the non-volatile memory of the first node to have it is used by the UniDOS, in this case:
// If D is modified then the value is used as the default reader number in A
// If E is modified then the value is used as the default reader number in B
// If NZ then no non-volatile memory manager is available
// A = error code
// If Carry = 0 then the node could not be initialized
// Altered - AF, DE
    this.init = function(d, e)
{
nvram_opened = false;
nvram_offset = 0;

c = 1; // initialised
z = 1; // non-volatile memory available
a = 0; // no error

return a, c, z;
};

// Checks if a drive name matches a drive managed by our DOS node
//
// Enter - HL = pointer to drive name
// (if set to certain characters, bit 7 must be ignored)
// C = length of drive name
// Exit - If Carry = 1 a match was found
// A = number of the physical reader (from 0 to 7)
// If Carry = 0 no match was found
// Altered - AF
    this.checkDrive = function(hl, c)
{
if (hl == 'n')
{
c = 1; // found the drive
a = 0; // it is drive 0
}
else
{
c = 0; // drive not found
}

return a, c;
};

// Returns the status of a reader
//
// Enter - A = Reader number
// Exit - If Carry = 1 a status could be retrieved
// A = Player status (see Media_ * flags)
// Bit0 = 1 if media is available on this drive
// Bit1 = 1 if the media manages the directories
// Bit2 = 1 if the media is write protected
// Bit3 = 1 if the media is removable
// Bit4 = 1 if the media is a stream (linear read / write only)
// The management of the $$$ and BAK files is then deactivated.
// Bit5 = 1 if the media is accessible via the new AlbiDOS APIs (always set to 1)
// The other bits are not used
// If Carry = 0 then the reader is unknown and no status could be retrieved
// A = 0
// Altered - AF
    this.getStatus = function(a)
{
a = 0x00000011;
c = 1;
return a, c;
};


// Returns the name corresponding to a physical drive
//
// Enter - A = reader number
// DE = address of an 8-byte buffer where to copy the name
// Exit - If Carry = 1 the name was found
// DE points to the first character after the end of the string copy
// (the stored string is terminated with bit 7 to 1)
// If Carry = 0 the name was not found and the buffer was not modified
// Altered - AF, DE
    this.getName = function(a)
{
if (a == 0)
{
c = 1; // found the drive
de = 'n'; // it is drive n
}
else
{
c = 0; // drive not found
}

return a, c;
};

// Returns the description corresponding to a physical drive
//
// Enter - A = physical drive number
// DE = address of a 32-byte buffer where to copy the description
// Exit - If Carry = 1 a description was found
// DE points to the first character after the end of the string copy
// (the stored string is terminated with bit 7 to 1)
// If Carry = 0 no description was found and the buffer was not modified
// Altered - AF, DE
    this.getDesc = function(a)
{
if (a == 0)
{
c = 1; // found the drive
de = 'net drive'; // it is drive n
}
else
{
c = 0; // drive not found
}

return a, c;
};


// Change the protection bits of a file (NewAPI only)
//
// Enter - A = physical drive number
// HL = pointer to standard name
// DE = pointer to the normalized path
// B = Protections to be modified
// C = New protections
// Bit 0 - Read only
// Bit 1 - Hidden
// Bit 5 = Archived
// The other bits are ignored
// The pointed memory is in the current ROM / RAM space
// Output - If Carry = 1 the routine is supported by this reader
// If Z then the protections have been modified
// If NZ then the protections could not be modified
// A = error code
// If Carry = 0 then the routine is invalid for this reader
// Altered - AF
    this.setProtection = function(hl, de, b, c)
{
c = 0; // not supported
return c;
};

// Open non-volative memory (NewAPI only)
//
// Enter - C = opening mode
// If 0 then the non-volatile memory with its current content must be opened
// If 1 then the non-volatile memory must be reset to zero before opening
// Exit - If Carry = 1 then the DOS node offers non-volatile memory service
// If Z then the non-volatile memory has been opened and is ready to be manipulated
// If NZ then an error has occurred
// A = error code
// If Carry = 0 then the DOS node does not offer non-volatile memory
// Altered - AF
    this.openNVRAM = function(c)
{
if (c == 1)
{
clear the NVRAM
}

nvram_opened = true;
nvram_offset = 0;
c = 1;
z = 1;
a = 0;

return a, c, z;
};

// Closes and updates the contents of non-volatile memory (NewAPI only)
//
// Entrance - None
// Exit - If Carry = 1 then the DOS node offers non-volatile memory service
// If Z then the non-volatile memory has been updated
// If NZ then an error has occurred
// If Carry = 0 then the DOS node does not offer non-volatile memory
// Altered - AF
    this.closeNVRAM = function()
{
if (nvram_opened)
{
TODO: update NVRAM contents;
nvram_opened = false;
z = 1;
}
else
{
z = 0;
}
c = 1;

return c, z;
};

// Reads data from non-volatile memory (NewAPI only)
//
// Input - HL = address where to write the read data
// DE = number of bytes to read
// Exit - If Carry = 1 then the DOS node offers non-volatile memory service
// If Z then the data has been read
// DE = number of bytes actually read
// If NZ then an error has occurred
// A = error code
// If Carry = 0 then the DOS node does not offer non-volatile memory
// Altered - AF, DE
    this.readNVRAM = function(hl, de)
{
if (nvram_opened)
{
if ((nvram_start + nvram_offset + DE) < nvram_size)
{
TODO: transfer DE bytes from nvram to address HL;
z = 1;
}
else
{
z = 0;
a = out of bounds;
}
}
else
{
z = 0;
a = not opened;
}

c = 1;


return a, c, z;
};


// Writes data to non-volatile memory (NewAPI only)
//
// Input - HL = address where the data to be written is located
// DE = number of bytes to write
// Exit - If Carry = 1 then the DOS node offers non-volatile memory service
// If Z then the data has been written
// DE = number of bytes actually written
// If NZ then an error has occurred
// A = error code
// If Carry = 0 then the DOS node does not offer non-volatile memory
// Altered - AF, DE
    this.writeNVRAM = function(hl, de)
{
if (nvram_opened)
{
if ((nvram_start + nvram_offset + DE) < nvram_size)
{
TODO: transfer DE bytes from HL to nvram;
z = 1;
}
else
{
z = 0;
a = out of bounds;
}
}
else
{
z = 0;
a = not opened;
}

c = 1;


return a, c, z;
};

// Change position in non-volatile memory (NewAPI only)
//
// Enter - DEHL = new position
// Exit - If Carry = 1 then the DOS node offers non-volatile memory service
// If Z then the new position could be reached
// If NZ then an error has occurred
// A = error code
// If Carry = 0 then the DOS node does not offer non-volatile memory
// Altered - AF
    this.seekNVRAM = function(de, hl)
{
if (nvram_opened)
{
if (DEHL is within nvram)
{
TODO: seek to position DEHL in nvram;
z = 1;
}
else
{
z = 0;
a = out of bounds;
}
}
else
{
z = 0;
a = not opened;
}

c = 1;


return a, c, z;
};



The other functions I haven't yet finished looking at... for seekNVRAM, what is inside DEHL? 

m_dr_m

Quote from: zhulien on 10:38, 26 April 21for seekNVRAM, what is inside DEHL? 

The address at which the next Read/Write must occurs.
E.g. if the seekNVRAM is invoked with DEHL = &4f0, and then your DOSNode_ReadNVRAM is called with DE = 2, HL = &8000, you must write the word at &4f0 from your NVRam at &8000 in CPC memory.

Note: you normally don't have to code those routines (especially for an Network FS!), since it's likely another node is already supporting the NVRAM is an efficient way. Unidos needs only one NVRAM, and if your node wants to store persistent info, I think we should come up with a better solution (e.g. new routine exposed by Unidos).

m_dr_m

I also have a question:


* For now the FatFS seems dedicated to X-Mass. How would one reuse the FAT32 handling for another device?
I was expecting to see 'Device Nodes' exposing read/write_sector, with a |DRIVE (mount) version able to bind a mount point to a (FS, hard device) custom couple.


And now for something completely different: it seems the NVRAM is accessed quite a lot! E.g. for a cat on floppy disc: one time at the beginning, and one time before displaying the number of free Kb. Why is that so?


Frankly, I would prefer all work memory to rely on CPC memory (I don't really care about HIMEM being lower!) and/or reusing some unused firmware space (I never use tapes!).
I know tradeoffs are difficult and you prioritised compatibility, but for a CPC developper, that looks a suboptimal solution has been picked just to deal with rare corner cases.


A solution might be to be only switch to high compatibility when ROM7 is re-init alone (e.g. for Games/Demos loaders).
Also, for programs like OCP, the alternative would be to patch them or obsolete them!



OffseT

#70
Quote from: m_dr_m on 10:34, 28 April 21
* For now the FatFS seems dedicated to X-Mass. How would one reuse the FAT32 handling for another device?
I was expecting to see 'Device Nodes' exposing read/write_sector, with a |DRIVE (mount) version able to bind a mount point to a (FS, hard device) custom couple.
A few more routines needs to be exposed than just read/write sector. You also need to be able to initialize the device, to probe its geometry, to format it. FatFs also requires about 600 bytes of memory for each new mounted drive, which is tricky to deal with in an AMSDOS compatible manner. :(

That's why for now FatFs is restricted to one single IDE device. Of course, it will be expanded in the future, that's also the reason why the ROMs are splitted in three parts.
One which is the FatFs itself.
One which will contain drivers for supported devices (at some point it could become also a modular ROM handling sub-rom/ram modules).
One which is containing all the tools for disc partitionning and formatting.

But before expanding FatFs support to new devices, I guess that we first have to optimize FatFs itself. The current code is just some adapted sdcc output which is quite ugly, slow and stack killer.
In fact, I'm just surprised it works that good. :P

Quote from: m_dr_m on 10:34, 28 April 21
And now for something completely different: it seems the NVRAM is accessed quite a lot! E.g. for a cat on floppy disc: one time at the beginning, and one time before displaying the number of free Kb. Why is that so?
True. NVRAM is used for two different things.
First it is used to store actual UniDOS configuration and current context (that's the main usage).
Second, it is used as a swap memory area to deal with AMSDOS/ParaDOS memory buffers which are stored at the same address that UniDOS ones for compatibility reasons.

That's why it is used when accessing floppy discs. 8)

Quote from: m_dr_m on 10:34, 28 April 21
Frankly, I would prefer all work memory to rely on CPC memory (I don't really care about HIMEM being lower!) and/or reusing some unused firmware space (I never use tapes!).
I know tradeoffs are difficult and you prioritised compatibility, but for a CPC developper, that looks a suboptimal solution has been picked just to deal with rare corner cases.

Parts of tape memory area is already used by UniDOS for internal stuff. Basically, almost all possible spare memory which is not breaking compatibility is already used. ;D


But well, compatibility is not related to "rare corner cases". Touching HIMEM, even about a few bytes, will prevent hundreds of softwares from working.
The goal was not to create a DOS which would work only with dev tools (which is basically what happens when HIMEM is changed).
You don't want to switch back to floppies to play games, to use Digitracker, Starkos, Soundtracker DMA :o , and so on.
The goal is really to use all of this in one single AMSDOS compatible enviroment... and directly from files (we don't want those nasty DSK anymore).


That said, you are right, this is not optimal (in fact, mostly when dealing with AMSDOS buffer cache while accessing floppy discs, other use-cases have almost no penality performance-wise). Anyway, the Nova card from PulkoMandy was created to get rid of this "on-disk" NVRAM while keeping a high compatilibty level. 8)
It is particulary valuable for guys like you and me which are still using they CPC for development purpose, especially when using FatFs (Albireo users won't notice big performance boost at all, but it will still save write cycles on SD card).


Quote from: m_dr_m on 10:34, 28 April 21A solution might be to be only switch to high compatibility when ROM7 is re-init alone (e.g. for Games/Demos loaders).
Unfortunately, the compatibility issue is not limited to ROM7 reinit. A lot of softwares (notably with Basic loaders) won't reinit ROM7 but will require the default memory layout to work properly.
Some of them are even directly using ROM routines which are totally undocumented... which is for now the main UniDOS compatibility flaw, even when it is installed as ROM7.

Quote from: m_dr_m on 10:34, 28 April 21Also, for programs like OCP, the alternative would be to patch them or obsolete them!

I agree, in a perfect world we could patch everything, and we would recode old-fashioned softwares in a smart manner.
But it won't happen.  :'(  Just look at modern games and softwares, their compatibility is often worse that old ones!
That said, I already created a patched version of OCP which works when UniDOS is not stored in ROM7. ;D
But this was an easy patch. Patching it to work on a system where memory is dynamically allocated is another story.

OffseT

Quote from: m_dr_m on 10:12, 28 April 21
Note: you normally don't have to code those routines (especially for an Network FS!), since it's likely another node is already supporting the NVRAM is an efficient way. Unidos needs only one NVRAM, and if your node wants to store persistent info, I think we should come up with a better solution (e.g. new routine exposed by Unidos).
That's true. NVRAM is a peticular feature which a network DOS node could/should avoid.

zhulien

I was thinking the NVRAM would be used to store settings for example, I didn't think it was constantly read and written to so many times.  This does mean a bit of rethinking... i.e. loading the security token from floppy disc instead of NVRAM - but having said that, you did say something else (likely faster than the network) would have an NVRAM, so all should be good still... saves me coding the NVRAM stuff - but that bit does appear somewhat straight forward.


With regard to FatFS... could it be possible to have 3 types of UniDOS nodes - type 1 which essentially is what we have now to handle files, directories, reading, writing, and type 2 which is a FS handler which can be 'chosen' if the defaults are not wanted (type 1 always goes through a type 2), and type 3 which is a trackdisc handler (which can optionally be communicated directly).  All Types could be queried from the primary UniDOS ROM to get a list of what is available.  Having the 3 means eg: a FatFS could be used with any device handler as the FatFS is only the software layer.  A track disc that supports formatting could work with any formatter software created for UniDOS etc.

m_dr_m

@zhulien That is more or less what SymbOs does, with 3 layers: Main DOS management (public API), FileSystems (Fat, Amsdos, ...), Device drivers.
That's why for instance HxC direct access was so rapidly supported.


@Offset Fair enough! What about doing the switch gymnastic only at drive change?
E.g.:
|A <- Backup Unidos, put Amsdos
cat  <- No NVRAM access
cat  <- No NVRAM access
|B <- Restore Unidos.


It's quite disturbing to see the X-Mass active for each floppy access. Makes you wonder if the right drive is selected.

OffseT

Quote from: zhulien on 14:26, 28 April 21
With regard to FatFS... could it be possible to have 3 types of UniDOS nodes [...]
Sure, something in this regard is planned.
At final stage, there will be additional modular items under the DOS node layer itself.

DOS node are the DOS device interface for UniDOS but then...

Some of them will remain stand alone handlers (i.e.: Zero, Albireo SD/UMS, M4).
Some of them will be split in both a device driver and a filesystem (i.e.: X-Mass IDE, HxC Direct)
Some of them will be split in both a device driver and a protocol (i.e.: Albireo serial port, Mini-Booster)


Internally, it is already working this way in FatFs DOS Node. FatFs part (filesystem) is totally independant from X-Mass device driver.

Powered by SMFPacks Menu Editor Mod