News:

Printed Amstrad Addict magazine announced, check it out here!

Main Menu
avatar_zhulien

Driver ROMs

Started by zhulien, 04:46, 05 October 16

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

zhulien



Although a little premature (and I don't want to hijack the M4 or X-Mem threads), but I am hoping everyone (hardware and software) people can consider to agree on a standard way to allocate/access RAM on CPC... - more from a software perspective.

The purpose is so that we don't have someone trash someone's RAM drive for example... but not only that... what about other tasks running at the same time.  Fetching from websites for example too given that is now a reality on CPC.



So far, I have written as part of MCP, a driver system for the following.  There is crossover in the drivers, but they are separately listed for easy identification by users who want to 'build' their MCP ROM for 'their' hardware - build process is via a BASIC app just selecting the required drivers and choosing build.


(low level) RAMs supported so far: Standard 64kb, Standard 128kb, 64kb dktronic, 256kb dktronic, 256kb silicon disc, 8kb multiface, 512kb xmem, 512kb ymem (untested), 1gb vram (in progress).  ROMs planned to be supported if possible: xmem rom writer, xmem rom reader, m4 rom writer (is that possible from cpc?), rom reader.  Various other WIP: joystick 1 driver, joystick 2 driver, multiplay mouse1 driver, multiplay mouse2 driver, multiplay joystick 1 driver, multiplay joystick 2 driver, inbuilt soundchip driver, playcity soundchip 1 driver, playcity soundchip 2 driver, amdrum audio out driver, cpcbooster+ audio out driver, cpcbooster audio in driver, (high level) ramdrive driver, romdrive driver (grab a bunch of files and autobuild roms for them - good for any slot in particular >=32...)


some of the low level for now, such as the sound chips are just for allocation and freeing so multiple apps can just allocate and free them to play their sounds.


The user can as mentioned build a ROM with their set of hardware (for now at least 1 set).  The current allocator is pure malloc, free etc... across RAM universally of type...


however I am wondering other's opionions on the following:


#1 - I am hoping to get away with 16kb max sized chunks for the normal allocation schemes, a few bytes at the start is used as a header

#2 - how important will it be to be able to allocate a full 16kb bank or even a full 64kb bank?  (this would allow eg: symbos (with modifications) to run, and shell out to AMSDOS and restore itself)

#3 - for now sound driver is a pure resource allocator/deallocator to avoid clashes, full freedom how to use it beyond allocation/deallocation is expected

#4 - to allow for full memory banks (16kb or even 64kb) i will need to store a status per bank 'somewhere'. The benefit of #1 is that each bank has it's own header, no additional RAM is really required outside the bank itself to know it's status.  however, if #2 is highly desired, then a status needs storing outside of that bank. this is still best outside the main 64kb memory - which means do we want to lose some of another bank (a system bank) for this type of memory management?  If NO, then that's good... If YES, then... my design needs tweaking and might as well use the 'system bank' for task lists (instead of allocating memory with malloc), and bank status, and bank redirection (for vmem)

#5 - yes, vmem will work invisibly to software on a 64kb machine with M4 card if they are using these drivers. On a 64kb RAM machine we get 2 16kb banks in the main memory, preserving BASIC below that (yes, not much) - but for binary apps better flexibility is there.  the benefits of 2 16kb banks on a 64kb machine is each can be individually and automatically paged out based on last access. 
A game that is using the drivers would have access up to 1GB memory.  On a 1MB CPC (with xmem and ymem) then of course we have 64 last accessed cached banks.  This last access caching requires a redirection table.  If it is 64kb large then it gives 1GB of 16kb banks).  Foregoing the VMem all together simplifies things a bit and speeds up access (not requiring redirection).  Does anyone use 64kb CPC anymore? 

#6 key features/goals of the memory driver ROM (aside from hopefully developers conforming to it)... public domain with full sourcecode provided (sorry not open source).  reset recoverable, can snapshot (to sdcard or floppy?) and reload automatically also not just the ram but the running tasks.  if >64kb then running tasks in background of BASIC, if 64kb only then really a developer tool for new software.Other future drivers would include drive access and other hardware, but for now focusing on the worst thing of everyone trashing eachother's RAM and RAMdrives...

andycadley


Thoughts:


1. No matter what you do, 99% of software isn't going to work with it - including BASIC. That, at a minimum, rules out using any of the standard 64K of RAM for storing anything that isn't completely transient. Which is an enormous restriction on an 8-bit machine.


2. The overheads for something like this are not insignificant. If you allocate in say 16KB blocks, then simply to keep track of what's been allocated you're going to end up losing 1 16KB block. Because of #1, that's essentially 1/4 of available memory lost on a 128K CPC.


3. The minute you start trying to merge such a system with something cross-platform, like symbos, then you have to also factor in all the constraints that already exist in symbos across all platforms. And simply having a shared allocater may not be sufficient to "shell out" if the firmware etc has to be reset etc.


Honestly, at this point, there is little point other than the intellectual exercise of "How would I have done this if I'd be designing the CPC?" Without buy-in from the firmware and all "firmware friendly" applications, the limitations are just too steep to make it practical on a machine with very limited resources.

zhulien

It's going to happen for a couple of applications.  One being the CPC Web Browser.  Having said that, it might be something only the Web Browser uses.  If there isn't any interest in other developers not trashing RAMDiscs etc allways or the tasks that are running in the extra RAM, fair enough too.  But it is a nice dream to have.  People can still have the choice of using these drivers and not having a trashed RAM drive.

andycadley


A much, much easier and more reliable approach is just to reserve all banked RAM for RamDisk of any flavour. This is the approached used by the 128K Spectrums and has the benefit that all 64K software is already going to be compatible.


Furthermore, applications don't have to work with massive layers of indirection if they do want to use the extra memory, they just Load/Save relevant files from RamDisk as required.

zhulien

Ramdisc is planned but it is no good for multitasking alone.

Prodatron

Hi Zhulien,
your approach is more or less what MSX-DOS2 (MSX2) and ISDOS (Enterprise 64/128) are providing. If an application needs additional memory it always has to request one or multiple 16K blocks, and the OS is reserving them in a dynamic way. These machines are more flexible in memory banking compared to the CPC (you can map any 16K bank [up to 256; MSX even more] to any of the 4 blocks of the visible 64K). The native OS was prepared for this memory management since the beginning.
If you start your own software suite it may make sense to introduce such a system inside the CPC-OS as well. SymbOS CPC currently of course doesn't take care about such things when it is booting from Amsdos. SymbOS EP (Enterprise) indeed is asking the native OS for all free 16K blocks and only uses these. So there is already something like this for SymbOS as well, but on another Z80 system :)

GRAPHICAL Z80 MULTITASKING OPERATING SYSTEM

andycadley


Quote from: zhulien on 09:08, 06 October 16
Ramdisc is planned but it is no good for multitasking alone.


The minute you allow the conditions "can survive reboot" and "can be used by multiple applications" your memory manager effectively has to act as a Ramdisc anyway (consider a program starting up after reset - how does it find where individual variables are stored? how does the memory manager even know if it's the same program?).


It can also take away many of the thorny problems associated with relinquishing memory after a reset if the user wants it for something else, since the user can just use standard DOS commands to delete things as needed. Additionally applications could then run from alternate storage like hard disks/flash storage by just swapping the Ramdisk for something more permanent (with the hard disk driver using the banked ram for it's own caching purposes if it so desires)

zhulien

The boot survive option is optional and invaluable when developing, it means when a program crashes, the entire dev environment and all source can re-load automatically without having to do anything but push the reset button.

m_dr_m


Very interesting! What is the status of this project?


I happen to use a minimal memory management for Orgams with the following features:
• Allocate by linked &100 chunks
• Routine for fast insertion (e.g. in a 60k blob).
• Even higher level API (line based rope-like data structure).
• Non destructive memory detection (how much Kb your CPC have).
• Persistent across reset. Actually, "crash-survive", since there are checksum ensuring the used memory wasn't trashed.
• Other.


For incoming tools, I was thinking of a more generic allocation API.
Here is a draft proposal.

zhulien

#9
it is still in progress but expanded into a general driver thread... With lots of notes examples and now logic for the kernel extension.  The mcp POC is also in another thread.


i will now ready your memory mapper notes. (read)


definately some crossover in goals.  have you seen my.driver and mcp threads?


https://www.cpcwiki.eu/forum/programming/cpc-drivers/

m_dr_m


Nice to hear,It looks very interesting and well planed! Yet I'm a bit confused:


* Which problems will it solve?
* Have you some use-cases examples?
* Do you mean for instance being able to use the same code from AMSDOS/SymbOs/FutureOs and all?

m_dr_m

By the way, I'm all for uniformisation and interoperability!

GUNHED

Therefore I made FutureOS.  :) :) :)
http://futureos.de --> Get the revolutionary FutureOS (Update: 2023.11.30)
http://futureos.cpc-live.com/files/LambdaSpeak_RSX_by_TFM.zip --> Get the RSX-ROM for LambdaSpeak :-) (Updated: 2021.12.26)

zhulien

Quote from: m_dr_m on 02:23, 18 January 21
Nice to hear,It looks very interesting and well planed! Yet I'm a bit confused:


* Which problems will it solve?


Many for example, why does Steve Davis Snooker only work with DkTronics Speech Synth and not any other? Why does Roland in Space only work with the SSA-1 and not any other?  Why does everyone's filemanagers only work with a small number of storage devices and not everyone connected to my CPC?


Quote from: m_dr_m on 02:23, 18 January 21
* Have you some use-cases examples?


Why can't I use the same software on my floppy replacement USB drive B, my SSD, my M4 SD card, my 3" floppy drive, potentially all at the same time?  Why isn't all my RAM available to every program that wants to use it in a uniform way?  even if multitasking, why can't they be friendly?


Quote from: m_dr_m on 02:23, 18 January 21
* Do you mean for instance being able to use the same code from AMSDOS/SymbOs/FutureOs and all?


Code can be shared across all Z80 machines and OSs to a degree if layered correctly, however I am primarily focusing on CPC AMSDOS - for now.  Now FutureOS is interesting, I suspect as a monolithic OS (with all the drivers built in) but it doesn't have AMSDOS/BASIC to play with - is it reasonably possible to adapt AMSDOS / Locomotive BASIC to work within FutureOS? or perhaps BBCBASIC?  Does FutureOS support multitasking?  But the author of FutureOS likely has his own goals and ultimate direction of the OS.


For me, I'd like my CPC to remain a CPC for most part, but allow me to utilise all the hardware in a compatible way.

GUNHED

#14
Quote from: zhulien on 02:05, 27 January 21
Many for example, why does Steve Davis Snooker only work with DkTronics Speech Synth and not any other? Why does Roland in Space only work with the SSA-1 and not any other?
Well, therefore we have the LambdaSpeak now. And it does all this and more.
But, yes I got your point. Problem is that a software solution (as great as your idea is!), needs support from upcoming software too.
Staying in the example of the games supporting one, but not the other Speech-Synthesizer. Why is that the case? Well, because the programmers are lazy - and they think one is enough.
Such a driver ROM / drivers system is really a great idea - but - who will support it? Will upcoming games do that? I doubt, they don't even care about current / new hardware (very few exceptions though).

Quote from: zhulien on 02:05, 27 January 21For me, I'd like my CPC to remain a CPC for most part, but allow me to utilise all the hardware in a compatible way.
What do you mean by this? Yellow on Blue? No external CPU? Firmware as OS?
http://futureos.de --> Get the revolutionary FutureOS (Update: 2023.11.30)
http://futureos.cpc-live.com/files/LambdaSpeak_RSX_by_TFM.zip --> Get the RSX-ROM for LambdaSpeak :-) (Updated: 2021.12.26)

zhulien

Quote from: GUNHED on 18:24, 27 January 21
What do you mean by this? Yellow on Blue? No external CPU? Firmware as OS?


Ideally Locomotive BASIC as it's core OS, or something that provides a Locomotive BASIC shell or compatible implementation could work.

Powered by SMFPacks Menu Editor Mod