CPCWiki forum

General Category => Applications (CPC and CPC-related) => Topic started by: Prodatron on 23:48, 21 March 24

Title: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: Prodatron on 23:48, 21 March 24
Since about two weeks Prevtenet and me are working on CP\/M, which is a virtual CP/M machine for SymbOS.
That was already an old idea originally by Dr.Zed around 2007 and later by Martin464 in 2023, who already started something (and had the cool idea about the name). Unfortunately I wasn't spending enough time for the support, which was shit (I am sorry especially to Martin464!), and so in both cases, it didn't happen.

But now we started a third try with Prevtenet, and this time it looks much more promising!

Prevtenet did the BDOS implementation, which mainly means converting CP/M functions to SymShell and SymbOS functions, realising the terminal code emulation and even do the complete FCB emulation by using the SymbOS file manager (which is based on modern file handlers, not on old-school FCBs).

In the meantime I was working on the VM skeleton for handling the TPA bank and the interbank-calls between the VM and the CP/M TPA. I also started a CCP for starting CP/M apps with parameters.

Yesterday the first goal was reached, DUMP.COM is fully working and can dump itself as an example.

cpvm2.png

cpvm1.png

In full screen mode it is about 2x faster than in CP/M 2.2 and about 3x faster than in CP/M Plus (without initial loading time, which would make it even more faster).

I have no idea if this will be the same for other CP/M apps as well, and what the terminal code emulation converter will eat additional.

And so as you have a lot of layers between the CP/M program and the final output...
-> CP/M application
-> wrapper/interbank calls from the TPA-bank to the VM
-> messages between the VM and SymShell
-> messages between SymShell and SymbOS
...it is still cool, that DUMP is faster in SymbOS, running in a VM, than in CP/M 2.2 :D

It's a first test and it seems to work.
It will be fun to run - multiple - CP/M apps in a multitasking environment accessing a FAT filesystem without knowing about but working fine with it. Of course they still can access CP/M filesystem as SymbOS supports both in a transparent way.
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: d_kef on 00:02, 22 March 24
This is very impressive indeed!!!
Looking forward to working with it.

d_kef
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: Prodatron on 00:26, 22 March 24
Quote from: d_kef on 00:02, 22 March 24This is very impressive indeed!!!
Looking forward to working with it.

d_kef
Thanks a lot!! It would be nice to have you as a consultant, as you are an expert.
I guess we will stay with CP/M 2.2 compatibility, as this also makes it possible to add MSX-DOS 1 emulation, which would be cool.
Other thing is, how to do a perfect FCB file pointer (CR+EX+S2) handling. We already did everything for this but are very afraid about edge cases.
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: roudoudou on 08:14, 22 March 24
Wow! Great work!
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: abalore on 08:33, 22 March 24
It's a big step forward, since a lot of applications could be suddenly available and make SymbOS much more useful. Does it take advantage of the full memory in the system or keeps the CP/M memory limitations?
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: Gryzor on 09:00, 22 March 24
This is just next level...
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: McArti0 on 10:02, 22 March 24
Do spectre and meltdown vulnerabilities work? :laugh: ;D

Seriously, will TurboPascal work?
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: GUNHED on 11:45, 22 March 24
Nice project!  :) :)

The idea to write a quick CP/M 'emulator/simulator/version' for FutureOS I already had in the late 90ies. However back the day I decided that it makes more sense to do the important tools / apps from scratch.
Today I would also create a 63 K CP/M 2.2 version - like the one for Symbos. However, I would use the Z80-DOS (instead of the old BDOS in 8080 code) and also add the ZCPR capabilities. (The ZCPR replaces the CCP, again Z80 code instead slow 8080 code). The result would be a rather Unix-like environment. And that would be something.

Hope this project will flourish. Good luck and good work!  :)
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: Prodatron on 14:08, 22 March 24
Quote from: abalore on 08:33, 22 March 24Does it take advantage of the full memory in the system or keeps the CP/M memory limitations?
Well, yes and yes  :) It provides a 63K TPA, and this is the limit for a CP/M application because of the Z80/8080 origin. Maybe it will later shrink to 62K if we need to place some more code on TPA-bank side for speed optimization, but it's still more than you usually have in CP/M Plus.
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: Prodatron on 14:14, 22 March 24
Quote from: McArti0 on 10:02, 22 March 24Do spectre and meltdown vulnerabilities work? :laugh: ;D

Seriously, will TurboPascal work?
That's a good question. Would be cool to have some of the big CP/M 2.x programs working like WordStar, SuperCalc and yes, TurboPascal as well. Let's see...
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: abalore on 14:15, 22 March 24
Quote from: Prodatron on 14:08, 22 March 24
Quote from: abalore on 08:33, 22 March 24Does it take advantage of the full memory in the system or keeps the CP/M memory limitations?
Well, yes and yes  :) It provides a 63K TPA, and this is the limit for a CP/M application because of the Z80/8080 origin. Maybe it will later shrink to 62K if we need to place some more code on TPA-bank side for speed optimization, but it's still more than you usually have in CP/M Plus.
Fair enough! If the OS supports a virtual drive with the extra RAM it will be equally useful. It would allow to use a real word processor with plenty of space and good speed.
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: HAL6128 on 17:57, 22 March 24
Why is it faster than in the original CP/M environment? Does this mean that the CP/M port for the CPC has been bad programmed (e.g. no use of the Z80 advantages)?
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: roudoudou on 18:43, 22 March 24
Quote from: HAL6128 on 17:57, 22 March 24Why is it faster than in the original CP/M environment? Does this mean that the CP/M port for the CPC has been bad programmed (e.g. no use of the Z80 advantages)?
deutsche qualität 8)
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: d_kef on 19:56, 22 March 24
Quote from: HAL6128 on 17:57, 22 March 24Why is it faster than in the original CP/M environment? Does this mean that the CP/M port for the CPC has been bad programmed (e.g. no use of the Z80 advantages)?
The original DR BDOS and CCP, used in all official CP/M implementations, including Amstrad's one, are coded in 8080 assembly. So no Z80 advantages there.
Later Z-system alternatives were written in Z80 assembly and were 3% to 7% faster, according to their authors.
But the real delay comes from the way CP/M drives the display. CP/M was written with the standard '70s output device in mind, a serial terminal.
Amstrad's BIOS emulates a serial terminal by translating the terminal codes and rendering the memory mapped display accordingly. That's the main source of delay.
I suspect that the authors of CP\/M have redirected the BDOS and BIOS display routines to the SymbOS ones, making it considerably faster.

d_kef
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: Prodatron on 14:57, 23 March 24
Quote from: HAL6128 on 17:57, 22 March 24Why is it faster than in the original CP/M environment? Does this mean that the CP/M port for the CPC has been bad programmed (e.g. no use of the Z80 advantages)?
d_kef already explained it very good, I think it's probably mainly because of the textoutput.

Prevtenet is continuing working on CP\/M, some more programs are now running:

cpvm1.png

cpvm2.png

But a lot of others are making issue, so it will be still a hard way probably.
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: McArti0 on 18:32, 23 March 24
CPC CP/M uses typical firmware routines like BASIC. All uses LowROM.
&BDCD  TXT DRAW CURSOR
&BDD0  TXT UNDRAW CURSOR
&BDD3  TXT WRITE CHAR
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: Prodatron on 23:34, 24 March 24
Quote from: McArti0 on 10:02, 22 March 24Seriously, will TurboPascal work?
At least Digital Research Pascal seems to work now, according to Prevtenet! :o
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: genesis8 on 09:22, 25 March 24
Congratulations for all the work there !

Pity for the poor lads who try write about the Amstrad scene, too much work, too little time :-)
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: zhulien on 03:51, 27 March 24
fantasitc!  soon, CP/M dev tools can be self-hosted on Symbos!!!  

Is it able to have multiple CP/Ms running at once or just the one? Either way, fantastic!
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: Prodatron on 11:42, 27 March 24
Quote from: zhulien on 03:51, 27 March 24Is it able to have multiple CP/Ms running at once or just the one? Either way, fantastic!
Yes of course you can start as many CP/M VMs as you like, as long as you have enough memory left.

The multitasking is still active even during the execution of a CP/M program.
If a CP/M program is waiting for a char or string from the console the task will go into sleepmode until something has been received, so it won't even waste CPU time.
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: Sykobee (Briggsy) on 14:41, 27 March 24
Absolutely amazing work!

Getting the major CP/M software working would be great achievement.

Try getting Super Star Trek running in MBASIC on CP/M under Symbos :D 
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: HAL6128 on 19:16, 27 March 24
...even in a multitasking environment (Z80!!!) it is faster!
Ok, this means now programming in SymbOS through a VirtualCPM in Pascal / C-/M-Basic / Maxam2 & hearing cool AY-sound nearby, sending Emails, writing true type texts in an editor, or playing Solitair... 
Amazing.
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: m_dr_m on 22:45, 27 March 24
Fantastic! Multiplying both SymbOs and CP/M appeal!
As @HAL6128, can't wait to test TurboPascal and Modula under SymbOs.
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: GUNHED on 15:07, 28 March 24
Quote from: Prodatron on 14:08, 22 March 24
Quote from: abalore on 08:33, 22 March 24Does it take advantage of the full memory in the system or keeps the CP/M memory limitations?
Well, yes and yes  :) It provides a 63K TPA, and this is the limit for a CP/M application because of the Z80/8080 origin. Maybe it will later shrink to 62K if we need to place some more code on TPA-bank side for speed optimization, but it's still more than you usually have in CP/M Plus.
You can take a look at the famous Dobbertin 63 KB CP/M 2.2 v1.4. It allows to transfer and execute code inside the system bank. That way the 64KB barrier is broken. BTW: It also supports Time Stamps for files (using the Dobbertin Time ROM - which was cloned and enhanced by dxs, see thread in this forum). Nice feature too.
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: Prodatron on 23:04, 28 March 24
MBasic is running now! Prevtenet just reported another progress :)

cpvm.png

We added direct BIOS call support, which is used by MBasic, and now another step has been done.
The new CCP has been added as well!
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: Prodatron on 15:21, 29 March 24
Prevtenet did a first release!

Here is a temporary download link:
http://symbos.org/download/cpvm.zip

I will just post his descriptions from the Discord group here:

Quote from: prevtenetFixed enough problems that a lot of things now work more or less correctly! (Microsoft BASIC, Turbo Pascal, Wordstar, lots of random utilities...)

The drive system has been redesigned. The default configuration is now A: same folder CPvM was launched from (for a live CCP session) or the location of the CP/M program (for a direct command) B: physical drive A: C: physical drive B: ...etc
But, in a live CCP session, you can type "cd c:\symbos" (etc.) to change the physical location of the currently active logical drive (at least for drives A-D).
This lets you configure drives as needed before launching an app.

Outstanding issues: some software still crashes for unexplained reasons and various things could be optimized. As a random example, repeatedly showing/hiding the cursor when doing console input through the BIOS slows down line input for certain programs. Various small things like that.

Using DIR in the new CCP:

cpvm3.png

Microsoft Basic:

cpvm2.png

NYYRIKKI already managed to code, assemble and execute a CP/M program 100% in SymbOS with Notepad, CP\/M and M80.COM/L80.COM:

cpvm1.png

Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: genesis8 on 20:17, 29 March 24
Just as a reminder, I have the Walnut Creek CP/M CD-ROM on my web site (https://www.genesis8bit.fr/frontend/cpm/199411_CPM.zip)

It's a ZIP of 407 Mo de CP/M files (text and programs).

(https://www.genesis8bit.fr/images/news-amstrad-cpc/walnut-creek-cpm-cdrom-front.jpg)
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: roudoudou on 20:20, 29 March 24
typo in the URL, 'n' is missing in 'frontend'
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: genesis8 on 20:31, 29 March 24
Thanks corrected, then same typo on my web site :doh:
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: prevtenet on 21:29, 29 March 24
This is a development build, of course, so expect ongoing tweaks/fixes/compatibility improvements.

I see there is particular interest in Turbo Pascal, so I'll prioritize getting that working well (it does at least boot up right now, but I haven't tested it thoroughly and parts of it [like TINST] seem to have issues with the Z80 shadow registers that are still being worked on - plus various terminal I/O improvements that could still be made).

Tips for those trying this version:
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: m_dr_m on 19:44, 30 March 24
Again, that is fantastic! Mega kudos, as Schopenhauer would have said.

For TurboPascal, is there a list of current issues and/or a roadmap available?
I'm willing to beta-test, although I might not be the ideal person (couldn't make it work on regular CP/M+).
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: funkheld on 19:15, 08 April 24
the writing is difficult to see.
Something still needs to be done here.

CP/M in FutureOs would be great.

Thanks.
greeting
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: prevtenet on 22:42, 09 April 24
Quote from: funkheld on 19:15, 08 April 24the writing is difficult to see.
Can you elaborate on what you're seeing?

In general all SymShell apps look better in fullscreen mode. However, there are some other display optimizations that are still being worked on, mainly relating to apps like Wordstar that like to constantly call the BIOS to check for pending keys.

Getting 80% of CP/M software working is easy, the part that takes forever is tracking down all the tiny undocumented expectations that the remaining 20% of apps rely on...
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: GUNHED on 14:37, 10 April 24
Quote from: prevtenet on 22:42, 09 April 24
Quote from: funkheld on 19:15, 08 April 24the writing is difficult to see.
Can you elaborate on what you're seeing?

In general all SymShell apps look better in fullscreen mode. However, there are some other display optimizations that are still being worked on, mainly relating to apps like Wordstar that like to constantly call the BIOS to check for pending keys.

Getting 80% of CP/M software working is easy, the part that takes forever is tracking down all the tiny undocumented expectations that the remaining 20% of apps rely on...
I guess, Funkheld is talking about the 3 x 5 pixel characters. They can be seen on the pictures above. And they also work on emulators, I guess. However, on a real CPC monitor it's different again.
Offtopic: I don't want to derail the topic (a new can be opened anyway). But if there is interest for CP/M under FutureOS, then we can work together and do this. My part would be to provide the 'BIOS' - anything else probably doesn't need to be changed (too much).
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: prevtenet on 21:45, 10 April 24
Quote from: GUNHED on 14:37, 10 April 24But if there is interest for CP/M under FutureOS, then we can work together and do this. My part would be to provide the 'BIOS' - anything else probably doesn't need to be changed (too much).
This is probably harder than it sounds because CPvM works at the BDOS (rather than BIOS) level to support FAT file access via SymbOS file handles. So at minimum this would require changing the build process, rewriting the banking system, and replacing all the SymbOS-specific parts of the BDOS with FutureOS-specific versions. This in turn would probably require a fairly complete rewrite of the file-management system (one of the more complicated parts of CPvM) since FutureOS uses track-sector-tables rather than file handles.

I don't use FutureOS much, so I don't have the motivation to do this myself. But, CPvM is open-source, so if someone else wanted to give it a shot I'm happy to provide technical suggestions. Maybe they'd even fix some bugs 8)

Updates: compatibility is steadily improving - we're still working on a shell tweak that should dramatically improve performance for apps like Wordstar and Turbo Pascal, but stay tuned for a new stable release shortly.
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: GUNHED on 12:38, 15 April 24
That may be of common interest in the thread: How does such a file-handle look like?
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: prevtenet on 00:15, 17 April 24
I am pleased to share a formal v1.0 release of CP\/M, which may be downloaded at the temporary link: https://github.com/danielgaskell/CPvM/raw/main/cpm/cpvm_1.0.zip. Run classic CP/M-80 software on SymbOS, including Microsoft BASIC, Turbo Pascal, WordStar, dBASE, and numerous other apps and utilities. You could now run a whole 1980s corporation on SymbOS!

While there are still a few improvements planned (mainly one that will improve scroll speed in Turbo Pascal and other apps), this version is considered stable enough for general use and testing. Try it out, and let me know if you want something to work but it doesn't.

Notes:
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: HAL6128 on 07:04, 17 April 24
Amazing. It's far-fetched, but it sounds like running a VirtualBox on a Window OS (CP/M OS runs on SymbOS).
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: Edoz(MSX) on 10:14, 17 April 24
Pretty Amazing that you were able to this in SymbOS. It is so super cool! Love to test the final version now!
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: Prodatron on 15:16, 17 April 24
This is just fantastic, I was dreaming about this for a long time, and now it has come true.
Thanks for the great work, prevtenet!

An own section for CPvM has been created on the symbos.org website:

http://www.symbos.org/cpvm.htm

Including screenshots, descriptions and resources.

Quote from: HAL6128 on 07:04, 17 April 24It's far-fetched, but it sounds like running a VirtualBox on a Window OS (CP/M OS runs on SymbOS).

Prevtenet is using the correct term "compatibility layer". So it is like Wine. I still prefer "virtual machine" as it sounds cool haha :D In fact a virtual machine runs a complete different OS inside another OS, which is similiar but not exactly the same for CPvM. But you still have a complete own 8080/Z80 64K environment which is all what is visible for the CP/M program.
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: d_kef on 17:07, 17 April 24
I wonder if the compatibility layer could "expose" SymbOS functions (eg. network or printer) to CP\/M?
Anyway, this is already a huge achievement!!!
Time to play with it now  ;D

d_kef
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: Prodatron on 21:09, 17 April 24
Quote from: d_kef on 17:07, 17 April 24I wonder if the compatibility layer could "expose" SymbOS functions (eg. network or printer) to CP\/M?
Very interesting question, I guess for the printer daemon this makes sense.
But I have no idea about network.
I guess there have never been CP/M apps dealing with TCP/IP (or I am wrong?), so I just wonder how this could work.


Quote from: d_kef on 17:07, 17 April 24Anyway, this is already a huge achievement!!!
Indeed, thanks a lot to prevtenet, I still can't believe, that this is possible...
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: d_kef on 21:41, 17 April 24
Quote from: Prodatron on 21:09, 17 April 24I guess there have never been CP/M apps dealing with TCP/IP (or I am wrong?), so I just wonder how this could work.
Now I'm offended. Take a look at my signature man!!! :P
Don't you know that you can use your Cyboard with my port of the KCNet Utilities?
The KCNet utilities run on CP/M Plus as well as on CP/M 2.2
They are currently build for Wiznet's W5100S but I think it would be fairly easy to adapt them for CP\/M if for example DNS and socket functions of SymbOS were available at the compatibility layer.

d_kef
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: Prodatron on 22:02, 17 April 24
Quote from: d_kef on 21:41, 17 April 24Now I'm offended. Take a look at my signature man!!! :P
Don't you know that you can use your Cyboard with my port of the KCNet Utilities?
The KCNet utilities run on CP/M Plus as well as on CP/M 2.2
They are currently build for Wiznet's W5100S but I think it would be fairly easy to adapt them for CP\/M if for example DNS and socket functions of SymbOS were available at the compatibility layer.

d_kef
Oh, d_kef, I am really sorry!!
I completely forgot about the KCNet tools, which is funny as this is east german stuff, so somehow I should know it :D
Of course I love your W5100 based Cyboard, thanks so much, I will never give it away as it is hardware which is still able to do real "multitasking".

Yes, you are right, let's try to do it! I will have a look at it!
Title: Re: SymbOS CP\/M - run CP/M programs in a virtual machine
Post by: prevtenet on 22:15, 17 April 24
Quote from: d_kef on 21:41, 17 April 24They are currently build for Wiznet's W5100S but I think it would be fairly easy to adapt them for CP\/M if for example DNS and socket functions of SymbOS were available at the compatibility layer.
Ooh, yes, cool idea. Probably the most trivial way to do this from CP\/M's perspective would be to provide custom BDOS calls for the SymbOS messaging system, which would make most of the OS theoretically accessible, but it should also be straightforward to implement bespoke calls for the socket functions.

Simple printer access should also be fairly straightforward by connecting CP/M's basic "line output" device (which currently does nothing in CP\/M) to the printer daemon, but I didn't have either the code or hardware to actually test this, so future goals :)
Powered by SMFPacks Menu Editor Mod