Locomotive Shell 1.0 - Extending & Modifying Locomotive BASIC

Started by zhulien, 20:11, 24 January 22

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

zhulien

I have been looking at Locomotive BASIC to see what could be done to extend it - internally rather than via RSXs.


There is a good CPC Wiki page here: https://www.cpcwiki.eu/index.php/Technical_information_about_Locomotive_BASIC#BASIC_tokens


and a good disassembly of the BASIC ROM here: http://cpctech.cpc-live.com/docs/basic.asm


The ROM is very packed with not a lot of space in it, but as a test I could setup a command in RAM and add a command to the command table, eg: QBEEP and point it to a routine in #be80 just to make it beep.


There are only 3 main tokens that are unused, and it appears the first 128 tokens are essentially paged (prefixed by &ff) between immediate tokens for non-commands (numbers and letters for strings etc) as well as functions.


Ignoring the fact the ROM is jam packed already, it looks trivial to add new functions as there are lots of unused paged tokens.


One thing stuck out to me though... How does "LINE INPUT" work?  is it just two command tokens following each other without a : between?  It seems INPUT is not a function but still a command.


I was thinking BASIC could be extended via the unused &e2, but ultimately that would mean introducing variable-width tokens for commands - but with a little luck it still 'could' work.  It could allow another 128 paged commands that are not functions.


What I wanted to look at more-so, was how to make BASIC use more memory than 42kb.  Either move all variables to the 2nd 64kb bank and have every variable access a (slow) far fetch... perhaps put the entire BASIC program behind the BASIC ROM, meaning instructions would also need to be (slow) far fetches.  That could give a very good 63kb HIMEM or such with an additional 64kb variable RAM.  Another possibility is to remove BASIC from the ROM and make it an executable that runs at &0040 with the BASIC program once again in the 2nd 64kb RAM.  Then... make the BASIC ROM into a proper OS shell.


Someone here said Vortex BASIC did use extra RAM for larger programs - so it should be possible right?  Does anyone have a disassembly of Vortex BASIC?


Locomotive Shell 1.0

I recommend we call shell programs with a different extension so they don't get confused with BASIC ones for the time being... unless we can work out a header system for them.

zhulien

#1
This is version 1.0 of the Locomotive Shell, it is currently less functional, than Locomotive BASIC.  I hope the original developers don't mind the name, it is a tribute to them for their fantastic work.


You can either replace BASIC with this ROM and put your BASIC in a different slot, e.g. 14 - or you can put this one in a different slot.  Access this using |shell, or your basic via |basic.



; tokens spare:      29
; bytes reclaimed:    1027
; current himem:   up 128 bytes

; commands removed
;   after, auto, clg, defint, defreal, defstr, deg, draw, drawr, move, mover, on sq, origin, plot, plotr, rad, release, symbol
;   tag, tagoff, troff, tron, wait, width, window, window swap, zone, fill, graphics paper, graphics pen, mask, cursor

; to replace with better alternatives
;   border, ent, env, ink, paper, pen, sound

; commands added
;   dir

Animalgril987

Hi  zhulien.
I don't think this will work. BASIC is a foreground ROM not background, and the OS will only initialise the first foreground ROM that finds, afaik.


Alan

zhulien

#3
Quote from: Animalgril987 on 20:59, 24 January 22
Hi  zhulien.
I don't think this will work. BASIC is a foreground ROM not background, and the OS will only initialise the first foreground ROM that finds, afaik.


Alan


Using EmuCPC, I have setup the BASIC ROM 1.1 as usual but in socket 5 put BASIC ROM 1.0 with |BASIC renamed to |JASIC - and it does initialise all the way to the cursor (then pauses), so I is definitely trying.


SUCCESS!!!


Yes it is very easy to make lots of BASICs, only 1 obviously is invoked at a time, eg, my BASIC 1.1 and my JASIC 1.2 work fine together, swap between them via |BASIC (with GOSUB & RETURN) and |JASIC (with GOSUB & REBURN).


All background ROMS work fine in both too, so... now a matter of reassembling rather than editing the ROM image to see how compatible a re-assembled one works.

zhulien

What's cool, is I mapped the ZX Spectrum tokens within locomotive basic I get about 90% coverage of Spectrum BASIC, so besides AMSDOS headers / Spectrum headers (if they have one) - it should be possible to run some Spectrum BASIC code in a 2nd modified Locomotive BASIC ROM.  Also, Locomotive BASIC has quite a LOT more commands than the Spectrum which means some freed up ROM space to implement those missing 10%.


I've also mapped ZX81 tokens (for a possible 3rd BASIC ROM).  However, ZX81 doesn't use ASCII and I didn't yet work out how the character set is mapped.  Needs a little more research.  Besides the non-ASCII of the ZX81 we also have close to 90% coverage in commands.

Sykobee (Briggsy)

That would be neat, having a language-level compatible version of spectrum basic running on top of the locomotive basic interpreter and editor, which are superior! And CPC typeface - don't import that awful Sinclair typeface :D.


I guess you'd need to handle the screen size difference and other aspects, and anything that relied on peek, poke, out and in to spectrum specific addresses would still fail.


Animalgril987

I stand corrected. Well done zhulien.  :D

zhulien

Quote from: zhulien on 20:11, 24 January 22
and a good disassembly of the BASIC ROM here: http://cpctech.cpc-live.com/docs/basic.asm


Would anyone know an assembler that would assemble that source?  It is way too big for MAXAM1.5 on CPC.

Bread80

I take it you've not seen my reversed assembled BASIC 1.1 at https://github.com/Bread80/Amstrad-CPC-BASIC-Source


Run it through RASM and it'll assemble straight away.


I'd like to modify it to be spread across two ROMs. Probably one for immediate mode and one for run mode. But that's a lot of work and I don't have time right now. Until then if you want to add stuff then you'll probably need to remove some stuff you don't use.


As to your questions:
Functions are prefixed with &ff. There's plenty of space to add more, but bear in mind they're in two blocks: one which takes a single numeric(?) parameter; another which takes any type of parameters. (Internally the single parameter ones are read by the interpreter and passed to the function. The other are read by the function itself).


For statements, yes, there's very few slots available. My preference would be to use one of those slots as a prefix for a second block of extended tokens. Failing that you can always replace some tokens you never use.


Re LINE INPUT: commands with multiple tokens are usually stored as multiple tokens. The LINE command will read the following token and process (or raise an error) as appropriate. Off the top of my head, SPEED INK/WRITE, GRAPHICS PEN/INK are two others that do the same.


If you want to extend the language then I'd recommend starting with the source of a similar command/function. It's not too complex once you understand it but you'll need to know the routines to call to read tokens, raise errors etc.

GUNHED

To work with two BASIC ROMs you can install both of them and have one of them parked. To park a ROM you can use any good ROM management software (ROManager f.e.). Just keep only one of them active.
BUT you MUST add the correct lower ROM = firmware to the corresponding BASIC ROM. This can be done with an external Lower ROM (using X-MEM, MegaFlash).
http://futureos.de --> Get the revolutionary FutureOS (Update: 2022.03.09)
http://futureos.cpc-live.com/files/LambdaSpeak_RSX_by_TFM.zip --> Get the RSX-ROM for LambdaSpeak :-) (Updated: 2021.12.26)

zhulien

Quote from: Bread80 on 13:04, 10 February 22
I take it you've not seen my reversed assembled BASIC 1.1 at https://github.com/Bread80/Amstrad-CPC-BASIC-Source


Run it through RASM and it'll assemble straight away.


I'd like to modify it to be spread across two ROMs. Probably one for immediate mode and one for run mode. But that's a lot of work and I don't have time right now. Until then if you want to add stuff then you'll probably need to remove some stuff you don't use.


As to your questions:
Functions are prefixed with &ff. There's plenty of space to add more, but bear in mind they're in two blocks: one which takes a single numeric(?) parameter; another which takes any type of parameters. (Internally the single parameter ones are read by the interpreter and passed to the function. The other are read by the function itself).


For statements, yes, there's very few slots available. My preference would be to use one of those slots as a prefix for a second block of extended tokens. Failing that you can always replace some tokens you never use.


Re LINE INPUT: commands with multiple tokens are usually stored as multiple tokens. The LINE command will read the following token and process (or raise an error) as appropriate. Off the top of my head, SPEED INK/WRITE, GRAPHICS PEN/INK are two others that do the same.


If you want to extend the language then I'd recommend starting with the source of a similar command/function. It's not too complex once you understand it but you'll need to know the routines to call to read tokens, raise errors etc.


Awesome, I am about 60% through manually unassembling BASIC 1.1 - but seems you have saved the work.  Amazing how many subroutines they have created in there that are purely to save a byte her and there - it certainly is one of the most jam-packed 16kb ROMS.  I am glad I spent the time doing this though as I have learned quite a lot of new things too.


I will take a look at yours because you have already done the work!  I have proven multiple ROMS can work, so my thoughts were:


1. shell ROM, remove unwanted keywords and make a OS Shell, this would be the default that we could configure our CPC with and it would have hooks to be native to UniDOS RSXs among other OS-type features, multi tasking and memory management.


2. of course Standard CPC BASIC, but it will be with a renamed RSX.


3. spectrum BASIC - token substitution and reclaim space from unused keywords to implement some spectrum ones that don't exist.


4. a Game-oriented BASIC?


I have looked at the tokens, and I need to experiment somewhat to see if we can introduce additional ones - the issue is all 256 tokens (minus a couple) are used, and are even paged for the function usage and character set - 8 bit paged token set already in there.


Splitting across 2 ROMS could certainly mean we can extend existing keywords in some ways with additional code.

zhulien

Quote from: GUNHED on 17:04, 10 February 22
To work with two BASIC ROMs you can install both of them and have one of them parked. To park a ROM you can use any good ROM management software (ROManager f.e.). Just keep only one of them active.
BUT you MUST add the correct lower ROM = firmware to the corresponding BASIC ROM. This can be done with an external Lower ROM (using X-MEM, MegaFlash).


So far my tests with multiple BASICS (not parked) have been successful, and unless the lower ROM calls the middle of BASIC directly, there shouldn't be any issues.


Upper ROM make the BASIC you want to default to, then use an RSX to get to any other BASIC you may have in your ROMboard.  It works nicely in my experiments.

zhulien

There are a lot of functions that flow to the next function to save bytes too so lots of code, the order that it is assembled in is very important.


It is interesting the way it also loops through the alphabet and individually looks through each first letter - saving lots of bytes for keywords of the same letter while also balancing the timeframe to find a keyword somewhat.


I am now trying to find one error in the original disassembly i found - where it jumps into the middle of a command... is that likely an error in the original code or just the disassembly?

Bread80

There were one or two data areas which hadn't been identified as such, and had been disassembled as data. In the BASIC ROM ISTR the infix_maths_table and infix_comparison_table at &cfed. I don't remember anywhere with a jump/call into mid-opcode though.


In the fp maths section of the firmware there is a function (process_inline_parameters at &3404) that takes the code after a call as parameters (and returns to the code after them). These data blocks certainly had some JRs landing mid-opcode.


As part of the unassembly process I wrote a utility to log code areas, data areas and jumps/calls which did 'odd' things. There shouldn't be any of them left.


The thing I can't really guarantee is addresses which have been calculated. Usually this is for data areas stored in RAM. I've looked for them, and fixed any I've found but it's impossible to know if I've caught them all. Most of these would only be a problem if you moved or modified the data storage areas.




And as for the optimisations, yes, they are a great lesson in writing dense assembler. There's a lot of JRs to a JP which save a single byte. And the compression of error messages. And there's a call to raise an error which uses the byte after the call as the error code to save having to load a register. I did find one JP which could have been replaced by a JR though. But the jump was into a neighbouring module so I think I can forgive them.

zhulien

just an update:


I can build fine, I can now create a custom shell and of course the original BASIC 1.1 just works out of the box as another ROM number (e.g. 14).


The shell I can remove or add commands without issues, other than storage within the ROM itself, but... I can put code elsewhere - such as other expansion RAM.


I made 2 lengthy attempts to split the code into two ROMS.  Actually I succeeded but it wasn't happy with them.  My first approach was to move all functions to the second ROM, but due to the way it is so tightly coupled with all theses byte saving measures - the 2nd ROM still had 8kb of baggage to support those functions.  In fact, the entire parser was in both the first and 2nd ROM out of necessity unless major reworkings are made.


The 2nd attempt was similar except I had put every command in the 2nd ROM instead of the functions - with just shells, the first ROM goes down to about 3kb (just the editor), but - as these were just POCs, I didn't patch every command to work in the other ROM, but since the baggage is in both (execution code memory management error handler etc...) it doesn't matter which ROM is actually paged in at any time for the functions to work.


I am now back to a separate shell and BASIC, I think that is the way to go - as mentioned BASIC can be extended, but - it's best to put additional code in the 2nd ROM (or RAM) for new functions, perhaps some of the easy to move functions but we need their tokens for our usage.


Now, one other option I have been playing with is removing the majority of the editor since protext can also edit BASIC programs.  Now, the shell is supposed to be a shell even though it is based on locomotive BASIC - removal of the graphics commands (plot, draw, tag, window, streams would all be a global stream etc) - the purpose being the BASIC would be enhanced for better creation of tools (batch processes and utilities).


Once I have trimmed the shell of unwanted commands / functions (and perhaps the editor?) I will share the code.

zhulien

#15

questions and ideas


i want a better startup sequence, ideally with better hardware detection... or should we just output a custom message that is user maintained?


such a custom message could alternatively be created by a separate hardware detection utility


fast text patch? and what is the best default colour scheme?


more tight integration with unidos?


companion second MCP rom POC currently allows for multi-tasking of machine code utilities in the background


some random read / write commands for files?


remove the sound commands (ent, env, sound) for a more powerful sound command?


remove border, ink, paper, pen?  perhaps just get rid of border and ink and make the border always change with the paper?


do we want to keep the line editor or use protext? we could perhaps gain another kilobyte and the following keywords:
     edit, list, load


is the default aspect ratio what we all want? 80 x 25? 
   
we could make a microfont like in the batman demo and have more columns in the same space?
   
graphics have been removed, we can add more powerful graphics if needed... but... it is a shell, this could be the basis of some
more powerful commands for easier coding of utils, and a separate 'gamebasic' for more powerful graphics?

does anyone use the remain function?

do we want to retain printer support in the shell?

new commands should use a driver system where possible, e.g. a new sound command, perhaps a play command could take    the form play "<drivername>", <other parameters> so that the same code could work parameterized across different audio    hardware.

zhulien

#16
We have just over a kilobyte of reclaimed ROM space in this ROM meaning that newly created commands can be paged into another ROM for a much more powerful command set.


I am thinking, full memory support with commands for tiles & sprites based on my existing library - unless someone has a better code set, maybe we can work together and add commands for them?  Our BASIC doesn't only have to be 16kb anymore either, we can even use more than 2 ROMS in addition to 4mb of real RAM.


Thinking of adding the following commands:


Status - Displays system statistics.
Tasks - Lists current tasks.
Spawn - Loads and runs a new task.
Kill - Kills the specified task and frees up resources.
Start - Manually start tasks.
Stop - Manually stop all tasks.
Reboot - Reboot the computer preserving the running status of tasks.
Recover - Attempts to recover the system.
Snapshot - Manually save the computer state.
Restore - Manually restore the computer state and resume tasks.




Bread80

I'd suggest a lot of the changes your making are more a firmware thing rather than a BASIC thing. If you put them in the firmware then there available for any software to use. And it's a lot easier to strip out space in the firmware - the floating point maths stuff can move to a sideways ROM. And the printer stuff sounds like a safe bet to get rid of. Maybe even the cassette routines depending on how you like to load and save stuff.

zhulien

I have modified the tokenizer, the detokenizer and the executor to cater for double-byte tokens prefixed by #e9.


It is still somewhat buggy but on a good path I think.


We have a second set of tokens almost 100 tokens from #80.  The second set of tokens can be dual purpose just as the first.  Interestingly, the standard set of tokens are actually overloaded, eg: line input, graphics pen, graphics paper - the input, pen and paper are dual purpose, stand alone and as dual words - the tokens are the same but the code is different.  So, using the 100 new tokens, we can have either new keywords that never previously existed, they can be multi-words as previously and overloaded too, however if a word is previously used, it is not an extended token - but... can form part of that overloading.


For example:  take 3 tokens


MCP
STATUS
LIST


and the following arrangements:


MCP - new token with 1 word provided
MCP STATUS - new dual word command
MCP LIST - overloading of the existing LIST command / token
STATUS - new 1 word command that is overloaded by MCP STATUS
LIST - existing list command


Of course we don't need to code all the overloadings, only the ones we actually want to use.  Effectively the CPC BASIC is a series of words that can be strung together any number of words deep to form a single command.  I think it's pretty cool what Locomotive Software did.




Current bugs:


- the detokenizer is working fine, i can verify this by manually keying the correct series of tokens into memory


- the tokenizer, it is working sometimes, I must have missed something because if I edit a code line, I still get some garbage at the end, likely missed a line size or similar


- stray syntax error, sometimes even when the command is coded and runs it still gives a syntax error, not sure why yet


- execution within a program sometimes stops at a command even though immediate mode the commands work, likely something to do with the modification to fetch the next command vs next extended command




Happy to provide the buggy source if you'd like to run a second set of eyes over it.

Powered by SMFPacks Menu Editor Mod