I've added an extra sample program to my BASIC source code repository. This sample detokenises a BASIC program. It includes a options to show various pieces of internal data such as the line address and length, addresses of variables and goto/gosubs and the values of the tokens.
Download in the examples folder at https://github.com/Bread80/Amstrad-CPC-BASIC-Source
Detokeniser.png
That's neat - although pretty dense information on screen. Where's the =1 encoded, I presume EF means set value (=)?
How about a listing mode that shows one line with the tokens, then the next line is the BASIC source
e.g.,
02C2 46 0030 EF : C0
1020 dovaraddr=1:'show variable addresses?
That listing has most of the options turned on, so it's dense.
The &EF is the token for an '='. More details at https://www.cpcwiki.eu/index.php/Technical_information_about_Locomotive_BASIC (most of what's there is correct).
The dolinedata flag prints all the bytes in the line on a separate line preceding the listing line.
This is really about double checking I'm correctly understanding all the tokenising etc. And for anyone else who wants to understand how it all works (the source code will provide the answers to most such questions).
If anyone has questions about the tokenisation then this gives them a tool to use to do some research. I'm not really intending it as a 'final product'.
PS I learnt doing this than BASIC often inserts spaces into the tokenised code, e.g. between a GOTO and the line number. If you poke &ff into &ac00 ("program line redundant spaces flag") then it doesn't and you get a shorter listing. (And BASIC knows to insert a space when you LIST the program).
don't forget the extended token set of the shell version :D
although it is still buggy at the moment - unfortunately I'm swamped with work and it's tax time here too for me - before I can get back to it and other things.