ShakerLand Portal

[Longshot]

Hello All

The 1.3 version of the compendium is now available here:
http://logonsystem.fr/html/engdownloadlogon.htm
English version is beeing checked by Marc.

SHAKER is currently in version 2.2 and is now composed of 3 executable modules (A, B, C).
The objective of SHAKER is to carry out extensive tests on the video circuits of the CPC.

In order to facilitate access to the results of these tests, a portal was created.
https://shaker.logonsystem.fr/



The main objective of this portal is to show the results produced by the tests of the different SHAKER modules on real machines.
(This represents approximately 1500 photos to cover all the tests for the 5 identified CRTCs.)

The data can help people working on emulator development to know the expected results when running SHAKER.
Only the 3 non-asic CRTCs are currently processed (0,1,2).

The portal also allows you to compare the results produced by some emulators with those of real machines.
The featured emulators are:
    AMSPIRIT, Version 0.687
    WINAPE, Version 2.0 Beta 3
    ACE, Version 1.22
    CPCEC, Version 2022-08-06

Thanks to
Siko for the portal.
Ced for the graphics.
Tronic for the hosting server.
Marc for compendium english checking.

DManu for the considerable work he provided on his emulator.
He has released a new version of Amspirit (0.704b):
https://forum.system-cfg.com/viewtopic.php?t=11535

[Longshot]

In addition to the SHAKER emulation tests, I post the DSK of a protection that I had written in 1989, and which crashed on certain CPCs. No emulator before Amspirit had managed to make it work.

If the protection works, it launches the REVOLOG demo.

The analysis of this problem has shown that the interrupt system on CPC is not reliable between 2 machines.
As a reference, chapter 25.7.2 of compendium 1.3 describes the analysis of this subject.
(English version will be online soon)

[Longshot]

Hey @Optimus.

Thank you for this video, which I have just discovered, which presents my work :

I too am not an expert in taking screenshots.

This is why there are some tests on the portal for which there are additional zoom screens (test B7 for example).

[Longshot]

The English version of the compendium 1.3 is available here:
http://logonsystem.fr/down/ACCC1.3-EN.pdf
Many thanks to Marc for his remarkable work.

[slingshot]

Seems the website is down
Is there any link to the English version?

[poulette73]

Hello,

You can try these links :

http://logon.system.free.fr/down/ACCC1.6-FR.pdf

http://logon.system.free.fr/down/ACCC1.6-EN.pdf

[Longshot]

Version 1.7 of the Compendium (French+English) and SHAKER 2.5 are now available here:
https://shaker.logonsystem.eu/

SHAKER is now composed of 4 executable modules (A, B, C, D).
As a reminder, the objective of SHAKER is to carry out extensive tests on the video circuits of the CPC.

On the ShakerLand portal, two new standards are presented.

SSM for screenshot management
This standard allows a program written in Z80A to ask the emulator to generate screenshots (or snapshots).

CSL for CPC Script Language
This is a new type of ASCII file format containing instructions that an emulator can interpret.
The instructions make it possible to replace the user with commands in order to control the emulation.
It is possible to insert a floppy disk, to reset the CPC, to change CRTC, to simulate keyboard input or to define different delays (in emulated µseconds).
It is also possible to override the standard name of a screenshot (or a snapshot) and generate it on demand, or launch another script.

In the context of SHAKER and the SHAKERLAND portal:
CSL files are used to automatically launch the different SHAKER test modules and run all the tests.
CSL files for each module (A, B, C, D) and for each CRTC (0, 1, 2, 3, 4) are available on the portal.

Since version 1.5 of SHAKER, the tests of the different modules generate SSM codes that allow automatic screenshot generation by an emulator.
These screenshots represent those available on the portal from photos taken on the real hardware.
Images are then produced by different emulators at exactly the same time (same µsecond in the frame).

They can also enable us to easily integrate all the images produced by an emulator automatically on the portal.
To this end, a file allows us to establish the correspondence of the 16-bit SSM codes with the images stored in the SHAKERLAND databases.

Non-regression tests on emulators
A common difficulty encountered by several emulator authors during an update concerns non-regression tests.
Indeed, a correction made in the emulation to satisfy the requirements of a particular program can generate side effects on other programs.

In order to simplify the non-regressions management between 2 versions, the following method uses at least one of the two standards, CSL and SSM.

The main method is based on the principle of the automatic screenshot generation by the emulator.
By then comparing the binary screenshots generated with those of a reference file using a script, it is possible to quickly detect any regression.

Thanks to Siko for the portal and Ced for the graphics.
DManu has just released a new version of Amspirit (0.947D), available at https://www.amspirit.fr/
Amspirit is capable of displaying this little intro on CRTC 1, available at https://www.pouet.net/prod.php?which=95354

Of course, accurate emulation is imperative to do cross dev via an emulator using up-to-date techniques.

[krusty_benediction]

Hello, I can create another thread if this is not the right place.

The former message presented SSM and CSL. These standards have been defined in the context of the saker tests, but they are useable in plenty of other contexts.
In my egocentric case: automation of actions of non-crossdev tools and mass check of CPC executables on various emulators.

@Longshot are you aware of other emulators than AmSpirit that implement these "standards" ? (quotes are only here to emphasize the point that it needs other implementation to be a standard).
 - If so, I'll be happy to get their name to use them. (I'm pretty sure CPCEC and ACE do not know such script)
 - If no, how have you tested the shaker on emulators that do not implement it ? Have you written a robot that implements at least CSL and interact with the emulator by sending the appropriate key/mouse events ?

I'd like to not have to implement a CSL interpreter to control emulators that do not implement it, but I'd like to use CSL with several emulators. thus this question

[andycadley]

I quite like the idea of CSL, but I'll be honest I don't really like solutions like SSM which require fudging Z80 opcodes as it makes it more difficult to reuse standard Z80 cores. 

And it seems like the kind of problem (similar to breakpoints) that should be definable by an entirely external control file - which would have the added benefit you could apply a control file to existing, unmodified code, to capture screenshots etc at key points.

[krusty_benediction]

I quite like the idea of CSL, but I'll be honest I don't really like solutions like SSM which require fudging Z80 opcodes as it makes it more difficult to reuse standard Z80 cores.

And it seems like the kind of problem (similar to breakpoints) that should be definable by an entirely external control file - which would have the added benefit you could apply a control file to existing, unmodified code, to capture screenshots etc at key points.

There is a single strong advantage of using an opcode in comparison to an external control file: it is executed only at the good time. But it takes space to store and time to execute (in cpu time at least to read the first 2 bytes, maybe for the other bytes too).
Current emulators do not play well with breakpoints in external files when several pieces of code are executed at the same address space, whereas only one of them needs to break. Maybe winape can overcome some cases by using counters, but i'm not aware of assemblers generating some kind of files or emulators handling the same kind of breakpoints (and counters are not enough to disambiguate cases).

[andycadley]

There are plenty of solutions to that though. More rigourous definitions of addresses such that multiple pages in the same place won't trigger it. Heck you could even allow for the definition of conditionals so that one could specify has to be executing a specific instruction at a specific address and exactly replicate what SSM is currently doing

But honestly, for the most part we're looking at something mostly for the purpose of very artificial benchmark tests and, as such, it's reasonably easy to orchestrate things such that the edge cases can be trivially avoided.

Fwiw, one of the reasons I've been thinking about this lately is because I've been playing around with CPC Analyser and it doesn't quite do what I want and that's at least partly down to being wedded to its own internal emulator. If there was a more standard way of utilising other emulators for debugging tools, it might unlock some interesting possibilities.

[Longshot]

The CSL format was created, in the context of SHAKER test automation, to allow scripting in CPC emulators.
It may not be perfect but it has the advantage of existing.

Using a common solution has undeniable advantages for those who use different emulators.
It is certainly more economical than generating YT videos, especially if the emulator can record a CSL and run in web mode to show the progress on a game, for example.

To my knowledge, three emulator authors have worked on the implementation of this format (Amspirit, Sugarbox, CLK).

I have not developed an external solution to apply the CSL on other emulators.

I will not discuss the use of Z80A opcodes to generate screenshots/snapshots (SSM format).
It was the simplest solution to test non-regressions.
The implementation of a specific (but neutral) opcode at the Z80A core level can be disabled with a test or compilation option, so it's not really a big deal. An emulator that uses this code should do so knowingly and allow its deactivation.

[arnoldemu]

@Longshot  I plan to support both these standards.

I have some questions.

CSL questions:

* 'csl_version' should this be the first non-comment/blank line in the file?

* What should be done in case of errors in the script (e.g. an instruction that is not supported), a parameter that is not supported (e.g. 'crtc_select 400', or attempt to do an action where the hardware is not configured for it (e.g. 'disk_insert' but hardware in emulator doesn't have a disc interface configured)?

* 'reset' soft/hard - I consider a soft reset to be like a reset switch is used where additional memory retains it's value and 'hard' to be like a power on/off. Therefore soft reset I also see as the reset input to a IC being applied and hard to be power on/off defaults (sometimes the same sometimes not). The document calls out the memory but does this also apply to all the hardware?

* 'crtc_select' what should be done if an invalid crtc number is specified and what does 'crtc_select pussy' do?

* Can 'disk_insert' support other disc image formats? (e.g. HFE)

* Can 'tape_insert' support other tape formats? (e.g. wav, csw)

* 'snapshot_load': There are extensions in v3 snapshot to configure the hardware with roms, load discs, load tapes. How should these be handled with this standard?

* 'wait_driveonoff' is this listening to port fa7e only or should it wait until the disc has spun up (motor on) or stopped (motor off)?

* 'csl_load': How do I determine where to load the file from if it is not specified with a path?

* 'snapshot': Which version should be used and how much information should be saved?

* I would like to note that ED,FF combination is used by Winape (and other emulators) to break into the debugger (a 'break opcode').  What is the preferred action when CSL is enabled?

CSL and SSM:
* screenshot: Some emulators may generate an image with filters (e.g. CRT filter) or they may introduce fake scanlines, or may vary in the area they capture. Is there recommendations for the configuration of the screenshot for CSL and SSM?

SSM:
* How do I determine which ED opcode is 'LL' and which is 'HH' is there a sequence to reset or do I count these starting from reset/power on?

Thank you.

[krusty_benediction]

If the standard has to evolve, it is interesting to add more commands to control the configuration of the CPC (ROMS, amount of RAM, extensions available) for use cases other than shaker; CRTC is not enough.
Details related to lookup folders should be specified from command line or emulator conf file to maximize the probability to write truly all-emulators compatible CSL (at least on those providing the ROMS/extension and so on).

A simple game use case would be a CSL file that plays alcon on playcity (so extension and some parameters) on a cpc.

A simple dev use case would be a CSL file that runs orgams (so ROMS to select) to load files from a mass storage device (so extensions to select)  and assemble it (so keys to send, but it is already done).
Next step would be to detect errors (honestly I do not see an easy/compatible way to do it unless by calling external programs on the captured screenshots), so it goes beyond just replaying stuffs for a game or an application

[Longshot]

@arnoldemu

This format is young and likely to evolve, and all constructive suggestions are welcome.

I will update the document with the missing details and a little later, with the new instructions.

In case of an error in the execution of the script, I think it is up to the emulator to stop the execution of the script, and to give the following information:
name of the script, csl line number, instruction that caused the problem and reason for stopping, version number of the script (if present) and supported version.

In my opinion, the emulator should manage a log of the last csl process.

'csl_version' should not be mandatory. It is specified in case of error if the instruction was encountered.

A hard reset should correspond to a power on/off of the machine, and therefore concerns all components.
I will clarify this point.

If an unknown crtc is selected, the script must stop and report an error. For example if a crtc is not handled.
("pussy" is just a joke to define crtc 1, which is the easiest to program to handle different demo effects)

* Can 'disk_insert' support other disc image formats? (e.g. HFE)

Of course. The extension is mandatory for this reason.

* Can 'tape_insert' support other tape formats? (e.g. wav, csw)

Same as for disk_insert

* 'snapshot_load': There are extensions in v3 snapshot to configure the hardware with roms, load discs, load tapes. How should these be handled with this standard?

If an emulator reads several snapshot versions, it should be able to report to the script manager any error encountered for the hardware configuration.
At best, it could indicate in the script the configurations required by the ".sna" in order to check, in case of error, who last modified the hardware configuration.

* 'snapshot': Which version should be used and how much information should be saved?

This should be the snapshot version managed and/or selected in the emulator.

* 'wait_driveonoff' is this listening to port fa7e only or should it wait until the disc has spun up (motor on) or stopped (motor off)?

I created this function with the idea of synchronizing the script on multiple accesses to the drive.
What can be gained by a wait associated with the drive motor rather than with the fdc?
This can be added in another version if it is of interest.

About the ED, FF code I had not paid attention because I have never used this type of code in dev.
In general, I think that it is necessary to allow the deactivation of the SSM function in the emulation.

Demo code can use this type of code
For example to transform an ED 3F (=nop,nop), into ED 49 (out (c),c) by modifying only 1 byte.
This SSM handling can be enabled when running code that specifically handles this format.

Is there recommendations for the configuration of the screenshot for CSL and SSM?

If it is used in a non-regression testing context, I think that the binary of the image generated by the emulator must be simple and identical for a screenshot requested at exactly the same time by the executed code.

This is to be able to compare the binary images contained in a reference folder easily (excluding timestamps however).
Another recommendation, for a test program using this format, is to create a reference document to indicate for each SSM code, what it corresponds to.

An excel file exists for the SSM codes generated by SHAKER.

In another context, I have no particular recommendation.

* How do I determine which ED opcode is 'LL' and which is 'HH' is there a sequence to reset or do I count these starting from reset/power on?

An SSM code is recognized when 2 consecutive series of ED xx are encountered.

For example, if we have ED 3F 00 ED 3E ED 3D , the LL code is 3E and the HH code is 3D.
The 00 (or any other code <> ED) resets the expectation of the 16-bit SSM code.

[Longshot]

@krusty_benediction

If the standard has to evolve, it is interesting to add more commands to control the configuration of the CPC (ROMS, amount of RAM, extensions available) for use cases other than shaker; CRTC is not enough.

Indeed.
I will offer instructions to try to accommodate as many configurations as possible (rams, roms, extension).

Details related to lookup folders should be specified from command line or emulator conf file to maximize the probability to write truly all-emulators compatible CSL (at least on those providing the ROMS/extension and so on).

I can ask the author of Amspirit to tell us how he handled this.