Author Topic: Another CPC emu, with bunch of questions (about Z80 timings and FDC)  (Read 13210 times)

0 Members and 1 Guest are viewing this topic.

Offline Lone

  • Supporter
  • CPC664
  • *
  • Posts: 98
  • Country: fr
  • Liked: 181
  • Likes Given: 74
Hello guys,


As a newcomer in this forum, let me introduce myself : My name is Thomas, and (as lots of you I can believe) I used to have a CPC 6128 in the late '80, when I was about 13.
Since that day, I stayed in computer (I actually work on software by now)...


And at the end of last year, I decided to code my own emulator, to bring back these old memories (I don't have any CPC at home, due place, wife and kid's factor).
I know lot's of emulator exists, but it was like a challenge to make my own work (and I must admit that it's a big thing to start to run old games on "my" hardware).


By now, my emulator (call it "sugarbox", until I find a better name) run most of non-protected games, if they don't intend to much on hard timing of CRTC and gate array..


So that's the point : I try to find informations about timing and FDC.


1/ The Z80 timings : I fond on this forum some informations about various tests. Lot's of them are now passed, but one of them since to be a bit far from my understanding : It's the "Interrupt Wait test", from the "plustest" of Richard Wilson. I don't understand what is supposed to be test, and how (so, i don't have a clue about why some opcodes are ok, and some other aren't...).


If anyone can give me some advice about it, it would help me a lot !


2/ The infamous FDC.... I try to understand what kind of datas are returned by this command : I understand that it returns the track data, from first sector to last. So, I assumed that it begin at the first data of the first sector, but I don't know how are manages other "extra" data, between two sector : Does it return everything until the enf of the track  ? or some of these datas are not returned ?




3/ The copy protections... I found also some words about them, but nothing about how it is supposed to work... Does these informations exists ?
I'm currently trying to make work "Chicago 90'" and "Le Necromancien", which are maybe a bit tough for a first try....




I'm joining a very alpha of this emulator, in case anyone want to give it a try (or want to test this Interrupt Wait test and give me a clue !)
Be aware it's far from stable, not really user-friendly (in fact, not user friendly at all), and not really fast.
It's designed for windows, and should work with version from XP until 7 (not tested on 8)
Unzip and run "Sugarbox.exe"
It support command line arguments :
sugarbox [-drivea path_of_dsk] [-command "command to start"]
sugarbox.exe -drivea "c:\dsk\ch90.dsk" -command run"ch90"




Thanks for your attention

 [ Invalid Attachment ]

Offline TotO

  • 6128 Plus
  • ******
  • Posts: 3.578
  • Country: fr
    • ?area=showdonations;u=4
  • Liked: 2705
  • Likes Given: 1606
Hello!

I suggest you to use the M.E.S.S. FDC 765 emulation part, as it's more accurate than 90% of the existing CPC emulators and allow to run protected games as well.
"You make one mistake in your life and the internet will never let you live it down" (Keith Goodyer)

Offline arnoldemu

  • Supporter
  • 6128 Plus
  • *
  • Posts: 5.335
  • Country: gb
    • Unofficial Amstrad WWW Resource
  • Liked: 2261
  • Likes Given: 3478

1/ The Z80 timings : I fond on this forum some informations about various tests. Lot's of them are now passed, but one of them since to be a bit far from my understanding : It's the "Interrupt Wait test", from the "plustest" of Richard Wilson. I don't understand what is supposed to be test, and how (so, i don't have a clue about why some opcodes are ok, and some other aren't...).
It's caused by the way the CPCs hardware stretches the timings of the instructions.
CPCs hardware forces all memory accesses to be on 1microsecond intervals.
Some instructions don't fit into this pattern, and have "extra" time at the end of them - one of these is INC BC. It takes 1us to read the instruction, and less than 1us to process it. Leaving some time left over. Normally this time is used up.

But, when an interrupt comes, it can be acknowledged immediately before the next 1us comes. The interrupt uses up some of this time, and so it can appear to happen earlier than expected.

On other instructions there is no free time, so the interrupt is forced to take 1us cycle for acknowledge, and we see that.

This explains these differences.

If you look at the official Zilog instruction manual and see how the instructions are divided into T states, you can guess at which will do this and why.



2/ The infamous FDC.... I try to understand what kind of datas are returned by this command : I understand that it returns the track data, from first sector to last. So, I assumed that it begin at the first data of the first sector, but I don't know how are manages other "extra" data, between two sector : Does it return everything until the enf of the track  ? or some of these datas are not returned ?
Read the official FDC datasheet, then read the data sheets of the compatible FDCs this will tell you quite a lot, but also be prepared to make a test program because there are some things that are not described in the documents. When I made Arnold, I was already familiar with how the FDC worked so that solved a lot of things, but then when I made some tests I found a lot more interesting things.




3/ The copy protections... I found also some words about them, but nothing about how it is supposed to work... Does these informations exists ?
I'm currently trying to make work "Chicago 90'" and "Le Necromancien", which are maybe a bit tough for a first try....
No information exists to explain exactly how these work.
Good luck with Le Necromancien, this requires you emulate the lowest level MFM disc structure to read it correctly. I would start with something a lot more simple and work from there.





Thanks for your attention
Continue with your work, because it will also help if you want to use the knowledge to make games and demos for this computer :)

My games. My Games
My website with coding examples: Unofficial Amstrad WWW Resource

Offline Lone

  • Supporter
  • CPC664
  • *
  • Posts: 98
  • Country: fr
  • Liked: 181
  • Likes Given: 74
Hello again....

First, thanks for your responses.Some time passed, and I at least had the time to try some tricks, to fully understand how this FDC works.For the memory, here is what I found out.


I finally manage to run "Chicago 90", with the original game dump (GAP protection inside), thanks to the coming back of cpc-power (
[size=78%]http://www.cpc-power.com/index.php?page=detail&onglet=dumps&num=548[/size][/font] ). I had some problems with it, due to my handling of the overrun data (data made ready by the FDC, but not read by the CPU).

When I finally added a fifo buffer (with a size of 16 bytes... Really empiric value : 10 bytes causes overflow, and 16 seems to be enough in this cas of figure), to hold data, my problem disapears, and the game loads then run correctly.

So it seems that I finally found a solution.... But what's a bit unexpected, is that I didn't read anything about any kind of buffer within the FDC...


Any clue ?



Offline ralferoo

  • Supporter
  • 6128 Plus
  • *
  • Posts: 969
  • Country: gb
  • Liked: 581
  • Likes Given: 222
I suggest you to use the M.E.S.S. FDC 765 emulation part, as it's more accurate than 90% of the existing CPC emulators and allow to run protected games as well.
Interesting. I'm going to take a look at this myself as my FDC implementation still feels a little ropey... :(

Offline arnoldemu

  • Supporter
  • 6128 Plus
  • *
  • Posts: 5.335
  • Country: gb
    • Unofficial Amstrad WWW Resource
  • Liked: 2261
  • Likes Given: 3478
The FDC doesn't have a fifo.

It stores the data from the sector in it's data register 1 byte at a time.

The FDC remembers if the byte has been read by the CPU or not.

If the byte hasn't been read and it wants to put another byte into the data register this is the time it will give the overrun error condition.

So, as long as you read the data quickly, this condition doesn't happen and overrun error never comes.

When sector data is read, a byte comes every 32us.

My games. My Games
My website with coding examples: Unofficial Amstrad WWW Resource

Offline Lone

  • Supporter
  • CPC664
  • *
  • Posts: 98
  • Country: fr
  • Liked: 181
  • Likes Given: 74
The FDC doesn't have a fifo.

It stores the data from the sector in it's data register 1 byte at a time.

The FDC remembers if the byte has been read by the CPU or not.

If the byte hasn't been read and it wants to put another byte into the data register this is the time it will give the overrun error condition.

So, as long as you read the data quickly, this condition doesn't happen and overrun error never comes.

When sector data is read, a byte comes every 32us.



That's were it's weird...
My understanding of the datasheet and all I read around is exactly what you're explaining. Even the 32us, which was the value I used for my FDC to tick (and eventually to overrun).


Anyway, with this game Chicago 90', I encounter overruns...


I think i'll have to disassemble a bit the code to understand where my timings are bad (which is the only solution i'm currently thinking of).

Offline ralferoo

  • Supporter
  • 6128 Plus
  • *
  • Posts: 969
  • Country: gb
  • Liked: 581
  • Likes Given: 222
When sector data is read, a byte comes every 32us.
I thought it was 13us for MFM data and 26us for FM data. I could be wrong though as I've not actually measure it - my emulator just returns the next byte as soon as the CPU has consumed the previous one.

Offline Lone

  • Supporter
  • CPC664
  • *
  • Posts: 98
  • Country: fr
  • Liked: 181
  • Likes Given: 74
I thought it was 13us for MFM data and 26us for FM data. I could be wrong though as I've not actually measure it - my emulator just returns the next byte as soon as the CPU has consumed the previous one.


I believe that these values are the response time to interrupt. In our case (no interrupt), these values have no meaning.


Offline ralferoo

  • Supporter
  • 6128 Plus
  • *
  • Posts: 969
  • Country: gb
  • Liked: 581
  • Likes Given: 222

I believe that these values are the response time to interrupt. In our case (no interrupt), these values have no meaning.
Well, except that the data rate of the encoded data on disk is 500,000 bits per second, which makes 250,000 bits per second after MFM decode, which is 31250 bytes per second, which is... oh 32us. Oh you guys were right all along! ;)

Offline arnoldemu

  • Supporter
  • 6128 Plus
  • *
  • Posts: 5.335
  • Country: gb
    • Unofficial Amstrad WWW Resource
  • Liked: 2261
  • Likes Given: 3478
It depends on the master clock to the FDC. Those timings seem more for 8Mhz clock - and in fact a lot of the timings in the datasheet seem to imply an 8Mhz clock is being used. But on CPC and Spectrum it's 4Mhz.

MFM is more of a data format on the disc, FM is slightly different.

This is how I understand it in terms of PC disc formats:

High Density, MFM, 16us per byte.
Medium Density, MFM, 32us per byte.
Low Density, FM, 64us per byte.

My games. My Games
My website with coding examples: Unofficial Amstrad WWW Resource

Offline arnoldemu

  • Supporter
  • 6128 Plus
  • *
  • Posts: 5.335
  • Country: gb
    • Unofficial Amstrad WWW Resource
  • Liked: 2261
  • Likes Given: 3478

I believe that these values are the response time to interrupt. In our case (no interrupt), these values have no meaning.
Yes your probably correct. It probably means that when the cpu has received the interrupt it has those times before the next byte arrives. It probably takes into account a certain time for the interrupt to be acknowledged and the interrupt routine to be started.

I don't know where they came up with 13us and 26us myself, unless they are worse case, or for 360rpm discs. Perhaps they are the timings for 8" media and it's rotation speed?
My games. My Games
My website with coding examples: Unofficial Amstrad WWW Resource

Offline ralferoo

  • Supporter
  • 6128 Plus
  • *
  • Posts: 969
  • Country: gb
  • Liked: 581
  • Likes Given: 222
Yes your probably correct. It probably means that when the cpu has received the interrupt it has those times before the next byte arrives. It probably takes into account a certain time for the interrupt to be acknowledged and the interrupt routine to be started.
I doubt it's the interrupt time, because you'd have the time of the gap between the IDAM and the DAM before the FDC would start giving you data, which is more likely be about 13 bytes, so about 400us.

Unless of course, the interrupt occurs when the sector data starts... I guess I'm gonna have to start measuring things! :O
Quote
I don't know where they came up with 13us and 26us myself, unless they are worse case, or for 360rpm discs. Perhaps they are the timings for 8" media and it's rotation speed?
They're definitely in the datasheet for the 765. That's where I read them at least.

Offline arnoldemu

  • Supporter
  • 6128 Plus
  • *
  • Posts: 5.335
  • Country: gb
    • Unofficial Amstrad WWW Resource
  • Liked: 2261
  • Likes Given: 3478
I believe the fdc would trigger an interrupt for each data byte (if not using DMA mode) and then again when the result phase is ready.
If using DMA mode, it would trigger a DMA signal which would automatically cause the byte to be read and transferred (DACK??).
Of course, the fdc inside the cpc is run in non-DMA mode so those interrupts are triggered, but not connected on cpc. So we must read the status register to see what is happening. This explains why there is so much reference to interrupts and things like that in the fdc docs.

Terminal Count (TC) is another pin which is not used on CPC which is normally used on PC to interrupt data transfer - e.g. to read x bytes from a sector then trigger TC to tell FDC you're done.

So many features which are not used on CPC.
My games. My Games
My website with coding examples: Unofficial Amstrad WWW Resource

Offline arnoldemu

  • Supporter
  • 6128 Plus
  • *
  • Posts: 5.335
  • Country: gb
    • Unofficial Amstrad WWW Resource
  • Liked: 2261
  • Likes Given: 3478
Unless of course, the interrupt occurs when the sector data starts... I guess I'm gonna have to start measuring things! :OThey're definitely in the datasheet for the 765. That's where I read them at least.
Yes I read them there too.

The datasheet talks as if you are using a certain clock and certain setups.

Some of the timings I have measured to be longer with a 4Mhz clock - it does state this somewhere in the docs too.
My games. My Games
My website with coding examples: Unofficial Amstrad WWW Resource

Offline Lone

  • Supporter
  • CPC664
  • *
  • Posts: 98
  • Country: fr
  • Liked: 181
  • Likes Given: 74
When I investigate a bit further, I found out that my overrun problems are caused by interrupt not disabled during the protection routine reading :
Putting stop breakpoints inside it, I can see that, after some reading, an interrupt occur, then send me to 0038, which is a jump to B941... Which begin to read from the gate array... then my next byte is ready from the FDC, the previous one is not read, and overflow occurs.


Well well well....


So, I can think of two bugs in my code :
- Either I don't handle correctly interrupt, and it should be disabled. But when I use debg with WinAPE, it run as my code, with interrupt enabled.
- Overrun during a ReadTrack command don't stop the Execute phase to enter the Result phase (and data are still output, which can cause eventually the GAP data to be read). Anyway, a test with overrun being 'only' informative (on status 1 in Result phase) fails.


In fact, the only way to make it work, is to add this buffer fifo : I put a screen here (sorry, no cut/paste from disassembly available currently...)
 [ Invalid Attachment ]


We see the code from 806A to 8088 : exactly &207 values are stored in 8089 (the last one being the one to check for the protection). Which means that if one or more bytes are missed (dur to overrun, due to interrupts), the last data written would be far more than the 207th one.


The sector data show us that there is 34 GAP data in this sector. For the protection to be ok, it means that the CPU can't miss more than 27 bytes, which is only 27*32 = 864 us in various interrupt.
512 data read from the disk is 512*32 =  16384 us, which means that not less than 16384 / 3328 = 4 interrupts occurs. Well, all of this seems to be a bit too random.


In fact, after all this, i take a look at the FDC from MESS, which seems to use an internal Fifo as well...


Any more clue ?


Offline ralferoo

  • Supporter
  • 6128 Plus
  • *
  • Posts: 969
  • Country: gb
  • Liked: 581
  • Likes Given: 222
We see the code from 806A to 8088 : exactly &207 values are stored in 8089 (the last one being the one to check for the protection). Which means that if one or more bytes are missed (dur to overrun, due to interrupts), the last data written would be far more than the 207th one.
That is the correct number of bytes for a single sector (512 from the execution phase and 7 from the result phase).

Quote
In fact, after all this, i take a look at the FDC from MESS, which seems to use an internal Fifo as well...
The 765 won't have a FIFO or it'd be documented in the manual (and also less critical to read it at the correct time). Modern PCs use the 82077 (or something similar) which is broadly compatible with the 765 but also includes a 16 byte FIFO.

Offline Lone

  • Supporter
  • CPC664
  • *
  • Posts: 98
  • Country: fr
  • Liked: 181
  • Likes Given: 74
That is the correct number of bytes for a single sector (512 from the execution phase and 7 from the result phase).
The 765 won't have a FIFO or it'd be documented in the manual (and also less critical to read it at the correct time). Modern PCs use the 82077 (or something similar) which is broadly compatible with the 765 but also includes a 16 byte FIFO.


In fact, the protection used by this game is a GAP type protection.
That means there is a special track and sector. In Chicago90, it's track 39, sector 41.
This sector (which you can see here : [size=78%]http://www.cpc-power.com/SectorData.php?fiche=548&slot=7&rang=0[/size]) contains special GAP data (the F7 you can see at the end).


The trick is to read with a "Read track" command, using a size of 3 (meaning 1024 bytes for a sector). After the 519th byte read, and copied to memory location 8089, the code stop reading incoming data (which cause a true overrun in this case).
All data that are after the 512 first bytes are not sector data, but CRC then GAP data.


Checking that the 7th byte after the sector real end is 7F and not the standard 4E value, is the protection.

Offline arnoldemu

  • Supporter
  • 6128 Plus
  • *
  • Posts: 5.335
  • Country: gb
    • Unofficial Amstrad WWW Resource
  • Liked: 2261
  • Likes Given: 3478
Correct, read track reads past the end of the sector and reads the gap bytes.
correct, if it stops reading an overrun will be signalled.

The gap bytes are special here.

So to read this kind of protection you need to have very accurate fdc emulation and reproduce the raw mfm data exactly to get the perfect results.
My games. My Games
My website with coding examples: Unofficial Amstrad WWW Resource

Offline Lone

  • Supporter
  • CPC664
  • *
  • Posts: 98
  • Country: fr
  • Liked: 181
  • Likes Given: 74
Correct, read track reads past the end of the sector and reads the gap bytes.
correct, if it stops reading an overrun will be signalled.

The gap bytes are special here.

So to read this kind of protection you need to have very accurate fdc emulation and reproduce the raw mfm data exactly to get the perfect results.


In fact, it's exactly what I planned to do :


My paradigm is quite simple : When I read the dsk file, I rebuild the full disk structure, in a big buffer, including everything : GAP, CRC, Sync,etc.
My FDC "tick" every 32us. In this tick, if the motor is on, the head advance, and the next byte is then used.


During a read track command,I check if the previous byte has been read. If it's not the case, overrun is set. Otherwise, it is prepared to be available for the next IN command from the CPU.


With this method, most of the disk are read correctly, even this GAP protection. The problem I have is that overrun is always set, due to the CPU that didn't read on time the FDC data...


I'm checking my various counter right now, expecting an error that would cause my timings to be wrong.


Offline PhilZeVibe

  • 464 Plus
  • *****
  • Posts: 469
  • Country: fr
    • CPCBox - CPC emulator in JavaScript
  • Liked: 163
  • Likes Given: 187
My paradigm is quite simple : When I read the dsk file, I rebuild the full disk structure, in a big buffer, including everything : GAP, CRC, Sync,etc.
My FDC "tick" every 32us. In this tick, if the motor is on, the head advance, and the next byte is then used.

That's the correct approach to emulate the FDC. But I think only ACE emulator has done it in full extent as of now.
The big issue is that the EDSK disk file format is not really adequate for that approach.

EDSK has multiple profound issues. Here's some off the top of my head:
  • It doesn't contain all the necessary information to rebuild the full disk structure. An example of missing information is CRC. Sure you can recalculate it from the data, but what if the CRC information in the original disk doesn't correspond to the data? Then, after EDSK conversion, that CRC value is lost forever.
  • It has redundant information. That lead to errors about how to handle conflicting information in some EDSK files.
  • It also contains purely useless information, like the FDC flags.
  • The format also allows physically impossible track descriptions, encouraging emu authors to do quirky FDC emulation to handle them.
I think that this flawed disk file format is really the main culprit behind all the bad dumps floating around on the web, and about the bad FDC emulation found in mainstream CPC emulators (mine included).
« Last Edit: 11:59, 25 July 13 by phi2x »
Atwood's law (2007): "Any application that can be written in JavaScript, will eventually be written in JavaScript."

Offline Lone

  • Supporter
  • CPC664
  • *
  • Posts: 98
  • Country: fr
  • Liked: 181
  • Likes Given: 74
After some testing :
It seems that there is, in fact, no buffer.


But the reading is not stopped by an overrun : Data continue to be outed until the end (with the bit overrun set).


I'll attach here a test, if someone can run it on a real CPC and post a result, it will help.


 [ Invalid Attachment ]




Concerning the problem of Chicago90, there is in fact 34 byte in the GAP data, so it is possible to miss 33 bytes due to interrupt, and complete the test.
(512+7)*32 byte to read = 16608, which is about 5 interrupts (16608/3330 = 4.97...)
33*32 = 1056us, which should be enough to run few interrupts (200 us per interrupt in this case).




Offline Lone

  • Supporter
  • CPC664
  • *
  • Posts: 98
  • Country: fr
  • Liked: 181
  • Likes Given: 74
After being able to run this test program on a real CPC, I can finally have a conclusion on this particular point :


My test program do a Read Track of the sector C1 of track 1. It reads the 255 first bytes, then stop, and display the 20 first one on screen.
The second run do the same, but add a 60-70us delay between "data is ready" on main status register, and the effective read through the IO port.


Results are following :


Track data are :
00 54 46 44 43 20 20 20 20 42 49 4E 00 00 00 07 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 E5 ......

Command is here : &42, &00, &00, &00, &C1, &02, &c1, &2a, &ff

First case reads 255 bytes, and display  :
00 54 46 44 43 20 20 20 20 42 49 4E 00 00 00 07 02 00 00 00
Which is expected : It's exactly what's on the sector


Second case reads 156 bytes, and return :


46 20 20 4e 00 00 00 00 00 00 e5 e5 e5 e5....


Which is more or less one byte every three.


I think we can conclude that :
- FDC don't stop the read track command if an overrun is detected.
- FDC on the CPC had no buffer.


To relate this on Chicago 90 :
It seems that my previous assertion is right. Must be a timing problem on my code....
So let's return to investigations !




I wish to thanks to fano for the test run, by the way.

« Last Edit: 12:50, 27 July 13 by Lone »

Offline TFM

  • Visit the mysteries of the CPC at www.futureos.de
  • Supporter
  • 6128 Plus
  • *
  • Posts: 9.899
  • Country: aq
  • Space Chicken for FutureOS is free!
    • index.php?action=treasury
    • FutureOS - The revolution on CPC!
  • Liked: 1976
  • Likes Given: 4650
Right, the FDC765 has no buffer, a byte must be read every about 25 ys. It's all in the datasheet of the Floppy Buch ;-)
 
TFM of FutureSoft
Also visit the CPC and Plus users favorite OS: FutureOS - The Revolution on CPC6128 and 6128Plus

Offline Lone

  • Supporter
  • CPC664
  • *
  • Posts: 98
  • Country: fr
  • Liked: 181
  • Likes Given: 74
Last update and final point about Chicago 90.


After being able to test on a new dump, all works correctly.
It was the dump that wasn't correct (not enough GAP data : 34 is not enough...)
The correct number is 80 bytes, so far enough to run few interruptions during the process.


Thanks to Maxit for the correct dump.


Anyway, the datasheet was not clear (at least for me) about the fact that an overrun will not stop the data output.
Now, we have a proof !