Universal Serial Interface for Amstrad CPC (a.k.a USIfAC)

[nicf82]

I had exactly the same problem when i first involved with esp8266, it seems there was a need to encapsulate each data packet in order to send and "peel off" the encapsulation to get data.  But after a bit of study, it seems there is a way you can send/receive bytes directly without needing "encapsualtion" (quoting from Usifac manual):

Thanks for that ikonsgr, thats really very helpful. Also the link to the proper docs will help!

Cheers
Nic

[VintageAdvantage]

Load ~70kb game in 3.7seconds e.g. ~20kb/sec transfer rate

Turbo! 

[ikonsgr]

And another small "teaser" of the the upcoming USIfAC II!
I've developed extra code to utilize this very cheap Usb storage device host module, and now, you can load programs and games from usb sticks too! And as you can see in the video, still with SUPER fast speeds! 

[GUNHED]

Great! Is there such a ROM for the regular USIfAC?

[ikonsgr]

Unfortunately there is not. You see, the new USIfAC II, is actually a "from the scratch" project, based on a much more adavanced (but still 8bit) 18F47Q10 PIC MCU, although the schematic of the board is simpler than the previous:
[attach=1,msg190474]

As you can see, there is only a  (much larger) 40pin Chip, which provides access to  full 16bit address bus and other CPU signals too. Also, this PIC is equipped with 8 Configurable Logic Cells, small  programmable logic modules which can be very useful in address and signal decoding, so no need for extra logic chips! 
I don't want to... "spoil the surprise" but the new USIfAC II will provide so much more functionality that maybe the name can't reveal:
-Dozens new RSX commands (yes, EVERYTHING is now executed directly, no need to write any code for direct function any more,also all previous functions for file rtansfer, format, dsk image copy and many more are converted to RSX commands executed directly from basic prompt...  ),
- Full  ROM and 765 FDC emulation,
- 1kb eeprom for saving  your own code,
- A new "direct function" that practically converts the HDD of your PC to an Amstrad HDD, providing a small group of RSX commands for changing directories, catalogue dirs, load/run/save programs/games (at incredible speeds of ~18-20kb/sec e.g. 4-5times of actual disk drive/gotek drive speed),even delete files, directly from Amstrad!
- Full read/write access to dsk images that can be also selected directly from amstrad! 
And ,finally, there will be a special "mode", which practically converts a CPC 464 to a "CPC 664", providing full Amsdos & 765FDC emulation simultaneously, thus giving the 464 full access to any dsk image (apart from the super fast direct file loading of course)!

[VintageAdvantage]

Impressive!

Especially if the DSK and 765 FDC emulation works out perfectly. Gotek and HxC feel a bit slow and dated these days on the CPC.And M4 has many issues with DSK images since it does not emulate the FDC on a low-level AFAIK (but goes over firmware functions). I must say I don't care much for "serial DSK loading over Serial / Wifi" loading, or having your PC act as a file server or that stuff, I just prefer to have all the DSK images on a USB drive.

Great job! There may be a market for that device, even if most folks will already have a Gotek / HxC / M4 / Dandinator...

[ikonsgr]

Impressive!
Especially if the DSK and 765 FDC emulation works out perfectly.

Well, from the numerous tests i have made so far, it seems that it can read data format and system format images, with more than 9 sectors/track or more/less than 512bytes per sector. The emulation is done in low level, of getting and responding directly to 765 FDC commands through the controller's I/O ports, so amstrad "thinks" it communicates with a dsik drive!    Also, i've included more advanced dual head image support and multi sector reads, which are supported with parados. I've tested them using  400kb and 800kb parados images and seem to work ok (both read and write). The only images i couldn't load so far, was the "sinlge disk" dual side/80tracks Batman forever demo & orionprime images (they give an incompatible system error or something), and a couple of game images with weird copy protections, but fortunately you can find everything in "cracked" or normal 180kb dsk image versions, so this really wouldn't be a problem.
For the moment, this advance FDC emulation is only available for direct connection to PC (although i intend to make it work with Wifi modules too, so no need to have a pc or laptop near amstrad  ), and i may expanded to work with the usb storage device host module too (although it will probably support only usual 180kb dsk images)

Gotek and HxC feel a bit slow and dated these days on the CPC.

Tell me about it. After months of testing and loading 100's of games,i've been "Addicted" to the super fast loading of ~20kb/sec ,so now, even the turbo track loaders of some games (that can ~double the speed of a disk/gotek drive from 4-5kb/sec to ~8-9kb/sec) seem veeeeery slow!    It's really an amazing thing to have a full 42kb game load on a CPC 464 in a couple of seconds!    I guess this kind of speed can only be beatten by a Rom board 
Of course the super fast speed is acheived only with direct loading of files and not through FDC emulation (meaning you need to have the actaul game files expanded from  dsk containers, fortunately there is a batch function  inside CPCLOAD  utility which can do exactly that, expanding 1000's of games in separate folders in less than a minute   ), which practically, is a bit faster than an actual disk drive on CPC 6128 and slower on a CPC 464 (obviously because with a cpc464 the MCU must emulate both Amsdos and FDC controller at the same time!).

There may be a market for that device, even if most folks will already have a Gotek / HxC / M4 / Dandinator...

Well, i'm afraid that (and despite that USIfAC II would be MUCH cheaper than the M4 board, selling price will  be ~15euros+shipping), Duke has already "catch" almost all the... "available market" , so, i don't expect much... "commercial success"!   
BUT,  as i've already noted about the previous USIfAC in the past (and despite the fact that i literrally spent 1000's of hours for more than a year developing this board), i made this  project MOSTLY as a hobby, a fun of making something creative & useful for the community, so i really don't care about "commercial success"    (although i confess i wished USIfAC to be a bit more succesfull, just for giving a motive to the software community, to develop new LAN/WAN games for amstrad... )

[VintageAdvantage]

This is great! Of course, "commercial success" is almost impossible with CPC projects. That's not why we are doing it for. But, even it is not a "commercial success", it is nice to at least have some recognition and users. After all, it's a labor of love and it hurts if, after hundreds of hours of work and love put into such projects, if a project is being ignored by the community for reasons such as "not inventend {here, in <country xyz>, by <scene member xzy>}" or the like.

I'll buy one for sure! Hope others will too. Great job!

[Skunkfish]

USIfAC 2 sounds very interesting, at that price I'll definitely be grabbing one when it's released.
I take a look at the first USIfAC in the new issue of my fanzine, I had no idea that a follow-up was in the works!

[Liartes]

Really interesting project, I have not much usage for serial/networking (yet) and prefer USB solution. Loading games from expansion slot would be less messy than using an external gotek.
Count me in for the release batch !

[GUNHED]

WoW! Very impressive!!!


As far as I understand (at lunch break now) it can emulate the FDC 765. That would be something awesome! Especially if there could be a version with different I/O addresses. This way the Vortex FDC765 could be emulated.


Would it be possible to emualate the FDC765 using the I/O addressed from the Vortex disc-interface?

[ikonsgr]

Does Vortex use the same 765 floppy Controller? And what port is using?
The emulation was based on this document, and it was REALLY hard and time consuming to make it work even in the first place! The "problem" is that 765 FDC does not simply  receive a command byte sequence and gives an "answer", but CONSTANTLY writes to two differents ports (main status register port AND  data register port) for EVERY SINGLE byte that it receives!  And in all this complicated "dicussion" the mcu (which emulates FDC) must give the expected  responces to amstrad for EACH and EVERY different situation! Even one wrong responce byte and everything is crashed! 

[GUNHED]

Tell me about it. After months of testing and loading 100's of games,i've been "Addicted" to the super fast loading of ~20kb/sec ,so now, even the turbo track loaders of some games (that can ~double the speed of a disk/gotek drive from 4-5kb/sec to ~8-9kb/sec) seem veeeeery slow!    It's really an amazing thing to have a full 42kb game load on a CPC 464 in a couple of seconds!    I guess this kind of speed can only be beatten by a Rom board 

And by FutureOS which transfers 20KB/second (in real live). For example read an complete 180 KB Data disc in 9 seconds. 

...(although i confess i wished USIfAC to be a bit more succesfull, just for giving a motive to the software community, to develop new LAN/WAN games for amstrad...   )

Regarding available hardware: The USIfAC is at present day the best (and maybe only) solution for serial communication. How much have been sold? I took some by myself and they are awesome!

Of course I'm also very interested in the USIfAC II - especially for this awesome cheap price. It may be even better of make a CPC network (I started doing that with the USIfAC first, but then I hat too much to do at work - now I got more time again). Awesome to see such a big jump in evolution from U1 to U2 :-)

Does Vortex use the same 765 floppy Controller? And what port is using?
The emulation was based on this document, and it was REALLY hard and time consuming to make it work even in the first place! The "problem" is that 765 FDC does not simply  receive a command byte sequence and gives an "answer", but CONSTANTLY writes to two differents ports (main status register port AND  data register port) for EVERY SINGLE byte that it receives!  And in all this complicated "dicussion" the mcu (which emulates FDC) must give the expected  responces to amstrad for EACH and EVERY different situation! Even one wrong responce byte and everything is crashed! 

Thank for responding!    I'm very well aware of FDC programming. So now to the great thing about the Vortex controller. Everything is the same like with the CPCs FDC765, the only differences are the addresses and the fact that the Vortex FDC765 can physically connect four drives - and that's awesome, that way one can use 3", 3.5" and 5,25" at the same time.

Now the FDC status for Amstrad is &FB7E, and for Vortex it is &FBF6
The FDC data port for Amstrad is &FB7F, and for Vortex it is &FBF7

If the USIfAC II would be able to "switch" these I/O addresses or maybe support both sets / both FDCs - that would be great.

The great thing is also that we have the VDOS ROM the use the additional FDC from Vortex, and it's also supported by FutureOS. There is even CP/M for it.

[ikonsgr]

And by FutureOS which transfers 20KB/second (in real live). For example read an complete 180 KB Data disc in 9 seconds. 

I don't think i get this. Future os is software not hardware, and also,i suppose you mean transfer from somewhere else than a real floppy disk ,because transfer data from a real disk can be maximum ~8-9kb/sec (and this ONLY with custom fast track loaders, usally speed is ~half of that).

Regarding available hardware: The USIfAC is at present day the best (and maybe only) solution for serial communication. How much have been sold? I took some by myself and they are awesome!

So far (that is, for the last ~1,5years USIfAC is available), i've sold ~120 ready boards and ~dozen as DIY. Duke i think mentioned some time ago that M4 board had  ~800 boards sold, so i suppose now it  must exceed 1000+, which, for the size of active amstrad community, is almost.... "a M4 for every Amstrad user"   

[GUNHED]

I don't think i get this. Future os is software not hardware, and also,i suppose you mean transfer from somewhere else than a real floppy disk ,because transfer data from a real disk can be maximum ~8-9kb/sec (and this ONLY with custom fast track loaders, usally speed is ~half of that).

Yes, FutureOS is software. It's aim was (and is) to get the maximum power out of the CPC. To test this do the following: Connect two drives to your CPC (f.e. two 3" drives or 3,5" or 5,25") and then put in two data formatted discs (or system or ibm), then copy it. It does copy it in about 18 seconds.

If you copy with a single internal 3" drive it will take you about 9 seconds to read that 180 kb and also about 9 seconds to write a disc of 180 KB.

That makes 20 KB / second. The FDC can do a little more, but you need to take in account that with 512 bytes sectors there's a lot of header and gap bytes. With 1K sectors speed can be improved a bit.

And since I like perfect software I like to use the USIfAC, because it's perfect hardware. 



So there about 150 USIfAC out there, that's great. I'm sure people are really using them.    (And not only buy them to have it and put in a box). 

[ikonsgr]

If you copy with a single internal 3" drive it will take you about 9 seconds to read that 180 kb and also about 9 seconds to write a disc of 180 KB.

Woooo! That's REALLY impressive! I couldn't imagine that it was possible to get such throughouput from the good old 765FDC, especially when we are talking for a single head/ single density disk (which actual reading is ~4 times less than with a 3.5" dual head, DD disk). Judging from my recent experience with 765, i say you must have eliminated every delay in 765 function in order to achieve this impressive speed!
  But, may i ask,is it reliable at such speeds?  Does it transfer data from disks correctly every time, and for any disk? Also from my experience with 3.5" floppy drives,i know that  there are minor differences in various disk drives, some support more "advance" commands, some only the basic, and i have also confirmed that some drives (usally old branded ones, from NEC or Samsung) have faster responce, hence greater actual transfer speed than others!

[GUNHED]

Woooo! That's REALLY impressive! I couldn't imagine that it was possible to get such throughouput from the good old 765FDC, especially when we are talking for a single head/ single density disk (which actual reading is ~4 times less than with a 3.5" dual head, DD disk). Judging from my recent experience with 765, i say you must have eliminated every delay in 765 function in order to achieve this impressive speed!
  But, may i ask,is it reliable at such speeds?  Does it transfer data from disks correctly every time, and for any disk? Also from my experience with 3.5" floppy drives,i know that  there are minor differences in various disk drives, some support more "advance" commands, some only the basic, and i have also confirmed that some drives (usally old branded ones, from NEC or Samsung) have faster responce, hence greater actual transfer speed than others!

How to do it?
- Sense where the head is actually (read an ID)
- Read all subsequent sectors in the way they appear (do not read 1,2,3,...9 - do read 1,5,2,6...)
- Couple other smaller things


But back to USIfAC which is still a bit more quick than 20 KB 

[ikonsgr]

How to do it?
- Read all subsequent sectors in the way they appear (do not read 1,2,3,...9 - do read 1,5,2,6...)
- Couple other smaller things

Well,i think that, this exactly what fast track loaders do in some games! Instead of reading one sector/time, they give a single command to read a whole track of all sectors in one command (and yes, interleave reading of C1, C6,C2,C7 is MUCH faster than serial, but you need to have the disk written that way, otherwise, if sectors are writtem sequnetially C1,C2,C3... on disk, you can't read them so fast) , but still, they can only ~double loading speed (at around 8-9kb/sec), that is ~half the speed of yours! Maybe the... "couple other things" is the secret to double the actual speed, or maybe game loaders do other things too, and can't get such incredible loading speeds... 

[GUNHED]

All the discs are written with an interleave. 

[Targhan]

Well,i think that, this exactly what fast track loaders do in some games! Instead of reading one sector/time, they give a single command to read a whole track of all sectors in one command (and yes, interleave reading of C1, C6,C2,C7 is MUCH faster than serial, but you need to have the disk written that way, otherwise, if sectors are writtem sequnetially C1,C2,C3... on disk, you can't read them so fast)

Actually,  reading with one instruction or several does not change the speed. For Orion Prime, I read the sectors one by one.

Amsdos have the sectors interleaved because by the time it reads #C1 and processes it, another sector is already below the drive head. So by the time the OS is ready, it has missed it. Hence the interleave. But if you're doing your own loader without the user of a buffer (copy the read sector from a buffer to the data final destination), having linearly stored sectors is the way to go.

All the discs are written with an interleave.

Most yes, but not all. Some DSKs were transferred with linear sectors, making them reeeeally slower to read.

[ikonsgr]

But if you're doing your own loader without the user of a buffer (copy the read sector from a buffer to the data final destination), having linearly stored sectors is the way to go.

Hmmm, maybe that's how you can increase speed. If you are using a buffer for sector read (like for example, amsdos does), bytes readed from disk, are first placed to buffer and then loader puts them to their right place in ram. But, if you are NOT using buffer, and instead, each byte received from FDC is putted DIRECTLY to it's actual place, you can practically double the loading speed! Still, future os does this in ~4times the usual loading speed from disk, and AFAIK no game loader can achieve such incredible loading speed... 

Actually,  reading with one instruction or several does not change the speed. For Orion Prime, I read the sectors one by one.

Hmm, maybe then, you can give me a hint of why i get a: "interleaved sectors,check your disk" error message when i'm trying to use the 800k single disk image of orion prime with my 765FDC emulation... 

[GUNHED]

Most yes, but not all. Some DSKs were transferred with linear sectors, making them reeeeally slower to read.

Very true, and making hfe's from DSKs introduces some additional errors I guess.
Best thing is always to make DSKs / hfe's from real discs (if possible).
Since we have all that non-real-drive devices it's getting more diverse anyway.
And yes, using one routines for a floppy, you can omit interleaves. 


I'm really curious to use the U2 with FDC emulation the first time.


For me this is super awesome, because FutureOS already support the Vortex FDC765 (just other I/O addresses). Because it does support four floppies. 

[Targhan]

>AFAIK no game loader can achieve such incredible loading speed
Orion Prime does, plus it reads 10 sectors per side. This is no black magic.

>Hmm, maybe then, you can give me a hint of why i get a: "interleaved sectors,check your disk" error message when i'm trying to use the 800k single disk image of orion prime with my 765FDC emulation...

Because you're doing something wrong .
Orion Prime uses linear sectors, BUT also shifts the sectors every track, for even more speed.
Track 1:  C1 C2 ... CA
Track 2: CA C1 C2 ... C9
Track 3: C9 CA C1 C2 ... C8

[ikonsgr]

>AFAIK no game loader can achieve such incredible loading speed
Orion Prime does, plus it reads 10 sectors per side. This is no black magic.
Because you're doing something wrong .
Orion Prime uses linear sectors, BUT also shifts the sectors every track, for even more speed.
Track 1:  C1 C2 ... CA
Track 2: CA C1 C2 ... C9
Track 3: C9 CA C1 C2 ... C8

Actually the image i got, have a bit different structure: first track 0, head 0 (and also next, track 0/head 1) are normal data format  tracks, with interleaved sectors C1,C6,C2,C7 etc.
All following tracks are PARADOS40 tracks with 10 sectors and interleaved sector ids 'A1' through 'AA', which are indeed, in "shift" interleaved order (a pair of tracks -head 0 & head 1- starts with A1 next pair of tracks starts with &AA e.t.c).
Now, to be more specific about the problem i have, after initial loading of first track (sequential read of track 0/head 0 sectors C1 to C8), program gives a "Seek command" to track 1 head 0, and then two read_id requests (where my emulation returns the 1st sector_id of track 1 head 0 ,which is "&A1"), after that, i got a message on amstrad screen " Interleaved Sectors. Check your disc.", and freezes there.
Let me also note that ,when i got a "read data" command from amstrad, my emulation returns the next sector id of the one requested (or if it's a multiple sector, next of last sector requested), or, if this was the last sector on the track, it returns the 1st sector id of next track (which is the same track number/head 1, if track requested was on head 0, or next track no./head 0 if track requested was head1) . You think this might be wrong for some special situations and i have to return something else?
Btw, note that my emulation, is tested and works perfectly with numerous formats, even with a ROMDOS D10 (10sectros/track, 80tracks, 2 heads) which gives a maximum of 796kb/disk! The only images that i couldn't load, was orion's prime and Batman forever having this irregular "mix" formats, first few tracks are normal "data format tracks" and rest, special PARADOS/ROMDOS tracks... 

[Targhan]

Yes, track 0 is a conventional Amsdos track, else it wouldn't work when doing CAT and RUN from the Amsdos.

The double ReadSectors of the track 1 should be A1/A2. My code receives something else, so consider this is wrong. Can you trace what your read sector returns?
Returning the next sector seems fine, provided you get it right, but on that part, I don't know what you are doing wrong. Also, watch out for the end of track! On single sided-drive, there is no reason the next sector after the last sector will be in Head 1!
Maybe your tests works because the Read Sector instruction is not often used and is not actually correctly implemented? Run Discology and run the "scan floppy" option to check if it looks good.