News:

Printed Amstrad Addict magazine announced, check it out here!

Main Menu

CPC-CPLINK - a coprocessor interface card for all CPCs

Started by revaldinho, 19:54, 10 November 19

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

GUNHED

Honestly - I guess it's very important to have things uniform and having some kind of 'standard', because then everything is way more easy for everybody. Of course we still can all reinvent the wheel again - but please with the same protocols (or what else is needed).  :)
http://futureos.de --> Get the revolutionary FutureOS (Update: 2023.11.30)
http://futureos.cpc-live.com/files/LambdaSpeak_RSX_by_TFM.zip --> Get the RSX-ROM for LambdaSpeak :-) (Updated: 2021.12.26)

shifters74

Hi all,
I have just reread some of the recent posts in more detail cos i been so busy coding i have not been fully reading  :doh:

Once i get v0.2 of MCP testing complete and released i have ideas about 0.3 but also open to suggestions about what people want next.  I think further discussion with biged's boot injection stuff and possibly zhulien's ideas is required!  I am not sure what some of what you are talking about with regard to standard driver support etc is (maybe revaldinho does ??) but would like to understand.  I am a lot more knowledgeable on the pi side than the CPC (though making progress!!) so happy to concentrate my skills there and work with others more knowledgeable doing the CPC side.

V0.2 of MCP is feature and code complete (yes really this time  :P ) and just testing remains given the complexity of the changes i have made for this release.  However testing is going well and I hope to release the code in the next week.

cheers

Shifters

biged

Ah, I'm very new to CPC too!  I do have some thoughts, but I need to write them down carefully and I can't quite do that yet. Watch this space!

But I do notice a number of relevant concerns coming up in this previous thread, about using a serial connection for filesystem access
(Notably, @ikonsgr and @Docent discussing.)

(What we have in the CPLINK is very like a fast reliable serial connection, but also with control of the software and hardware at the far end: usually a Pi, but possibly a Teensy or Arduino or other.  A Pi is the favourite case for my purposes, as it's high performance, can run a full Linux, and has some useful peripherals built in.)

zhulien

Who here is open to talk in google hangouts?  We can make a group for MCP development & drivers - sometimes brainstorming is easier interactively.


I did actually start some driver work for CPC a while back but i think things can be improved and if there are input from multiple people who made the hardware - that can also be invaluable.


note: I cannot publish the google sheet into this forum for some reason, not sure why. if you want access before it goes into a wiki or git... let me know your gmail.

revaldinho

@zhulien , I can't see the Google Sheet in the Forum post, it just says




Google logoWe're sorry. This document is not published.

zhulien

I have started a Google hangouts group too for chat - this is likely short term, I wonder if a realtime chat could be added to CPC Wiki or whether an IRC forum exists?  But... hangouts is ok - it keeps our chat history to some degree too.


Msg me if you want to join the hangouts group also with your gmail.

Bryce

The Forum did have a chat function for a while, but it was removed later. Probably due to no-one using it.

Bryce.

zhulien

Quote from: Bryce on 11:07, 03 December 19
The Forum did have a chat function for a while, but it was removed later. Probably due to no-one using it.

Bryce.


Do you know if it had persistent channels like IRC?  or Just a temporary peer to peer chat while the session exists?

Bryce

I think it was just temporary, but you'd have to ask @Gryzor .

Bryce.

Gryzor

At one time we had IRC, then a local chat. Not many people used it and I removed it due to the resources it required...

zhulien

Rather than me hi-jacking this thread, I will create another non-hardware specific Driver thread. 

GUNHED

http://futureos.de --> Get the revolutionary FutureOS (Update: 2023.11.30)
http://futureos.cpc-live.com/files/LambdaSpeak_RSX_by_TFM.zip --> Get the RSX-ROM for LambdaSpeak :-) (Updated: 2021.12.26)

biged

Quote from: shifters74 on 18:16, 29 November 19
However testing is going well and I hope to release the code in the next week.


Hope things are still going well.  I said I'd try to write some thoughts down - here goes.


I think I've been approaching CPLink in a bottom up way: how to move bytes, how to get started with software both ends.  And I think MCP is a kind of top down way: how to provide a CLI, how to return printable results to requests that can be typed.


I'd like to arrange that the communication is reliable, and then rely on it, so there's no need for error detection and recovery. This is not a long-distance network, this is just a couple of FIFOs.


In the absence of MCP, I think the way I would have progressed is as follows:

       
  • define the format of a packet
  • define some standard types of packet
  • allow some room for expansion
  • start to provide some useful facilities at the packet level
  • implement a command line interface to those packets


So, for example, if packet types are a single byte, then I might have started with

       
  • 00 is a ping packet perhaps with a welcome message as the return
  • 01 is a packet going out of standard output, like to a printer or terminal emulator
  • 02 is a request to read data, it's standard input, like from a keyboard or a terminal emulator
  • 03 to open a file
  • 04 to read a block of data
  • 05 to close the file
and so on.  That's only a sketch of course.  It doesn't include any knowledge of CPC conventions, like for example the multiple streams feature of the VDU subsystem. Maybe byte FF would be an expansion, so the next two bytes are a longer type code, so we have lots of room for further facilities.


It seems possible that various facilities might be use at the same time: one might play a MIDI tune at the same time as spooling output to a printer.  For this reason I might well have said that packets should have a limited length.  Then packets can be interleaved without having to wait too long.


I think packets should always state their length. That means a whole packet can be skipped if it can't be understood - it's got an unknown type byte - or if there's no handler for it, or if it needs to be forwarded without understanding to a handler.


With this view, packet contents will sometimes be printable ASCII, but you never expect to type a packet as such.  A shell such as the present MCP client would pack data into packets and unpack responses from packets.  The present MCP server likewise would handle whole packets, and would have a tactic for dealing with packets of unknown type.


It might well still make sense for a server to be able to answer a query about which packet types it understands.  And it would certainly make sense for a CLI shell to have a HELP command which lists and maybe describes the verbs it is able to handle.


Now, possibly, just possibly, one could mix the bottom-up picture above with the present MCP picture, by having the type byte as the first byte, and any packet starting with a '+' byte is treated as newline-delimited text, or '---'-delimited text.  Or, if we can dispense with the +++ and --- because we've gained confidence that the communication channel is reliable, we can say any type byte which is alphanumeric is the first byte of a new-line delimited text.


But I'm not sure I like mixing the levels like this...  in the absence of any input from wise CPC experts, I'd probably use Acorn's Tube protocol as a model, and possibly the serialised version of that protocol.


Any thoughts?

shifters74

#63
Hi Biged,
a lot of what you have discussed is exactly what MCP v0.2 does currently including the packets.

V0.2 is finished and with Revaldinho for a last bits (makefiles etc) before it gets merged into his repo.  Once merged into his repo I will drop the release notes here.  The code is worth a read through (well commented!) as i think we are not too far apart in what we are trying to do.  The protocol used in 0.2 is flexible, extendable with just a few #define changes.  See the mcp_pi main loop which handles different packet types (text version of commands, binary version of commands, etc).  I implemented it with expansion in mind!


I am currently working on v0.3 which will provide:
1) binary equivalents of the text memory commands included in v0.2 (the text equivalents were proof of concept and will be removed in 0.3 as the binary will be the api to use memory and hence the text ones dont make much sense so will be removed.)
2) GetFile and PutFIle from PI storage to CPC storage and visa versa
3) GetMemFile and PutMemFile from pi storage to cpc memory
4) other ideas in the pipeline but not set in stone including RSX commands, CPC file viewer etc
5) SEXEC which executes a shell command on the pi and returns results to cpc e.g. "ls -al"
6) Optimisation of the incoming binary pathway (BinaryData coming from cpc to pi)

cheers

shifters

shifters74

#64
Hi Biged,
one of things i have factored into the v0.2 design is that some commands you will type e.g. PING, HELP etc.  Other commands e.g. the memory commands you can but mostly will not hence why there are binary versions of them (destined for v0.3) i.e. the difference between a user typed command and an API  - the TextCommand is a user typed command (but can be done in code - see the basic and c commands included with MCP!)  - the BinaryCommand is an api call expected to be binary etc.

There are other packet types defined in v0.2 including BinaryData (for moving memory around etc) etc - see mcp_pi.h for the different type examples when 0.2 is published.

I have added a Version command just for you so it will tell what MCP your command is talking too!

cheers

shifters

shifters74

Quote from: biged on 17:00, 05 December 19

It seems possible that various facilities might be use at the same time: one might play a MIDI tune at the same time as spooling output to a printer.  For this reason I might well have said that packets should have a limited length.  Then packets can be interleaved without having to wait too long.

You can do this in concept in v0.2 as it provides command interleaving i.e. you can have commands that execute in steps so can have several things happening at the same time if you write the commands correctly (to support the interleaving). The SHUTDOWN and REBOOT commands both execute in multiple steps. 

Quote from: biged on 17:00, 05 December 19
Now, possibly, just possibly, one could mix the bottom-up picture above with the present MCP picture, by having the type byte as the first byte, and any packet starting with a '+' byte is treated as newline-delimited text, or '---'-delimited text.  Or, if we can dispense with the +++ and --- because we've gained confidence that the communication channel is reliable, we can say any type byte which is alphanumeric is the first byte of a new-line delimited text.

The reason for the delimiters is that i wrote the MCP and a number of basic and c commands but what is executing on the CPC will not always be in my control hence the need for delimiters when you get sent duff data to the MCP.  As its an overhead of 6 bytes its no big deal but its main purpose is to stop the command processor on the MCP grinding to a halt as it has no idea what it looking at because of the duff data its been sent.  I admit the +++ and --- is not fool proof but its a start!

shifters

shifters74

Please see http://www.cpcwiki.eu/forum/programming/cpc-drivers/  for zhulien's thoughtful look at developing a driver standard.




shifters74

Released MCP_pi v0.2 (wath associated CPC basic and c compiled commands) and merged in Revaldinho's repo (thanks!)
See attached release notes.
shifters

revaldinho

Quote from: shifters74 on 12:11, 10 December 19
Released MCP_pi v0.2 (wath associated CPC basic and c compiled commands) and merged in Revaldinho's repo (thanks!)
See attached release notes.
shifters

Making great progress on the MCP, @shifters74 - thanks for supporting this hardware project.


For anyone else interested, I have a couple of boards left from the first batch, already assembled and available now. There will be a slight delay 'til after xmas for further cards, as 74HCT40105s seem out of stock and are on back-order at Mouser and digi-Key in the UK (no worries, these are still active parts). I will need new PCBs at some point too and there's a ~3 week lead time for those.




LambdaMikel


nicf82

I've been experimenting with my new cplink trying to create an interface between the CPC and MQTT for home automation. One think I have noticed is when using a python script on the RPi and a C program on the CPC, I seem to lose a lot of data in the transfer! I'm assuming this is due to the buffer in the FIFO becoming exhausted because its not being drained quickly enough, confirmed by inserting a delay in the send loop on the CPC - but I had assumed that even running python the RPi would never be outpaced by the CPC!? Does this sound correct, if so is there a better way to solve the problem other than:

       
  • Running a C program on the RPi to speed things up
  • Slowing the CPC down with an artificial delay in the tx loop?
Both solutions seem to me like they might still be error prone and only work due to good fortune. I guess, as has been discussed earlier in this thread, what i may need is some kind of packet based protocol with parity checking but that seems very heavy weight for my simple needs.

Cheers!
Nic

biged

You won't need any delay loops, provided you're checking the status registers and reacting accordingly.  A writer should not write a byte until there's space in the FIFO, and a reader should not read a byte unless there's something to be read.  So, at both ends of the link and in both directions you need to be checking the status register.


On the wiki we see this Basic code:

130   IF i<>1024 AND (INP(&FD81) AND 2) THEN OUT &FD80,tx[i] :i=i+1
140   IF j<>1024 AND (INP(&FD81) AND 1) THEN rx[j]=INP(&FD80):j=j+1

nicf82

Hi biged


I tried an experiment using only code from https://github.com/revaldinho/cpc-cplink - it involved running the (C) loopback client program on the CPC and the (python) loopback server program on the RPi, and I got thousands of errors, but 0 errors if I ran the C server program on the RPi - this is also what led me to the conclusion that python was too slow. And I can't blame that test on my bad code  ;D


Aside from that test,


My CPC C code just uses the fifolib.h/.s library so it think that this should handle checking the status registers also?


I will double check what my python code is doing but it is basically a copy and paste of loopback_wpi.py from the repo with a few extensions of my own but the send/receive functions are unchanged and appear to check the registers.


Cheers!


Nic

revaldinho

I think you might have just got out of sync. If you leave the loopback code running on the Pi while stopping and starting the CPC program, then the Pi will be attempting to return values from a previous run.
If you kill both the Pi program and the CPC one, power cycle the CPC, then restart the Pi program and then the CPC program do you still see the errors ?

(don't forget that to reset the FIFO from the CPC side you need to write to the status register - OUT &FD81,0 )


When using the status register for checking byte by byte there's no limit on how slowly you can run - see the demo with the CPC running BASIC and the Pi using Python ...  :D

revaldinho

I just tested this here to remind my self of what's supposed to happen with the short demo...  :D


Running in C on the CPC and Python on a Pi 3A+ I get this (see snap in the attachment) - ie no errors for Test 1 and 2 which check the FIFO status on every byte transferred, but lots of errors on Test 3 which doesn't do any status checks at all.


This is normal.


YMMV on Test3 as the error rate will depend on the speed of your Pi and what else you might have set the Raspbian distro to be doing while the loopback code is running. The same Pi loopback implemented in C can often get zero errors here on my 3A+ ... but never on my Zero. I think that this is what you might mean by the Pi Python showing errors but the C version not ?


The key thing is that tests 1 and 2 are 100% reliable but the byte by byte status check limits maximum throughput to about 50KBytes/s in either direction. The limitation is simply the speed at which the Z80 code can check status and send/receive bytes.


Test 3 give interesting results on different Pis and with different implementations of the Pi loopback, but really it just confirms that trying to implement a blind burst mode (where speed can go above 100KBytes/s) needs to be accompanied by CRC type packet checks and the ability to retransmit. I think that the overhead for doing this probably outweighs any saving made in the raw bit rate, so I recommend always to use the byte-by-byte status checks.


Once you've run this test and seen that the last test3 has reported errors, you need to stop and restart the pi loopback side of the demo, because the pi is continuing to receive and sent bytes whether the FIFO is ready or not ... which is what I meant by being out of sync.
[size=78%]  [/size]

Powered by SMFPacks Menu Editor Mod