CPCWiki forum

This is about another retro computer, but MIGHT have some connection with the new thread about the Spectrum loader.


I still have my Epson HX-20 (from 1983), and I still play with it from time to time.


A major inconvenience is saving/loading progs to tape.


I've recently been experimenting with saving progs (and data) to my laptop, using the same cable I would use to connect to a cassette player, SoundRecorder to save the signal. and then MediaPlayer to play it back for loading.


This works quite well, and seems pretty reliable, and has the massive benefit that each file will be saved as a discrete file, no need to search a tape for the correct file.


The data is stored as a .WAV, although the default is a serious overkill quality, but I've tried reducing the quality somewhat and determined that the reduced file will still LOAD back.


So far, this is OK.


BUT, I would like to try to do a bit more with the data stored in the .WAV - i.e. turn it directly into something readable/printable.


This will entail turning the .WAV data into binary, and then into ASCII.


I've been looking into the raw data inside the .WAV - hmm??   But, if the computer can do it, even the HX-20 (and even the Spectrum and the CPCs) then it cannot be rocket science!


In the case of the HX-20, the recording uses a process where a 1 Khz signal is used for a zero bit, and a 2 Khz signal is used for the one bit.   I cannot see that there is any 'gap' data, it's just a continuous stream of bits.  Do the other computers do something similar?   Then, it's just down to the relationship between the timing, and the data, to translate the sampled data??   I.e. there'll be twice as much data for each '1' bit as for each '0' bit?


My original .WAV looks pretty hopeless, prob just too much sampled data.   The reduced one I have looks more hopefull, you can more easily see sound ON and OFF, but this may be misleading.


Are there any sources on the technical structure of the cassette recorded data?   Has anyone looked into what's recorded on the tape?   And how that relates to the actual readable data?


Geoff

Would you mind posting a wav file of a simple program for us to take a look?


Many computers have some sort of header tones or header bytes and data packed in blocks with a control value to determine if it is corrupted.


More when we have a wav file!  ;D

Thanks for the interest.


Yes, I'd have expected some sort of 'punctuation'!


I'll try to attach the data file I created.


The prog I used to create this was:


10 open #1, "cas1:data"
20 print #1, "AAAA"
30 print #1, "BBBB"
40 print #1, "CCCC"
50 print #1, "DDDD"
60 print #1, "EEEE"
70 print #1, "FFFF"
80 close #1


the default operation of the W7 SoundRecorder saved this as a .WAV at 44100, 16 bit, stereo but I subsequently used the XP version of SoundRecorder to reduce this to 22050, 8 bit, mono.  Even with this reduction, there may still be 11 times more data than needed??


Hopefully, I'll attach the .WAV, but I might need to change the .ext!  Yes, changed to .MPG, but it IS a .WAV


Geoff

Did you sent the right wav?


I mean, there's 3364 bytes there, and I would expect it to have no more than 48-50 bytes (24 data bytes, AAAABBBBCCCCDDDDEEEEFFFF, and some form of mark to indicate beginning of data).

Yes, that's the right .WAV.


I'm not sure what you're thinking of?   I don't see how it could be anything like the size you mention.


The prog ran, and it took quite a few seconds to feed the data through.   Each second of playing produced 22050 samples.   I assume the recording process converts the bytes into bits, so you're looking (I guess) at the possible number of bits BEFORE the sampling.   The data rate (of bits) is either 1 Khz or 2 Khz (for a zero bit or a one bit).   The sampling will be sampling the cycles, I think, not the indiv bits.   So the data will be at least 2000 cycles per bit?   Multiply by (I suspect) 11 for the relationship between the 22050 and the 2000.


I don't fully understand it all myself, that's why I'm asking.


My starting and stopping of the recording left some margin, so there will be some sampling of nothing before and after the actual data., but I don't think that will have added too much.

Rob,


Doing a bit more calc, your suggested 3364 might be reasonable.   Number of bytes x 8 bits x a multipler of about 11 to reflect the sample rate relative to the date rate and we're getting towards 4000.


The important thing is - how have you got from the data in the .WAV file to the 3364 bytes?


Geoff

I've checked my 'C' prog which is reading the .WAV file - at the moment, it's just reporting on the detail in the headers, But as I learn more about the rest of the file, I'll add more to the prog.


I confirm that my data shows that the data in 28.4 seconds long.   It was slow doing it.


The file contains 627203 samples.   That is about 22050 samples per second.  So that agrees.


Still cannot see where your 3364 comes from?


Oh, looking through the file, I note that the wave numbers gradually reduce through the file.   Could this be due to the battery voltage slightly reducing over the duration of the recording?


Geoff

Hi Geoff!


There's no direct relation between the number of samples in the wave file and the bits represented on it.


The wav file contains silences, which obviously don't count towards the amount of information stored, and neither sampling or the recording hardware and storage medium are that precise.

QuoteThe important thing is - how have you got from the data in the .WAV file to the 3364 bytes?

Well, I developed a program to help me visually correct wav recordings, either because the original medium was damaged or the recording was unstable.


It's really easy to see the problem, but no audio editor works at wave level, so I had to develop my own tools.


Here's a screenshot of the tool I'm using to take a look at your file:


[attach=2]


Bits are encoded as sine or square waves of given frequencies. In your case, there are 1000hz waves (most probably meaning a 0) and 1800hz waves (most probably meaning 1).


My program recognises the waves and lets me change the values if I see an error. It also allows me to export the bitstream to a file than can be checked in a hex editor.


Not all waves represent data bits, there will be header/sync bits (also known as pilot tones) and some may represent parameters such as the number of data bits or control sums.


In the case of a wav from a Dragon computer, there will be 128 or 256 bytes with value 0x55 used as pilot tone, then a byte marking the start of a block of data, followed by one that tells the kind of block, another which tells the size of the data, then the data itself followed by a control sum and then the end of block mark.


Our problem is that we don't know how the HX-20 organises the information. It may even use more than 8 bits to store a byte of information. Still, it amazes me that you'd need 3364 bytes to store 24 data bytes.


It'll be interesting to learn how the computer does it.

Rob,
The wave on your illustration looks VERY similar (not as 'clean') to the wave shown in the Technical docs for the HX, although that says the '1' wave should be 2 Khz rather than 1.8 Khz.   But I suppose 1.8 is close enough, certainly seems to SAVE and LOAD happily enough.
Very interesting.
The same doc did not give any hints about any 'punctuation', or extra info along with the data.   The doc is really about the built-in microcassette, not an external cassette which I am in effect using, although I know that someone has rescued a recording from a microcassette to a .WAV, and that file then loads fine as if from an external cassette, so I assume the data formats are the same.
As I said, you say there are 3364 bytes in my file.   That could be correct. Yes, it's a lot more than the data should be, but there could be 'punctuation', there could be duplication.   I suspect from listening to the data, and the fact that the LOAD recognises and reports on the filename, that a filename is saved, and I think this is duplicated.  IfI listen to a recording, I hear the start of the data, and I hear a brief squalk, then a constant tone, then the same brief squalk, then more constant tone, then into solid data.  The 'header section may have some other data there, i.e. a single char filetype (i.e. B for Basic Prog).
If there's another data type I could try that might help more?
Massive thanks for your help.

Geoff

Is it similar to the BBC Micro?

Don't know about BBC.


I've changes my prog, to incl 'bar chart' type listing.   I'll compare data about sample 6891.


I'll have to try a conv to ASCII, and just do repeated tries until valid text appears?


Rather like Bletchley park 1942, using similar methods until valid German text starts to appear!


Geoff


WAVE1.ZIP contains TEST.TXT

I've had a further play with my file.


Looks like samples 1 to 346 are a flat tone, no wave.


Samples 347 to 5737 seem to be a mix of '0' and '1', and look like they may be valid binary.


Samples 5749 onwards look like a fixed, static, '1' wave, i.e. the 2 Khz.


I've checked samples up to 50,000 only.   Out of 627203.   Could be the HX went into a 'wait' or something, or the PC receiving the signal did something odd.   I always thought that the 28.45 seconds for my 8 line BASIC prog was long, but it finished cleanly.   I'll look again at the binary .WAV file?


Anyway, there is NO correlation between MY samples about 6891 and the wave shown in your graphix/screen image.


?????


Geoff

Success!  8)


I managed to find detailed info on how the data is stored, and I managed to get the data out of the last block of your file (just using a small set of data to test my code, you know).


The funny thing is that, according to the data, you saved the file from your HX-20 machine on October 22, 2017 at 18:42:08 (local time I guess).


[attachimg=1]


You can find the documents I mentioned above here: http://www.vintagecomputer.net/fjkraan/comp/hx20/doc/info.html


The one detailing the tape format is tmS_06-07.pdf.


I'll try to decode the whole file and then I'll upload the source code somewhere.


Take a look at the spec and you'll see that it's possible to waste 3364 bytes just to store 24 of them.


Thanks for giving me an interesting challenge!

Rob,


Thanks.


Yes, I'd seen those docs.   I pored through them - I thought - all of them, looking for just this info.   There's some detail in an earlier doc about the general operation of the tape system, which gives some info, but I didn't get to this bit.   Maybe the somewhat indistinct heading got me missing it.


Yes, this doc does explain everything.   Incl the 'punctuation, and the blocks, and the chks.


I've just about worked out my own logic for reading the .WAV data and getting the bin and thereby the ASCII.


Clearly, it WILL be possible and practical to process the saved file, and product an ASCII version.   That's part one.   Then the possibility of taking a changed ASCII file and producing a SAVE file as a .WAV for re-load into the HX - more of a fiddle, but possible.   Need to re-build the .WAV file header, etc.


Editing progs on the HX is of course possible, but it is a fiddle with the 20x4 screen, and not much better with a larger 'virtual' screen with the 20x4 actual screen as a window on it.   Back when my TF disk drive was working, I would save larger progs to the disk as ASCII, transfer to another machine (at first the PCW, later a PC) do the major editing there, and then transfer back.


Re one point, I think that the system uses 'reverse waveform'.   Is that so?   All the indics seem to show the low part of the waveform first, then the high part.


I'm thinking I can just count the data bytes in the .WAV, using the specific format of .WAV I've chosen.  I'll need to step through x low bytes, followed by x high bytes.   There seems to be 2 or 3 of each for '0' bits, and 4 or 5 of each for '1' bits.   So, if the total is less than 8, that bit is a '0', and if more than 7, then it's a '1'.   That should do the job?


THANKS again.


Geoff

Rob,


Just for your info, the date stamp on the PC files for the original .WAV data files are 22/10, and a time 23:40 something.   The timestamp in the file will be generated by the HX, and the time set on that will prob be a little out as the battery and charge is a bit hand-to-mouth.


I was never aware there was a timestamp put into the file.   Nothing in the normal use of the HX ever shows it!


One of the BASIC commands is FILES <device> which will play through the complete tape and list the filename of each file found in turn, but it takes a while to do a long take with xx files on it.  The filename is in the header block.  Rather more useful if you've got the Disk BASIC loaded, as FILES "A:" for example does a DIR of the disk - a LOT faster!!


Geoff

Hi Geoff,


Definitely reading/saving programs from/to wav files is possible and even not that difficult.


I'd like to finish parsing your file and report the results back to you and then try to create a wav file from data for you to test on the machine.


Could you save a basic program and post the wav file here?


Cheers,
Rob

Rob,


Thanks again.


Got one here, not too large.   Saved as ASCII, so should be readable.


Geoff

Does your program look like this one?



10 PRINT"EPSON BIG PRINTER"
20 PRINT:WIDTH"LPT0:",8
30 PRINT"WHICH SYMBOL DO YOU"
40 INPUT"WANT TO USE";A$
50 CLS
60 PRINT"ENTER THE WORDS"
70 PRINT"YOU WISH BIG PRINTED"
75 PRINT"ON ONE LINE ONLY"
80 INPUT B$
90 G=LEN(B$):IFG>20GOTO60
100 X=G*6-1:DIMP(X,n D7):CLS
110 PRINT B$
120 FORY=0TOX
130 FORZ=7TO0STEP-1
140 P(Y,Z)=POINT(Y,Z)
150 NEXTZ,Y
160 FORY=0TOX
170 PRINT
180 FORZ=7TO0STEP-1
190 IFP(Y,Z)THENC$=A$ELSEC$=" "
200 LPRINT C$;
210 NEXTZ,Y
220 WIDTH"LPT0:",24:END


;)


Enough for today though.


I should be able to write it back to a wav file but I'll need to implement the CRC computation code first.


Also, I'm using the list of pulses that comes out of my DragonCas program which, given the good quality of your recordings, works quite well and I had only to discard only the pulses incorrectly detected in the silences between blocks.


We can discuss the preferred format of supplying the list of waves to the program, as it may not be something that can be done automatically.


Have a good night!

Rob,


No!


;)


OK, almost yes!


Actually, you show line 100 slightly off.


Should be:


100 .......:DIMP(x,7):CLS


so you've got a couple of spurious chars in the DIMP bit.   Maybe a bit of 'punctuation' not spotted?


Everything else seems spot on.


Wonderful!!


Thanks again.


geoff

Such are the perils of copying and pasting late at night...  ;D


The program is ok, I just copied the text from the resulting binary and accidentally copied the crc bytes along.


I'll integrate the HX-20 support into DragonCas as that way it'll be easy to check and convert wav files.


Then, I'll make a small command line program to turn any file into something the HX-20 can load.

Well, not bad for a couple of hours of work:


[attachimg=1]


I still have to check how to compute the CRC and fix a couple of minor things. But all in all, I think it's coming along nicely  8)

Ok, I tested the first wav file you posted, and it seems that both copies of the header block are damaged (looks like it started writing as the motor was starting to spin up).


As header blocks contain the length of the data blocks, my code was reading them without data. So I forced them to have 256 bytes, and I got the expected result:


[attachimg=1]


In yellow you can see the full data block and in green your data.


I'll take a look at the CRC algorithm this weekend.

Success!


I discovered hot to compute that damned CRC, so now I can try to pick a file on my computer and convert it into a wav file for the HX-20.


The information on the HX-20 is quite good, but I had to investigate to learn that the start value of the CRC computation depends on the block number... times 0x911!


Anyway, I'm glad I got it working  8)

The crunch will come when I try to LOAD something.


Can get around that a little, if you just create a different data file, as long as you tell me the format, I can try to INPUT #1 from it, and if I can do that OK, then the format must be OK.   If a data file is OK, then a prog file should be OK too.   I believe!


Thanks.


Geoff

Hi Geoff!


The time has come!  ;D


Please try the attached wav file on your HX-20. It will most probably fail due to some weird thing I wasn't aware of, but it will be worth the try.


It's a small basic program. If it loads, run it and, if it works, please post a picture!

Rob,

Massive thanks for your efforts.

BUT..

I've tried to LOAD this, and no response.   Nothing.

I immed tried one of the files I'd previously saved, and this worked fine, loaded, prog runs.

Tried your file again, same command, nothing.

I tried LOAD "CAS1:Rob1" - if the name was wrong, the HX would usually report 'SKIPPING xxx' with the name of the prog it HAS found, which would prompt me as to the filename you've actually used and retry, but I don't even get that.

**** Interesting - I note someone else has downloaded the file.   Is there someone else with a HX20?

Geoff

Well... I was sure it most probably wouldn't work on the first attempt  :)


I'm pretty sure that the file is correct as I can successfully decode it as with your previous examples.


Maybe the volume is too high or the frequencies aren't quite right.


Could you try lowering the volume on an audio editor to 40% and try again?


The frequencies are 1000 and 2000hz, but on your machine they are around 1000 and 1800hz, I'll try to generate a new wav with those parameters.


And maybe, it expects a sine wave and not a square wave...


I'll upload some new tests shortly!

Here you have two tests: both at 50% volume, one with square waves and one with sine waves.


Please try them. Try also different volumes in case your output is too saturated and is distorting the waves.


If they don't work, I'll try adjusting the frequencies, but they should be fine.

I realised that I was using the wrong file type for an ASCII basic file...  :picard:


Could you please try the attached file?

Rob,

Thanks for new versions.   I've just downloaded them, I'll got to try them now.

I don't think it's anything to do with volume.   As I understand the data in the file, the levels are the same as normal, and I've not changed any settings on anything.

But I'll check that anyway.

However, the data in the file is NOT as before, but I don't know if this is important.   The data in Rob1 does NOT represent a wave, with (+) peaks and (-) peaks, in effect it has (-) peaks and (0) peaks>

The original data showed numbers that were + and others that were -.   There were no zeros within the data.   Zero represents a flat line in the middle of the wave, i.e. no signal.   Then the values 1 to 7F represent the level of the (+) wave rising above and back down to 0, while the values 80 to FF represent the (-) part of the wave.   Yet your file is either FF or 0.

I don't think the sine/square wave thing is relevant.   The wave form should be sine, but even the higher resolution .WAV showed no sign of any sine wave form, it has always been - in effect - as near square wave as makes no different.   I;m sure the computer is just detecting the frequency, and the form is irrelevant.

Geoff

Rob,

Just unpacked files etc, heading for other computers.

Just looked at binary data in Rob4 - YES - this looks much more like what I _THINK_ it should be.   I guess you've already changed whatever was up with Rob1?

Geoff

Rob,

No joy.

Tried Rob4, nothing.

Just to be on the safe side, please confirm EXACTLY the filename in the data header.   I think the LOAD process is fussy.   There is space for 8.3 filename, I think, but normal operation seems not to use the extension, but it may check it?   If just one file, may not show 'SKIP' message.   Also, process DOES differentiate between case, so Rob4 and ROB4 are different.   Also, please ensure any chars of name data not used are set to space?

BUT - as noted prev message, data in Rob4 looks much more likely.   Compared to Rob1.   Not tried anything with Rob 2 and Rob3.

Tried different volume settings with Rob4.   No visible difference.

Geoff

Quote from: GeoffB17 on 13:42, 12 November 17I don't think the sine/square wave thing is relevant.   The wave form should be sine, but even the higher resolution .WAV showed no sign of any sine wave form, it has always been - in effect - as near square wave as makes no different.   I;m sure the computer is just detecting the frequency, and the form is irrelevant.

It almost certainly has a low-pass filter on its audio input train, and can't even tell the difference between sine and square. But most machines of the time count zero crossings — it has to be very clear that the wave was below zero and that it is now above zero. Just touching zero from either direction could be sufficient if the filter also eliminates DC bias, but you should definitely try very obviously to cross it.

Well, the earlier Rob1 file was NOT crossing the zero (cente) LINE.   My original saved file was.   Rob's latest Rob4 file IS now showing values that cross the line, so this should be better.
BUT, my HX is not even recognising the file.   I'm just hoping to rule out any problem with the filename.   I had a .WAV from somewhere else, and that declined to load until I realised that the filename (in the file header)was a mix of case (i.e. Jump rather than JUMP) and then it loaded fine.   I think that the HX would always SAVE as capitals, although I never tried to save as a mix.   In any case, when the LOAD process recognises the file, there's a message saying LOADING Rob4 or whatever. I never get such a message. YET!!
Of course, getting that message will not be the end of the matter.   Might get that message, and then get another like IO ERROR, which I was getting from the microcassette drive due (I guess) to the rubber drive band having corroded.
Geoff

So, crossings may be enough?   I'll keep that in mind.   I assume that this will be relative to time, so that x crossings in a given time unit will indicate one frequency, and 2x crossings in the same period will indicate twice the frequency?
Geoff

Quote from: GeoffB17 on 18:00, 13 November 17
So, crossings may be enough?   I'll keep that in mind.   I assume that this will be relative to time, so that x crossings in a given time unit will indicate one frequency, and 2x crossings in the same period will indicate twice the frequency?
Geoff

Yes, that's generally sufficient for most micros. Some sort of counter counts the amount of time between one crossing and the next, and the value of that counter indicates the length of wave encountered. Some machines, like the Commodores, simplify that even further and count only the crossings in one direction — in that case from negative to positive if memory serves.

So the process can be as simple as:

while(loading) {
    counter = 0;
    while(no upward zero crossing) counter++;
    if(counter > threshold) post_bit(1); else post_bit(0);
}

... though if you want to get really robust then you should really use the solid tone at the start of recording to figure out what value 'threshold' should take, and possibly even constantly recalibrate it as you go, and perform a low-pass or band-pass filter before processing anything, but that's usually in-hardware on a real machine.

I'm checking everything I can, but I don't see any obvious mistake.


Here you have a new version with the file name in uppercase and reduced the frequency of the 0 waves from 2000hz to 1800hz.


I've check the header data against your files and everything seems in place.


By the way, all my wavs do have crossing waves, but they are saved as 'unsigned' values, so if you're checking them, they range from 0 to 255, being 128 the middle value.


Edit: I forgot to say that I changed the name of the program to ROB1 just in case.

Thanks Rob.

Tried LOAD "CAS1:ROB1" but nothing.   Tried various volume settings, but volume not partic loud even at max.

Tried FILES"CAS1:" as well, which should do a sort of DIR, but this recognised nothing either.

Using ears to check playback, seemed OK, although seemed to be a lot less data than usual.   But then, I guess your prog is pretty small as well.  Please detail what SHOULD be there, I'll try to enter same thing, and create matching .WAV as created by HX, to compare.

Geoff

That's a great idea!


The program is as follows:
10 PRINT"Hi Geoff!":END


I'm pretty sure it's something silly  :)

Hello,

I've created the prog on my HX, and saved it as ASCII, as ROB1.

The default save process saves as 16 bit stereo, I've converted to 8 bit mono, but it's still twice the size of your file.

Oh, after I'd saved the 'reduced' version, I ran the LOAD process,and it loaded fine.

I'll check for differences later, but I guess you'r 'tools' can do more than mine.

Thanks.

Geoff

Hi Geoff!


I've fixed the date format, the spaces between blocks and added a 0xD as first character of the basic listing as in your file.


Please try the attached wav file.

Rob,

I saw your new file last night, but the battery on the HX had run down, and anything I tried to do was not reliable.

Tried again today.

Tried to load ROB7, took no notice.

TRied the FILES"CAS1:"  which may have indicated if I was getting file name wrong, but that revealed nothing,so still not happy.

Just now, my main PC is dead, getting sorted, so I cannot access my dev work.   There's a limit to what I can do on this W7 laptop.  At least, I can access email etc and the web, and this is the machine that records/replays the .WAV files.

Geoff

Which program do you use for playing the wav files?


I had problems with several audio editors, and the one working for me is audacity.

I'm just using Media Player.

Audacity seems heavy handed?

MP has worked fine for various different 'qualities', and levels.  The files I've saved work OK.

Geoff

I'm still stuck with the W7 laptop, although this does have all the .WAV files on.

As mentioned before, I'm using the W7 SoundRecorder to do the recording, the plus of this is that it's quick & easy, and lo time limit.   But allows no manipulation of the saved file/data.

I also have on this machine the XP version of SoundRecorder, filename sndrec32.exe.   This will allow various options over and above the recording.   BUT, it will not record more than 60 secs.

It does however show an image of the recording, like Audacity, and allows some editing, and allows save in different 'quality' settings, etc.

If I use this prog to look at the file I made rob6t2, and your file rob6 (rob1), which should be the same thing, the appearance of the data is quite different. My file is totally 'blocky', i.e. more a square wave, even totally a square wave.   Your file shows much more like a sine wave.  As in a string of balloons of slightly different sizes.

It's unfair to say that one is right, and the other is wrong, because they may sound very similar.   But one clearly works, and the other does not.

Do you have the XP prog.   SNDREC32,EXE?

Geoff

I suspect that programs treat the 'zero level' slightly different.


Please try to load my last file through audacity.

Well, I'm not sure if this tells us anything.

I went into Audacity,and loaded your recent file ROB7.   A said it was 32 bit, should be just 8?   Anyway, I didn't change any setting in A (not that I know how).   I set HX to LOAD, set A to play.   Nothing.

Just to confirm, I then loaded my last file, ROB6T2.   Looked fairly similar on the A screen, actually.   Repeated process.  Loaded into HX fine.  At this end, no visible difference between MP and A, MP just a little easier.

So, seems to make no difference between A and MP.  Neither process seems to recognise your file.   Both processes seem quite happy with original SAVE file.

What next?

Geoff

Maybe there are settings re time scale, but A shows just solid bar.   not correct definition to suggest any wave form.

SNDREC32is more helpful,in that you can see wave form, or lack of it,and gaps between bits?

Oh, just testing, but determined that SNDREC32 will NOT record a signal for more than 60 secs, however, it WILL load, and modify, and save an already created file (say created by the W7 prog).

Geoff

Hi Geoff!


Could you try the new wav file?

Hello Rob,

Just found your message from late last night.

Tried the file ROB8.

Sorry, but no change.   Neither LOAD or FILES indicated anything.

Loaded the file into SNDREC32, and in structure the file looks just like previous version, i.e. the waveform that looks line a string of round balloons.

I've got a working PC again, still getting things sorted on it, but I'll get vack to my prog that was doing some initial analysis, and have a look myself.

Thanks again for your efforts.

Geoff

Rob,

I'm looking through the raw hex data of your file.

I'm looking at bits only, but I'm not finding the sort of variation of data that I think should be there.   Am I just missing it.  The file is big, so it's a bit like looking for a needle in a haystack?

Your file is 44.1 Khz.   I'be confirmed that less still works fine, certainly 22,050, maybe even lower.

There will be a number of bytes of sample data for each cycle.   I calc that at 44.1, for 1 Khz, there will about 44 samples, made up of 22 of minus, followed by 22 of plus.   If it needs to be a 2 Khz signal, then it'll be 11 bytes of minus followed by 11 of plus.   BUT, these numbers can vary by 1 or 2, I assume the timing has a few %% leeway, based on my study of the files I've created.

If I'd got to the point of trying to translate the data, I'd expect be finding the first (-) byte in the .WAV data, counting the following (-) bytes and then the following (+) bytes.   Note the total and reset the count.   If the total is say 30 or more, then that bit is a '1', if the total is less than 30, then the bit must be '0'.

One thing on 'reading' the data, but as suggested, how to make adjustments when 'writing' to keep the overall timing OK?

Geoff

QuoteI'm looking at bits only, but I'm not finding the sort of variation of data that I think should be there.   Am I just missing it.  The file is big, so it's a bit like looking for a needle in a haystack?

Well, there's a lot of sync '1' bits, then at least 40 '0' bits, and then the actual data begins.

QuoteYour file is 44.1 Khz.   I'be confirmed that less still works fine, certainly 22,050, maybe even lower.

Indeed, but if you are trying to digitally process the wave data it makes things easier to have more samples, at least in my experience.


Anyway, I made a new wav with its waveform reversed. It shouldn't be that as the machine calibrates itself, but it doesn't hurt to try.


I'm trying to take a look at how HXTape (http://hxtape.sourceforge.net/) works, but it's written in php...


I really think we're quite close to nailing it.

Rob,

Thanks for Rob9 - however, tried the usual things, and nothing is detected by the HX.

I looked at the file within SNDREC32, and the image looks MUCH more like my last file (Rob6r was it?).   The wave is now more blocky, looks generally more like the files I've created.  However, still not a help if data not seen at all.

Regarding HXtape, I think I looked at that, but as far as I could see, it was simply mimicing what I can do already, i.e. save the normal tape output to a PC as an audio file, and re-load the same file again later.   I don't think it tries to do anything with the audio data.   However, if it does give any hints, well great.

PHP isn't really a problem as far as the language is concerned, it lies somewhere between 'C' and HTML, the problem would be setting up all the surrounding stuff, usually something like Apache to drive the HTML and the PHP module to be invoked via the HTML as it's processed by the Apache.   It is possible to set up Apache on a single user machine, but it seems a bit heavy handed.   Why this process was done via PHP I don't know.

Are we getting close?   I'll be pleased when the HX actually recognises SOMETHING, even getting an IO error would be massive

Geoff

Rob,

I've looked again at the HXTAPE stuff, which I had tucked away on my HD.

Yes, there could well be a lot of useful info in there.   It's a bit back-to-front to what we're trying to do, hence my confusion, and in fact is maybe bypassing any supposed .WAV file.   The SAVE process is intended to write - say - a BASIC prog to a tape which can then be LOADed, or even in fact bypass the tape totally by sending the audio data (not even .WAV ?) in a form that appears to the HX as if it's coming from a tape.   Is that your understanding?   The LOAD process is the reverse.

I note it can cope happily with encoded BASIC, and will automatically translate the tokens for the BASIC keywords.   That's helpful.

The source is, as you said, php, but it looks substantially 'C' to me.   Many of the commands/functions have direct 'C' equivalents.   I see some that I'm not familiar with, but I'm sure the operation of those could be simulated.   I've got a couple of books here on PHP.

I'm trying to see if it uses any special libraries, but I think not.

The function to handle the CHK would be very helpful.

There may be stand-along processors for PHP, but note that PHP is usually interpreted rather than compiled.

So, on second/more detailed look, there IS prob a lot here worth checking.   May be able to bypass the whole .WAV thing?

Oh, there's an email address for the writer.   I tried that a while back, it's no good now.   I traced him also via some GitHub things, but that was out of date too.

Geoff

Doing some further digging, the hxtape process is intended for use on a linux system, using the linux PHP CLI (command line interface) i.e. NOT as part of Apache, or connected with .HTML files

It seems there is a Windoze variant of this, usually known as PHP.EXE - this appears to be a similar system, although it may not support ALL the functions used in the hxtape progs.   Have to try it and see, any not supported may have to be recreated within the package.

PHP.EXE comes as 'thread-safe' and 'not-thread-safe' variants.  I'd guess that for what we might want to do, the non-thread-safe versions (smaller) will be perfectly OK.

According to one of my books, if the full cmd line activation does not work you need to do it via a batch file.

Geoff

Hmm, found an item on the web.

Seems that hxtape package does use a linux library, something called OSS (Open Sound System), although how this is used in the progs I didn't spot.   This package may NOT be available to Windows.   In fact, it's not readily available for linux either, having been superceeded?

Do a google search for 'use of hxtape' and follow an item from Stephen Paukner - some usefull comments there?

Geoff

Rob,

I've been trying some digging, both in the PHP code, and about the web to find out more about some things.

Firstly, there's the oddity in the progs, in that the SAVE process is actually doing the loading of the data onto the tape (for the HX to LOAD) while the LOAD process is the opposite.   I suppose it depends on your thinking from the computer end, or the HX end?

Yes, as noted, the system uses the OSS stuff.   Which is assumed to be part of the Linux system.   The prog defines a 'stream' via dev/dsp, and then opens this for input or output depending on the module.   So what happens within the 'dsp' area is outside of our control?

The BIG question is what is the relationship between whatever happens within dsp and our present use of the .WAV file?   It may be, not a lot.   When I SAVE a file from my HX, the HX turns it into audio, which would happen anyway.   This audio goes into the computer audio input, and I assume into the DSP system?   I assume then that SoundRecorder (prog) gets the data from DSP, and at this point it's raw data?   It becomes a .WAV, and a specific variety of .WAV, ONLY when I save the data from SR?   Is that so?

There are some WinDoze processes regarding DSP that seem to be accessible, unfortunately the process for getting data out of a dsp block is more accessible (it seems) that the process to put data back.   The former is specifically sound.   The latter seems to be video and sound, although it could (?) be just sound, but either way seems rather more complicated.

I like the idea of having a specific file for each prog, or data file.   The .WAV format is rather BIG, but with current HD sizes this isn't a major problem, and reducing the bit rate helps.   So .WAV is OK.   The hxtape stuff doesn't hint at the size of his files, it would be a big help to see one?   I wonder is anyone have saved one - there are certainly .WAV files for HX20 on the web.   I'll look!

Oh, I see that the hxtape process uses a few 'wait' spaces?   Not sure yet if that's time based, or a block of data?

Thanks.

Geoff

Rob,

Talking to my brother just now.   He's got a number of linux machines, so may be able to sort out a run of hxtape with my HX-20 attached, to save something, and check load back.

Would that be any help, looking at the file that hxtape creates?

Brother is offshore just now, need to wait until he comes back.   So could be a couple weeks or so.

Actually, the laptop I've been using to save/load the files has linux on it, but I don't know too much about linux - just played a bit.   Linux is interesting, but it's such a chew on to do even the simplest of things.

Geoff

It could be interesting to see hxtape's output and compare.


Most probably I have not enough sync waves or way too much.


I'll try to give it another go this weekend.


Cheers,
Rob

Thanks Rob for the update.

I've looked again at the site I mentioned a couple of posts ago, re Stephan Paukner (NB, it's Stephan, not Stephen as I noted before).

I've sent him an email, in case he has a saved file he could let us have.   His blog posting suggests he HAS saved thing OK.

One thing I've just noted, he refers to the useage of the HXBASIC prog - the format of the use of this (to de-tokenise the prog) suggests that hxtape much be saving the file pretty much in the HX-20 format.   If you look at the code for the prog, it pretty much reads the file and replaces the tokens with the ASCII command, not much else.   Resulting in, in effect, the file as if it was saved with the ,A parameter (which to be honest is easy enough to do anyway).

If he sands anything/replies, I'll let you know.

Geoff

Rob,

I've got a reply from Stephan Paukner, with some attached files.

Not very helpful, I suspect.

The .BIN files seem to be encoded basic, and could well be just the memory image (or however it's stored) that would be in the saved file.   The .BAS files are the ASCII translation.   No hint of any of the 'punctuation' present in the saved data.   Is there evidence that the .PHP prog is removing stuff, so as to save what's here, and adding the framing etc data bach when putting it back to tape?

Oh - don't know why yet, but I was medding with that little data file from a while back, the one with the AAAA, BBBB, etc.  I wrote a little prog to read it back in, and it didn't work.   I tried the FILES command, and nothing.   I tried to load a couple of the progs, and they worked fine.   So that DATA1 file is NOT reliable.   I'll try something again.  The FILES command should work OK with a data file, just giving a couple of different codes with the filename compared with a basic prog file. 

Geoff

Rob,

I've re-done that test data file, very slightly altered.

This time, I did the usual save as a .WAV, which defaults to 44.1 Khz, 16 bit, stereo.   I then used the SndRec prog to cinvert the file, I did it to 8 Khz, 8 bit, mono, which VASTLY reduces the file size.

I then wrote another little prog to read the data from the reduced file, which worked fine.

I then tried the FILES command, this one correctly saw the file, and reported correctly on file name, type, etc.

So the earlier one was clearly dud in some way.

I then ran the reduced file through my own display prog (text wave only !) - this should display all samples (+) to the right, all samples (-) to the left, so should give a wave, of sorts, top to bottom.   Of course, a 1 Khz sample should have about 8 samples, and a 2 Khz wave should have 3 or 4.

For your info, I attach a zip with the new .WAV, also my processed wave.   Can you spot anything on the wave regarding data?

Geoff

Rob,

Ooops, I forgot to mention, although it may not matter to you.

The HX data file is called DATA4.   My then modified version is DATA48.

Geoff

I can read it right, my problem is saving the files back to wav.


[attachimg=1]



Most probably, as I said earlier, I have too many sync waves or less than needed.


Ok. Your file has almost 13 seconds of '1' bits before the first block, I have nowhere as much and that could be the problem.

Rob,

Thanks for the note.

Can you tell me the line numbers in my file for the 'first block' you refer to, and any other significant things?

Oh, on your dump, the 'Penalty...' etc stuff should not be there.   It's not part of the data.   But it looks like stuff from a previous LOAD I did, so it may have been in the buffer.   Or might it have been in your buffer?   Did I even send you 'FOOTBALL' (re American Football !)

Geoff

Hi Geoff!


Please try the attached wav file. I've make sure all the leading tones and spaces are exactly like the ones in your files. I hope it works this time!


Regarding the numbers, and assuming long gaps, they are as follows:


- 12584 '1' bits leading tone
- 1st copy of the header block, in the image from my earlier post, offset 0 to 85 (86 bytes)
- Interblock pause. 168 '1' bits followed by a small silence and then another 168 '1' bits.
- 2nd copy of the header block, offset 86 to 171 (86 bytes)
- 2344 '1' bits followed by a small silence and then another 2344 '1' bits.
- 1st copy of the data block, offset 172 to 433 (262 bytes)
- Interblock pause. 168 '1' bits followed by a small silence and then another 168 '1' bits.
- 2nd copy of the data block, offset 434 to 695 (262 bytes)
- 2344 '1' bits followed by a small silence and then another 2344 '1' bits.
- 1st copy of the eof block, offset 696 to 781 (86 bytes)
- Interblock pause. 168 '1' bits followed by a small silence and then another 168 '1' bits.
- 2nd copy of the eof block, offset 782 to 867 (86 bytes)
- 12584 '1' bits trailing tone


Cheers,
Rob

Rob,

Tried your file.   No response.

I tried the FILES command, nothing.  I tried different volumes, nothing.

I tried to LOAD the file, not sure if it was prog or data, but like I've said before, even getting an IO error would be something.

Nothing.

Tried a known file (something I's saved), no problem.   Worked.  So connections OK.

Can't see that the HX is even getting into the header, may not be seeing even the start of the data?

Thanks for the notes on the data, but not sure what the numbers refer to.   Doesn't seem to be the line (sample) ##s in my file.  Might be numbers from one of your progs.   How might the numbers refer to time, maybe I could work something out, as in file has xxx samples and is yyy secs long, therefore xxx/yyy samples per sec so 10 secs in should be about line lll.

Geoff

Just a couple of extra notes.

I think there was a comment in the PHP code regarding the sync bits, he said that the docs said 'at least 40 bits' but he's found that it needed to be exactly 40.   Bit, if you look at the code, where's there's a string of "000000000" ending with a "1", which should be the sync data, I think there are less than 40 "0" there.   Unless there's something else somewhere in the code.

Secondly, the wave form.   The docs refer to the 'normal' and the 'inverted' form, and I'm not sure they specify which to use.   The PHP code allows the use of a parameter to invert, but I think the default is normal.   I think I saw a note somewhere that suggests that the microcassette uses the inverted form, and the standard cassette (which all my bits should be) is using the normal wave (i.e. + part followed by - part).

Geoff

Well,


I cannot do much more without having access to a HX-20 and testing it myself.


Every parameter I can check is identical to the ones in you files. I've check that there are indeed 40 '0' bits at the end of the sync data, we tried reversing waveform...


I'm not doing anything else until I can borrow a machine and test it, as it's taking too much of the time I don't have. Sorry.

Rob,

Yes, I understand your point of view.   I'm sure that it's even more frustrating for you, when you have no control over parts of the process.

Thank you VERY much for what you've done so far.

I don't know what the chances of finding a HX-20 may be.   There seem to be quite a few still about, but in Spain?

There were some on the market just recently, someone in Poland selling some ex-Bundwehr.   Maybe someone in Germany as well.   Evidently, there had been quite a few machines, slightly modified re power supply and added ROM, but the basic function unaltered.

Even better, if you could find someone in Spain who has one?

As for next step, the only other thing I can think of right now would be to investigate the ROM code re the low-level routines covering the cassette system, to see if there are any low-level routines that can be used to provide any insight as to what data IS ACTUALLY getting in.   We know that data is getting OUT, we can see what's received.   So far, we've been depedant on the high-level routines to know what's getting back, and clearly they are fussy, i.e. they get the right data, or they get nothing.   The m/code is quite well documented.   I'm pretty sure that the bulk of the code is common to CAS0/CAS1 (microcas/ext cas) with specific entry points using common sub-routines, although lowest level ports will be different.

I've got the FORTH ROM in my HX, I can easily check the existing forth code for cass input, and maybe create a new version to report on the steps, or maybe even go in further?

Best wishes,

Geoff

Rob,

I've still got the FORTH angle to pursue, but I've found another angle.

Looking on the web trying to follow various leads, I found some info about a prog to analyse a WAV file by frequency.   Uses some soft of FFT maths, (fast Fourier t?).   Prog is reading WAV.

Prog is in PERL.   Uses separate libs for reading the WAV, that lib allows writing the wav back again.

I've installed PERL, got it running, got the WAV lib, installed that, and tested it.   Works quite well.   May well be helpful.   Seems to allow easy re-gen of the wav with various settings, i.e. 44,100, mono, etc, and also generation of sine waves at specified frequency.

Got the FFT lib now as well, but not installed that.   May have to change that a bit more than the test prog re wav, if I want the graphical output?  But may not need that, just need quick conv to zero (?), 1k and 2k (1.8k ??) frequencies?   If this prog is able to analyse the frequency spectrum, then it much be able to detect the different frequency components?

The important this, the PHP code we had before is substatially dependant on some audio components that we have co access to.   The PERL stuff is NOT, it's processing the raw WAV data, I guess, in a similar way to how you were.

I'll let you know more when I see if this helps any.

Thanks.

Geoff