Julian's Long List of Ideas Thread

[zhulien]

claude and gemini flavours added
proxy added e.g. https://cyborgshell.com/proxy/?url=  (for gemini)

[zhulien]

ZOSCII is ready for commercial use!  https://cyborgunicorn.com.au/zoscii/zosciibb/

Anyone want to create a CPC Assembly Language encoder?

I will try soon if nobody better than me can...

CP/M source here in the src/c folder.  https://github.com/PrimalNinja/cyborgzoscii

and Linux/Windows zencode, zdecode, zstrength

Please report any bugs to me.

[andycadley]

So it's basically a one time pad (except you might re-use the pad, which leaves it open to histogram attacks).

And you have to use 16-bit character codes on an 8-bit machine because they will be faster than 8-bit ones?? And it doesn't really explain what the "character code" is supposed to be pointing at (bitmap data?)

[zhulien]

So it's basically a one time pad (except you might re-use the pad, which leaves it open to histogram attacks).

And you have to use 16-bit character codes on an 8-bit machine because they will be faster than 8-bit ones?? And it doesn't really explain what the "character code" is supposed to be pointing at (bitmap data?)

No, one time pads relate to encryption and there is no encryption here. Zoscii relates to ASCII as its a character encoding which is the intent. If you look at the source and the whitepaper you will see the performance gain on an 8bit system vs ASCII lookups. It is quite clear.

It increases performance of text output by not requiring offset calculations to a base ASCII table. It removes the need to waste ROM space with an ASCII lookup table. It does use 2x RAM (or ROM if messages are stored there) vs ASCII but RAM is reusable and ROM usually is not.

The side effect of this scheme is an unfalsifiable undeterministic plausible deniable security option with a practical infinite entropy (if you use a random file as the ROM file).

If you want to communicate with someone with an unknown shared ROM file you have 100% security with a good enough ROM file (check the JS encoder/verifier tool for ROM strength). No message is ever actually sent.  If you want to make your most important data inaccessible to anyone without the ROM file you can have 100% secured ability to do so with zero of your data in the address file. Don't lose your ROM though,  your data will be lost forever.

[pelrun]

Saying "one-time-pads are encryption and this isn't encryption" is playing games with semantics. I can claim my face is an airplane but that doesn't make it true. A cryptologist would laugh in your face if you tried that argument on them.

This is worse than a one-time-pad (you're reusing the pad) and inherits all the same flaws that scheme has - sure, if you don't have a copy of the pad, you can't decrypt the ciphertext, but actually *keeping the pad secure* is hard. Exponentially more so once you need to communicate it to another party. If nobody is actively attacking you, sure, nobody is going to find your pad, but nobody cares if it was in plaintext either. But if you were under surveillance by a competent intelligence agency then this is going to fail and fail fast.

[andycadley]

It increases performance of text output by not requiring offset calculations to a base ASCII table. It removes the need to waste ROM space with an ASCII lookup table. It does use 2x RAM (or ROM if messages are stored there) vs ASCII but RAM is reusable and ROM usually is not.

It's much less efficient, in Z80, than just storing your character layout as a series of page aligned tables. In which case you can convert an ASCII code to a pointer with:

LD E, (HL) ; Where D holds a constant pointer to the character table

On other systems with a character mapped display, like the C64 it's even less efficient since you have to dereference a 16-bit pointer (expensive in 6502) in order to copy a single byte into video memory rather than just copying the byte in the first place.

If you want to communicate with someone with an unknown shared ROM file you have 100% security with a good enough ROM file (check the JS encoder/verifier tool for ROM strength). No message is ever actually sent. 

This is literally the definition of a one time pad. Except you have to use a different "ROM file" for every single message, otherwise it is subject to histogram attacks (over time it becomes obvious what the characters represent because distribution of letters in a language isn't uniform)

[MaV]

This is worse than a one-time-pad (you're reusing the pad) and inherits all the same flaws that scheme has - sure, if you don't have a copy of the pad, you can't decrypt the ciphertext, but actually *keeping the pad secure* is hard. Exponentially more so once you need to communicate it to another party. If nobody is actively attacking you, sure, nobody is going to find your pad, but nobody cares if it was in plaintext either. But if you were under surveillance by a competent intelligence agency then this is going to fail and fail fast.

This is literally the definition of a one time pad. Except you have to use a different "ROM file" for every single message, otherwise it is subject to histogram attacks (over time it becomes obvious what the characters represent because distribution of letters in a language isn't uniform)

Let's look at it from a more positive angle. This method probably would have postponed the premature death of Mary, Queen of Scots in 1587 by a couple of months which is a nice gesture, methinks!

[zhulien]

One time pad = illegal to export, stores payload as encrypted file.

Just because you can substitute a banana for an apple on your breakfast cerial doesn't make an apple a banana. People are more than welcome to confuse the two.

8bit performance, obviously I am not referring to character mapped output like a c64, where does a c64 add an offset of a table to a character position? We are also not comparing with an optimal way to handcode but the typical way most people do.

[zhulien]

So it's basically a one time pad (except you might re-use the pad, which leaves it open to histogram attacks).

And you have to use 16-bit character codes on an 8-bit machine because they will be faster than 8-bit ones?? And it doesn't really explain what the "character code" is supposed to be pointing at (bitmap data?)

Histogram attacks require some type of pattern to analyse. Not random address with no context. 

[zhulien]

CyborgZOSCII - Zero Overhead Secure Information Interchange
ZERO encryption ZERO ciphering 100% SECURE
Full Manual Now Online: https://zoscii.com/wiki/

[zhulien]

Also cpc basic compiler is almost finished, more a transpiler form a tiny basic to z80 assembly language. 

So far...

Complex expressions with correct precedence, if then else constructs, while wend constructs.  What I am missing and highly likely anyone can do a better job than me is...

- Keyboard scan for keypress and text input command without using push (if possible). Eg. Dw kw_input,buffer,buffer length to pop parameters off, but you shouldn't push. And dw kw_keyacan, keycode

It's already generating faster than typical compiles z80 code.

- fast text 

[zhulien]

CyborgDesktop 

[prevtenet]

No, one time pads relate to encryption and there is no encryption here.

A few thoughts on ZOSCII. This isn't a "ZOSCII is snake oil" critique - systems like ZOSCII have their uses - but especially if you're pursuing commercial applications, I recommend reading up more on current cryptographic techniques so you can engage with clients using the language and concepts they will expect you to know.

First, some bona fides. While my doctorate is in a different field, I sometimes moonlight as a historical cryptologist: I've published a few peer-reviewed papers on undeciphered manuscripts, broken a minor 17th-century epistolary cipher, and accumulated a small library of historical cryptographic manuals. In many ways ZOSCII has more in common with historical cipher systems than modern ones, so perhaps I can give some context that modern cryptographers might be lacking.

Formally, ZOSCII would be classified as a "shared-key homophonic substitution cipher." Let me break down the implications of those terms in reverse order:

Cipher: The US Committee on National Security Systems defines a "cipher" as "any cryptographic system in which arbitrary symbols or groups of symbols, represent units of plain text, or in which units of plain text are rearranged, or both." ZOSCII may not be a typical modern cipher (or subject to the same export controls), but it is 100%, objectively, a cipher. You cannot say that ZOSCII is not encryption or not a cipher. It is.

Homophonic substitution: A "substitution" cipher is one in which plaintext symbols (usually single letters) are substituted with matching secret symbols. A simple substitution cipher is one-to-one: B always becomes L, C always becomes Q, and so on. A "homophonic" substitution cipher is one-to-many: B might become 01, 34, or 82; C might become 72, 91, or 11; and so on.

Most diplomatic ciphers until the 19th century were homophonic substitution ciphers. The specific historical cipher that ZOSCII most resembles is a "book code." Both parties have a copy of the same book, and they encipher letters or words by reference - 24-14-3 might mean page 24, line 14, word 3, or a letter in a word. This is exactly how ZOSCII's ROM lookup works (with the same plausible deniability).

So why doesn't modern cryptography use homophonic substitution? Because it's not actually as secure as it seems. (Indeed, most historical homophonic ciphers have now been broken.) If the key is so much longer than the message that there are no repeated symbols, then yes, the encryption will be mathematically unbreakable; this is how one-time pads work. But if you send enough text with the same key, you will inevitably get repeats, and from there it's a relatively straightforward statistical process to reconstruct the plaintext and ROM.

Substitution ciphers are particularly vulnerable to cryptanalysis because they do nothing to hide the structure of the underlying text. Once you guess the identity of a symbol, it means the same thing everywhere it is used, so you can start using clues from the nonrandom structure of language to identify additional symbols. (Think Wheel of Fortune.) This same property makes substitution highly vulnerable to "cribbing": guessing likely sections of plaintext until one makes sense, progressively revealing more of the plaintext and key. This type of analysis was originally done with pen and paper, but is now easily automated by both classical and quantum computers.

Large numbers of alternate symbols (homophones) make all of this harder, of course, but only until the message gets long enough to start repeating symbols again. This is why nobody is going to solve the "ZOSCII challenge": it's too short and too random to permit cryptanalysis. Using the fact that nobody has broken it as evidence of ZOSCII's security doesn't make ZOSCII sound secure, it makes you sound like you don't know anything about cryptanalysis. The real test is, is ZOSCII still unbreakable after sending 1 GB of text using the same 64 KB ROM?

("That's not the intended use-case" is a perfectly valid response, but it does significantly limit the commercial applications.)

Shared-key: ZOSCII requires that both parties have the same ROM file (a "shared key"). This means that they must communicate "in the clear" about which ROM they are using, and if someone is snooping on that conversation, the whole game is lost from the start.

This is the bigger reason modern cryptography doesn't use substitution ciphers. Sure, if you make the key long enough, the message is secure; but how do you make the key exchange secure?

Modern cryptography solves this by using "public keys". Alice privately knows key A and publicly shares key B; Bob privately knows key C and publicly shares key D. Keys A-B and keys C-D are mathematically related so that messages encrypted with A+D can only be decrypted with B+C and vice versa. This is pretty much magic, and even after studying these ciphers at length I don't feel like I intuitively understand them. But that's the state of the art, and it produces mathematically secure encryption for indefinitely long messages without a shared key.

ZOSCII does have a useful niche here: there's some built-in security-through-obscurity, and public-key encryption is impractically slow on retro hardware, so for us retro fans this is a useful way to encrypt small personal files (i.e., collectively not much larger than the ROM). But the security for commercial-scale encryption is much less than currently stated.

[zhulien]

If ZOSCII is a cipher then so is ASCII.

If ZOSCII is a cipher so is any address table.

The definition you gave makes ciphering concept pointless. Its like me saying every human language novel is sourcecode which obviously it is as I already created a compiler (actually transpiler) from human text to z80 as a proof, it works 100%.  So I guess that also makes every data an executable too.  My compiler compiles test to jumps forward by how many characters each word is. Practical no... but it proved the point when someone argued otherwise.   Like your ZOSCII is a cipher. But in reality for practical purposes human language isn't usually considered sourcecode and neither is ASCII or address tables usually considered a cipher.

[andycadley]

There is no practical difference between a substitution cipher and a character encoding. Other than that a character encoding is public and therefore not attempting to obscure the underlying text.

[prevtenet]

There is no practical difference between a substitution cipher and a character encoding. Other than that a character encoding is public and therefore not attempting to obscure the underlying text.

Yes. To belabor this point:

Cambridge English Dictionary: "Cipher: a system of writing that prevents most people from understanding the message."

Merriam-Webster Dictionary: "Cipher: a method of transforming a text in order to conceal its meaning."

Britannica Encyclopedia: "Cipher: any method of transforming a message to conceal its meaning."

Wikipedia gives a more technical definition as used by cryptographers: "In cryptography, a cipher (or cypher) is an algorithm for performing encryption or decryption—a series of well-defined steps that can be followed as a procedure . . . In casual contexts, "code" and "cipher" can typically be used interchangeably; however, the technical usages of the words refer to different concepts. Codes contain meaning; words and phrases are assigned to numbers or symbols, creating a shorter message . . . Ciphers, on the other hand, work at a lower level: the level of individual letters, small groups of letters, or, in modern schemes, individual bits and blocks of bits."

Why is ASCII not a cipher, but ZOSCII is? As Andy pointed out, the difference is intended secrecy. If I write a letter in Cyrillic, that's just an alternate alphabet. If I write a letter in an alphabet that only I and the recipient know, with the intent to keep the message secret, that's a cipher. ZOSCII uses a secret ROM to save files in a format only those who know the secret can read. That makes it a cipher.

Here's the famous Mary Queen of Scots cipher mentioned earlier in the thread:



By zhulien's definition, this is just a lookup table, and therefore not a cipher. But this is almost literally the textbook definition of a cipher, one of the first examples most people will encounter when studying classical cryptography. (Formally, it's a hybrid type of cipher known as a "nomenclator.")

[zhulien]

ZOSCII has no rearrangement of characters at all, so doesn't fit that definition of a cipher. There is no transforming of a message. Intent to conceil is up to the user of ZOSCII. The primary purpose of ZOSCII is for printing text fast using addresses without offsets. ZOSCII is like directions to the treasure or a treasure map without telling you which planet it is for.

[prevtenet]

ZOSCII has no rearrangement of characters at all

Neither does the Mary Queen of Scots cipher.

There is no transforming of a message.

ZOSCII converts characters to their matching symbol in the lookup table, exactly like the Mary Queen of Scots cipher.

Intent to conceil is up to the user of ZOSCII.

Sure, but the concealment abilities of ZOSCII are heavily advertised on the website. If you wanted to draw a technical distinction you could say that "ZOSCII is an encoding scheme that can be used as a cipher if the encoding is kept secret."

I am not sure why you are reluctant to call ZOSCII a cipher, unless perhaps you are worried about export controls. I don't think ZOSCII would be export-controlled in AU since the underlying algorithm is basically just a book code, which should fall thoroughly under the public domain exemption; I'd actually be more worried about getting in trouble for false advertising, since you're currently advertising ZOSCII as both mathematically unbreakable and somehow not encryption, neither of which is true. In any case, you can't dodge the legal responsibilities of distributing a cipher by just not calling it a cipher.