Byte delivery ROM compression algorithm

[abalore]

Hello,

does anyone know about a compression algorithm with the following features?

- Can run from ROM with compressed data stored in ROM
- Have a small RAM footprint
- Can deliver uncompressed bytes one by one


Thank you!

Edit: Doesn't need to have big compression ratio, 1:4 would be good

[Prodatron]

ZX0 standard un-compressor:

https://github.com/einar-saukas/ZX0/blob/main/z80/dzx0_standard.asm

Beside the fact that it is fast and the compressor has a great compression ratio (like ZIP), it should fullfill all your needs:

- it can run from ROM (no self-modifying code or local variables)
- has no RAM footprint beside very little stack usage
- can deliver uncompressed bytes one by one, if you replace the LDIR command with something for your needs

[abalore]

ZX0 standard un-compressor:

https://github.com/einar-saukas/ZX0/blob/main/z80/dzx0_standard.asm

Beside the fact that it is fast and the compressor has a great compression ratio (like ZIP), it should fullfill all your needs:

- it can run from ROM (no self-modifying code or local variables)
- has no RAM footprint beside very little stack usage
- can deliver uncompressed bytes one by one, if you replace the LDIR command with something for your needs

Doesn't that LDIR need the previous content to be uncompressed into RAM? What I need is that the previous content is not uncompressed anywhere.

[Prodatron]

So you don't have the so-far uncompressed data somewhere?
Yes, ZX0 is using the already uncompressed data as a dictionary.
Ok, in this case I misunderstand the third requirement, and it won't work for you unfortunately.

[abalore]

The uncompressed data is to feed a device byte by byte, and there is no available RAM to uncompress the contents.

I'm developing a custom algorithm but still achieve compressions of 1:2, need to improve it to 1:4 at least.

[redbox]

The uncompressed data is to feed a device byte by byte, and there is no available RAM to uncompress the contents.

I'm developing a custom algorithm but still achieve compressions of 1:2, need to improve it to 1:4 at least.

If you're not going to use a buffer then you're likely restricted to bitwise operations.

When you say byte, is the data a full 8-bit byte (i.e. 0-255 in range)?  Or 4-bit or 2-bit even?

And do you have common patterns in the data?

If you posted some sample data it might be possible to come up with a better solution.

[abalore]

The uncompressed data is to feed a device byte by byte, and there is no available RAM to uncompress the contents.

I'm developing a custom algorithm but still achieve compressions of 1:2, need to improve it to 1:4 at least.

If you're not going to use a buffer then you're likely restricted to bitwise operations.

When you say byte, is the data a full 8-bit byte (i.e. 0-255 in range)?  Or 4-bit or 2-bit even?

And do you have common patterns in the data?

If you posted some sample data it might be possible to come up with a better solution.

It's music data in raw AY format. I developed a compression algorithm based on pattern search, with the addition of searching by channel, which makes the pattern finding a lot easier. I made a music player running from ROM over ROM music data with a RAM footprint of only 48 bytes. I will post the results soon in the form of a new disc to cartridge converted game.

[m_dr_m]

I have something like that for Orgams (to uncrunch help text and messages).
The dictionary is part of the data in ROM.

Right now there is no UI, it's not generic (byte 0 in source data has a special meaning: separation between messages) and it's a bit messy (interleaved with auto-ROM installation). But I can share the source !

[abalore]

I have something like that for Orgams (to uncrunch help text and messages).
The dictionary is part of the data in ROM.

Right now there is no UI, it's not generic (byte 0 in source data has a special meaning: separation between messages) and it's a bit messy (interleaved with auto-ROM installation). But I can share the source !

I made a pattern based routine that achieves around 10:1 compression in AY data. It's hard to share because it's a bunch of tools in both the CPC and in C# for ripping, channel separation, compacting zeroes, best pattern size finding, compression, and of course a ROM player. I'll share all eventually if I can put things in order to make it usable.

[GUNHED]

Hello,

does anyone know about a compression algorithm with the following features?

- Can run from ROM with compressed data stored in ROM
- Have a small RAM footprint
- Can deliver uncompressed bytes one by one


Thank you!

Edit: Doesn't need to have big compression ratio, 1:4 would be good

Well, I use Exomizer for that. Except that it doesn't do the 1 byte at a time thing. But I'm sure the decompression code can be altered.