After I finished working on optimizing aPLib's decoder, I was told that pucrunch did compress better, and while looking for info about pucrunch (it turns out that sometimes aPLib wins, but most times pucrunch is a bit better), I found another packer that clearly beats both on compression: exomizer 2.
http://hem.bredband.net/magli143/exo/ (http://hem.bredband.net/magli143/exo/)
Since there was no Z80 decompressor, I made it (that was back in 2007), and also made an optimizer to take the output and remove two unneeded bits and rearrange the bit organization, so depackers can be optimized a bit more. A few days ago Antonio Villena optimized the depackers, making them faster and also chopping a few bytes off them, so I'll take this chance to start this topic.
The depackers sizes are now between 163-187 bytes, and they also need space to generate a table of 156 bytes, but that table can be discarded after the decompression is done, so if you place it in a buffer it shouldn't take any extra memory. Speed is slow, because the algorithm is complex.
http://www.speccy.org/metalbrain/exo_v4.zip (http://www.speccy.org/metalbrain/exo_v4.zip)
The normal versions of the depackers use the output from exomizer, in raw mode forwards, so you produce the data file with command line:
exomizer raw -o output input
The optimized versions also require to use my optimizer, so the command lines would be:
exomizer raw -o temporal input
exoopt temporal output
exoopt is the data optimizator, the Windows executable is placed in optimized/bin, and the sources for other systems are in optimized/scr
As for the difference between deexo.asm and deexo_simple.asm, the simple version requires the table to be aligned in a 256 boundary, and doesn't handle literal sequences (which are rare, and you may also force exomizer not to generate them using the parameter -c).
Is there a dramatic decompression speed difference between this and the original deexo code? I tend to use BitBuster these days because the Z80 decompressor is so much quicker, but Exomizer does pack better in ~90% of files; this normally includes the 156 byte table for deexo.
A quicker deexo would sway my usage back to Exomizer.
Quote from: tastefulmrship on 20:05, 17 January 12Is there a dramatic decompression speed difference between this and the original deexo code? I tend to use BitBuster these days because the Z80 decompressor is so much quicker, but Exomizer does pack better in ~90% of files; this normally includes the 156 byte table for deexo.
A quicker deexo would sway my usage back to Exomizer.
I haven't tested it yet. The main difference is in the exo_getbit routine, which now takes around 17 states less than before (and it's called many times). I think the difference may be noticeable, but not dramatic.
Here's a little test;
AMPSKULL.SCR - Normal &4000 screen file.
AMPSKULL.BIT - BITBuster - 8700 bytes long.
AMPSKULL.EXO - Original Exomizer deexo code. - 8567 bytes long.
AMPSKULL.EX2 - The new Exomizer deexo code using optimizer! - 8571 bytes long.
... using the following commands in a CMD box.
exomizer raw ampskull.scr -o ampskull.exo
exoopt ampskull.exo ampskull.ex2
You can see the decompression times for a &4000 byte screen.
It would be great to compare with some very well done Crunchers on CPC side. For example MadRams TurboCruncher!
Quote from: TFM/FS on 22:31, 17 January 12
It would be great to compare with some very well done Crunchers on CPC side. For example MadRams TurboCruncher!
There's a "compression size" table half-way down this page (http://www.cpcwiki.eu/forum/programming/dumb-question-15839-bitbuster-1-2/15/) by
SyX, but not one for "decompression times" that I can find.
For interest; I used TurboCruncher to pack my original NTWICCT disk, but my next version will be BITBuster. Initial builds are saving 5k on the main executable alone! I plan to pack the tunes as well, so I could possibly fit two "sides" on one .dsk!
EDIT:
Also, if you want to do a "proper" comparison between all packers (PC & CPC), then this should be done against a large number of files; executable & data files of differing sizes and memory locations. A lot of the CPC packers are primarily used for executable binary compression, but are they any good when it comes to simple data compression, like a screen?
A packer like Exomizer (or Packfire) is good for 64k game/demo compression where you're looking for memory space rather than decompression speed.
One like BITBuster would be better suited to trackmo decompression due to its speed; however, I guess most demo coders would prefer to load in the raw data "whole" to cut down on CPU time lost decompressing the loaded data chunks, as sector loading can be done while showing the current routine or as part of a transition... per or alternate frame.
Thx, I know that table.
I ask because compression times and space is very different for different data sources. f.e. text, pictures, code, sound etc.
Several months ago, Urusergi started optimizing the depacker routines, starting a process that has taken almost three months, involving great contributions by Antonio Villena and even a few bits myself. This process has recently culminated (or at least I hope so :P ).
On one hand, the depackers have been reduced a lot, and they're also a bit faster. On the other hand, there are new alternate versions (not so small) optimized for speed, so we can choose between four variants (between 0: smallest size to 3: highest speed). Variant 1 is specially attractive, because at the price of 3-5 bytes, it has a 14% speed gain.
Also, the optimizator program (exoopt) has been rewritten from scratch by Antonio Villena, so now you can pass the speed variant (and direction, that used to be always forward) you want as parameter, and it will automatically generate the depacker you need. You can even enter several files and it will produce the simple version if there are no literal chains, and the complete one if they exist. This new optimizator, does what it did before (inverting the read order of the bitstream, and removing 2 redundant bits) and more, doing some other changes in the data that in rare occasions might be bad (increasing the data size a bit), but allow for futher simplification of the depackers.
The new versions can be found here:
SourceForge.net Repository - [emuscriptoria] Index of /deexo (http://emuscriptoria.svn.sourceforge.net/viewvc/emuscriptoria/deexo/)
And an already compiled exoopt, here:
http://retrolandia.net/foro/showthread.php?tid=48&page=6 (http://retrolandia.net/foro/showthread.php?tid=48&page=6)
The history of the optimizations, in case someone is interested, started here (in spanish):
http://amstrad.es/forum/viewtopic.php?f=6&t=2608 (http://amstrad.es/forum/viewtopic.php?f=6&t=2608)
A wiki page with all this information, code and sources would be very useful in future when the above links have been "lost" among the thousands of other posts. 8)
I've been using MegaLZ because when I started that was the only thing I found. Seems most people use Exomiser on the CPC though, so I'll definitely look into this.
Quote from: steve on 12:33, 27 January 13
A wiki page with all this information, code and sources would be very useful in future when the above links have been "lost" among the thousands of other posts. 8)
That's right, you should create one :P :) ;)
Quote from: steve on 12:33, 27 January 13
A wiki page with all this information, code and sources would be very useful in future when the above links have been "lost" among the thousands of other posts. 8)
I created an
Exomizer wiki page (http://cpcwiki.eu/index.php/Exomizer) a while ago, but I don't have time to update it at the moment as it currently doesn't explain how to use the optimised versions of the package.
That's great, I did a search of the wiki before I made that post, but I spelt exomizer incorrectly so your page was not found. ::)
Haha! You are not alone :laugh:
Beware! A bug has been detected in the official depackers. Magnus will surely publish an updated version (2.07) soon. Until then, you may get the latest versions from the repository:
SourceForge.net Repository - [emuscriptoria] Index of /deexo (http://emuscriptoria.svn.sourceforge.net/viewvc/emuscriptoria/deexo/)
More info (in spanish) here:
http://amstrad.es/forum/viewtopic.php?f=6&t=2608&start=150 (http://amstrad.es/forum/viewtopic.php?f=6&t=2608&start=150)
Hi everyone,
We've just made a small new release. The code hasn't changed at all, but after being asked a pair of times about the depacking routines license, I've decided to put them under the LGPL 2.1 . The license and updated files can be found here:
https://sourceforge.net/p/emuscriptoria/code/HEAD/tree/deexo/ (https://sourceforge.net/p/emuscriptoria/code/HEAD/tree/deexo/)