News:

Printed Amstrad Addict magazine announced, check it out here!

Main Menu
T

Dumb question #15839 - BITBUSTER 1.2

Started by tastefulmrship, 12:44, 28 September 11

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

tastefulmrship

Hi all!  :-[
Please forgive even more of my petty ignorance!

I've been trying to get my head around this for a while now and I have to give up and admit I need help... again!

When I add some .pck files (created with BITBuster 1.2 on a PC) to my .dsk in WinAPE 2A18 they come up as Unknown ASCII/Binary files and, therefore, refuse to load directly into CPC memory; is there an easy way to get these files to show as Binary?

Thanks again,
- JTMS...

redbox

Have you ticked the 'Add/Remove AMSDOS headers' option in the WinAPE Disc Editor before you copy the file into the DSK from your PC...?

tastefulmrship

Quote from: redbox on 12:48, 28 September 11
Have you ticked the 'Add/Remove AMSDOS headers' option in the WinAPE Disc Editor before you copy the file into the DSK from your PC...?
Ah ha! Top stuff!! Thank you!

redbox

Quote from: tastefulmrship on 13:07, 28 September 11
Ah ha! Top stuff!! Thank you!


Np - I make that mistake all the time  :)


Funny to see we're both playing with crunchers today.


Do you have a link to the Bit Buster PC program and CPC decruncher routine?

tastefulmrship

Quote from: redbox on 13:10, 28 September 11
Do you have a link to the Bit Buster PC program and CPC decruncher routine?
Here is the one I'm using at the moment; http://cpcrulez.fr/applications_tools_cruncher_bitbuster_Z80.htm

However, I'm finding that I cannot resolve back to BASIC (or any other MC routine)... it must be something I've done.

redbox

Quote from: tastefulmrship on 13:14, 28 September 11
However, I'm finding that I cannot resolve back to BASIC (or any other MC routine)... it must be something I've done.


Well if you want to use it under BASIC it says in the source "You must save secondary registers if you want to use the routine under Basic." - but calling routine main (at beginning, being &A000) does this for you.  Of course you'll need to change the source (set to &4000) and destination (set to &C000) in main to where your code is and you want it to be depacked to before you compile the binary.


If you want to use it in MC, then just LD HL,source:LD DE,destination and CALL depack.  Assume most registers are destroyed I expect.


It does pack quite nicely, but not quite as well as Exomizer though.

redbox

Er, yeah.


Caught in an infinite loop it appears...


Use exomizer  8)

tastefulmrship

#7
The problem was the source files still had AMSDOS headers... and BITBuster doesn't really like AMSDOS headers!
I re-packed the headerless files and all is fine!

I've had differing results with all crossdev packers; there doesn't seem to be a 'best of', as it depends on the source file itself as to which packer results in the smallest file. Sometimes PUCruncher is better, sometimes Exomiser and so on. It's personal taste, I guess. BITBuster does the job perfectly for me.


EDIT:
Quote from: redbox on 13:38, 28 September 11
Er, yeah.
Caught in an infinite loop it appears...
Use exomizer  8)
Use headerless source files! Tick that AMSDOS header button before copying to PC.

redbox

Quote from: tastefulmrship on 13:40, 28 September 11
The problem was the source files still had AMSDOS headers... and BITBuster doesn't really like AMSDOS headers!


Hahahahaha, that's funny considering how the thread started!  Nice find though.

Quote from: tastefulmrship on 13:40, 28 September 11
I've had differing results with all crossdev packers; there doesn't seem to be a 'best of', as it depends on the source file itself as to which packer results in the smallest file. Sometimes PUCruncher is better, sometimes Exomiser and so on. It's personal taste, I guess. BITBuster does the job perfectly for me.


I agree, depends on the situation.  And exomizer requires a buffer which some others don't.


I'm writing some new software with lots of crunchers in though, so I'll include PUCrunch and BitBuster now.

TFM

Have you ever compared CPCT aginst PC crunchers? My experiences with CPCT are very good. Thank's Madram!
TFM of FutureSoft
Also visit the CPC and Plus users favorite OS: FutureOS - The Revolution on CPC6128 and 6128Plus

Nich

Quote from: TFM/FS on 16:35, 28 September 11
Have you ever compared CPCT aginst PC crunchers? My experiences with CPCT are very good. Thank's Madram!

In my opinion, CPCT is one of the best native CPC crunchers, but compression is very slow indeed, even on a CPC emulator with the speed set to maximum. The likes of Exomizer and MegaLZ offer much better compression than CPCT and have the advantage of very fast compression - but on the other hand, CPCT's decompression routines are much faster than Exomizer and MegaLZ.

I really ought to write a CPCWiki article on how to use Exomizer with a CPC... ::)

TFM

An article would be great!

How much better is exomizer compared to CPCT when looking at file-size?

Excuse my words, but I don't even give a crap on compression time. That's what I do once in a lifetime for a a given version of a program. So I can wait that minute (especially when using an emulator).

But what really matters is (aside of the size of the compressed file) the DEcompression time, because I will have to wait for the decompression every time I want to use the program. And if CPCT is better there (wow!). Unexpected, but that kick's PC-butts ;-)

However, if the filesize of a exomizer-compressed file is way better that other crunchers, that I wouldn't mind to wait 2 seconds longer for decompression. However a difference between 12 KB and 15 Kb wouldn't be an argmument for me.

I don't have exomizer, bitbuster and all that stuff, but it would be interesting to have some tables to compare their compression rate :-)

TFM of FutureSoft
Also visit the CPC and Plus users favorite OS: FutureOS - The Revolution on CPC6128 and 6128Plus

redbox

Quote from: TFM/FS on 21:17, 09 October 11
An article would be great!

Agreed.  Something to remember when using Exomizer is that the input has to be in lowercase, and not uppercase as shown in all the documentation.  So for example, the correct way to use it on your PC is:


exomizer raw filename.bin


and not


EXOMIZER RAW FILENAME.BIN


Took me a while to work out why it wasn't working when I was using it  ;)

Quote from: TFM/FS on 21:17, 09 October 11
I don't have exomizer, bitbuster and all that stuff, but it would be interesting to have some tables to compare their compression rate :-)

See here for a comparison: http://cpcrulez.fr/applications_tools_cruncher_COMPARATIF.htm

TFM

Yes, thank's I saw that before, but ... it's French!  :-X
TFM of FutureSoft
Also visit the CPC and Plus users favorite OS: FutureOS - The Revolution on CPC6128 and 6128Plus

redbox

#14
Ich finde deutschen einfacher als franzosisch, es ist wahr...

Aber franzosisch ist wirklich nicht zu hart TFM!


Original        &7025
CPC_T v3.0 Beta 1    &35D4
Exomizer v2.0 beta 6    &2AD5

TFM

French and Spanish are the most beautiful languages in the world.

To learn German must be a pain in the arse back. However even I can do a bit in English, and if I wouldn't have that pain in the head (no I had no alcolhol for 5 weeks!), I probably would get along with French better.
TFM of FutureSoft
Also visit the CPC and Plus users favorite OS: FutureOS - The Revolution on CPC6128 and 6128Plus

redbox

I found German much easier than French.  German generally sticks to the rules, French just breaks them all the time.

Exomizer does crunch an amazing amount, but I have heard of problems with decompression sometimes.  It has been used a lot though, Fano uses it in the new R-Type and Axelay mentions using it a lot.  SyX talked to me about compressors and he has found mixed results too.

But, as you say, it is all down to personal choice and thankfully there are a lot of options so you can see which one fits best in each situation.  Like you say, it's not always about the compression but sometimes about how fast decompression is etc.

Rather than a Wiki article about just Exomizer, it would be nice to see one about Crunchers in general which weighs up the pros and cons of each one.

SyX

#17
Apropos of compressor (the Babel subject is other world, jejejeje, and wait to add the dialects to the discussion ;) ), i don't trust in exomizer, i have used in a few projects without problems even in backward mode, but i have crunched a few files that the z80 decruncher can not decrunch and passed a lot of time debugging my code :-[

Because that, now i use aplib exclusively, the decruncher is short (only 163 bytes, while exomizer are 191 bytes + 156 bytes for the table  exo_mapbasebits) and noticebly faster than exomizer, but exomizer crunch better.

Another great thing with aplib is that i can decrunch in place easily, loading the file at the end of destination ram zone with only a clearance of 16 bytes (SOURCE_ADDRESS + LEN_DECRUNCH_FILE - LEN_CRUNCH_FILE + 16), normally is 3 ó 4 bytes, but with 16 i'm completely sure :) Although i would like it was possible to decrunch through $FFFF, because in Renegade i have a hollow of a few hundreds of bytes where put my code in $8000, and i would be nice make a big file from $8200 - $8000, instead of making two files ($0000-$8000 / $8200-$FFFF).

I have studied a few Z80 decrunchers from zx, msx, master system, gb, ... and a few 68000 decrunchers, i have converted packfire tiny mode to z80, and how aplib, it doesn't need a buffer in ram how exomizer (the table is in the first 26 bytes of every crunched file) and the decrunched files are the same size that exomizer (or even a few bytes smaller),  but is much much more slower and the tiny mode only can compress up 32 KBs, not a single byte more.

I have used the table and files that appears in MegaLZ Benchmarks and i have updated with these 3 compressors, plus ZIP and 7Z to compare them with modern compressors.(*) Table update in my next post

And i'm going to attach my packfire tiny decruncher, feel free to optimize, it's so slowwwwww  :P ; ---------------------------------------------------------------------------
; Packfire tiny decruncher for Z80 by Syx 2011
; Original MC68000 code by Franck "hitchhikr" Charlet
; ---------------------------------------------------------------------------

; ---------------------------------------------------------------------------
; packfire_decrunch: HL = Source / DE = Destination
; ---------------------------------------------------------------------------
packfire_decrunch
    PUSH HL
    POP  IX                     ; IX = tabla de frecuencias
    LD   BC,26
    ADD  HL,BC                  ; HL = Crunch data
    LD   A,(HL)                 ; Initilize bit counter (A)
    INC  HL
literal_copy
    LDI
main_loop
    CALL get_bit
    JR   C,literal_copy
    LD   IYL,$FF
get_index
    INC  IYL
    CALL get_bit
    JR   NC,get_index

    EX   AF,AF'
    LD   A,IYL
    CP   $10
    RET  Z                      ; Decrunch finish
    EX   AF,AF'

    CALL get_pair               ; Before the CALL IY < 256 always, after return can be IY > 255
    LD   (sm_len + 1),IY        ; Bytes to copy *** IY > 255 ***

    EX   AF,AF'
    LD   A,IYL
    CP   3
    JR   NC,es_mayor_2
    DEC  A
    JR   Z,es_1
    DEC  A
    JR   NZ,es_0
es_2
    LD   A,$04                  ; table_len[2]
    LD   BC,$0020               ; table_dist[2]
    JR   fin_cmp
es_1
    LD   A,$02                  ; table_len[1]
    LD   BC,$0030               ; table_dist[1]
    JR   fin_cmp
es_0
es_mayor_2
    LD   A,$04                  ; table_len[0]
    LD   BC,$0010               ; table_dist[0]
fin_cmp
    LD   (sm_d0 + 1),BC
    LD   B,A
    EX   AF,AF'

    CALL get_bits
    CALL get_pair

    PUSH HL
    LD   H,D
    LD   L,E                    ; A3 = A1
    LD   B,IYH
    LD   C,IYL                  ; BC = D3
    OR   A                      ; Clear Carry
    SBC  HL,BC
sm_len
    LD   BC,$0000               ; Can be > 255
    LDIR
    POP  HL
    JR   main_loop

; -------------------------------------------------------------------------------------------------
; Used Registers A (D7), DE (A1), HL (A2), IX (A0)
; Free Register BC, A', BC', DE', HL'
get_pair
    EX   AF,AF'
    EXX
    INC  IYL                    ; Here IY < 256 always
    LD   C,0                    ; A6 = 0.L

calc_len_dist
    LD   A,C                    ; D0 = A6.W
    AND  $0F                    ; D0 &= $0F
    JR   NZ,node                ; IF D0 == $x0 THEN D5 = 1.W
    LD   HL,1                   ; D5 = 1.W

node
    LD   D,A                    ; Save for the BIT
    LD   A,C                    ; D4 = A6.W (A6 <= 52 .B)
    RRA                         ; D4 >> 1
    LD   (sm_ix + 2),A
sm_ix
    LD   A,(IX + 0)             ; D1 = A0[D4].B (A0 => IX | D4 <= 26)

    BIT  0,D                    ; D4 = 1 | D0 &=D4 =>D0 &= 1 (Because D0 is changed next)
    JR   Z,nibble               ; Use low nibble
    RRA
    RRA
    RRA
    RRA                         ; D1 >> 4 use high nibble (nibble alternating)
nibble
    AND  $0F                    ; D1 &= $0F
    LD   B,A                    ; D1 must go in register B for get_bits

    LD   (sm_d0 + 1),HL         ; D0 = D5.W (In the last pass has the value for sm_d0)

    ; 1 << D1 (D1 = $0 - $F)
    LD   DE,$0000
    CP   8
    JR   C,ponlo_en_e
ponlo_en_d
    SUB  8
    OR   %01011000
    JR   sigue_generando_el_set
ponlo_en_e
    OR   %01111000
sigue_generando_el_set
    RLCA
    RLCA
    RLCA
    LD   (sm_set + 1),A
sm_set
    SET  0,A                    ; Transform in SET x,D or SET x,E

    ADD  HL,DE                  ; D5 += D4.L
    INC  C                      ; A6++
    DEC  IYL                    ; D3--
    JR   NZ,calc_len_dist

    LD   A,B
    EXX
    LD   B,A
    EX   AF,AF'

get_bits
    LD   IY,0
    INC  B

getting_bits
    DJNZ cont_get_bits

sm_d0
    LD   BC,$0000
    ADD  IY,BC
    RET

cont_get_bits
    CALL get_bit
    ; Simulate ADC IY,IY
    JR   NC,solo_duplica_iy
    ADD  IY,IY
    INC  IY                     ; It looks that INC IYL works fine
    JR   getting_bits
solo_duplica_iy
    ADD  IY,IY
    JR   getting_bits

; ---------------------------------------------------------------------------
get_bit
    ADD  A,A
    RET  NZ
    LD   A,(HL)
    INC  HL
    ADC  A,A
    RET

TFM

Thank you so much SyX!!! This is alrady a complete wiki article :-)))
TFM of FutureSoft
Also visit the CPC and Plus users favorite OS: FutureOS - The Revolution on CPC6128 and 6128Plus

SyX

Well, we would need to add the results with CPCT and others CPC native crunchers and then we will have the article :)

If you want TFM, i can do it, but tell me what CPC compressors i should test ;)

TFM

Quote from: SyX on 21:13, 10 October 11
Well, we would need to add the results with CPCT and others CPC native crunchers and then we will have the article :)

If you want TFM, i can do it, but tell me what CPC compressors i should test ;)

Well, in case we take the French site in account then CPCT and the latest Cheese would be enought I guess. But we can just copy-paste-steal the results from them ;-)
TFM of FutureSoft
Also visit the CPC and Plus users favorite OS: FutureOS - The Revolution on CPC6128 and 6128Plus

Octoate

@SyX: Thanks for your great effort to create a kind of benchmark :)! Btw, maybe CPCAlive (http://www.cpcalive.com/cpcalive_en.html - this page is a bit strange...) can help you with testing the CPC cruncher, because it should be possible to use and control CPCAlive directly from the command line and also use files on the harddisk. Maybe you can create a kind of batch file to test the CPC compressors with it.
--

SyX

Thanks Octoate, although i have used WinApe for it... the most boring was waiting to the crunchers finish its task even using the Full Speed/Frame Skip 50/Fast Disk Access (my netbook is guilty :P ),  than prepare the DSK with the files (that is automatized by a shell script).

All the files crunched are 1,2 Megabytes, if somebody want them, only ask ;)

And the table update with the native CPC crunchers is:       |  SIZE|MegaLZ|BITBUS|HRUS21|PCD6.2|RIP.01|CHEESE|CPCT31| CROWN| APLIB|   EXO|PFIREt| PFIRE|   ZIP|    7Z
-------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------
code-1 | 27932| 12654| 12827| 12865| 12904| 12118| 15771| 14262| 16558| 12222| 12207| 12215| 10833| 12680| 11155
code-2 | 14807|  7767|  8119|  8012|  8090|  7689| 10094|  9510| 11935|  7802|  7597|  7597|  7149|  7904|  7367
code-3 | 35840|  8406|  8181|  8291|  8373|  7891| 10785| 10179| 10772|  7964|  7810|   (1)|  6302|  8667|  6542
-------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------
scrv-1 |  6912|  4672|  4757|  4705|  4764|  4551|  5219|  5104|  5250|  4692|  4568|  4568|  4430|  4687|  4455
scrv-2 |  6912|  4074|  4170|  4144|  4173|  4030|  4691|  4802|  5317|  4116|  3971|  3970|  3901|  4228|  3980
scrv-3 |  6912|  3604|  3717|  3722|  3753|  3544|  4174|  4273|  4854|  3677|  3517|  3516|  3370|  3679|  3448
scrv-4 |  6912|  2211|  2270|  2359|  2366|  2165|  2783|  2696|  3697|  2228|  2063|  2063|  2008|  2301|  2121
scrv-5 |  6912|  4399|  4470|  4486|  4524|  4298|  5050|  5048|  5558|  4421|  4249|  4249|  4158|  4476|  4173
scrv-6 |  6912|  4233|  4300|  4345|  4364|  4129|  4873|  5084|  5634|  4291|  4093|  4093|  4053|  4297|  4086
scrv-7 |  6912|  3788|  3902|  3921|  3942|  3721|  4534|  4512|  5468|  3800|  3625|  3625|  3506|  3878|  3624
scrv-8 |  6912|  5275|  5345|  5308|  5373|  5139|  5928|  5888|  6480|  5305|  5080|  5088|  4994|  5015|  5344
scrv-9 |  6912|  4958|  5081|  5048|  5095|  4844|  5677|  5732|  6285|  5031|  4818|  4818|  4673|  5035|  4752
-------+------+------+------+------+------+------+------+------+------+------+------+------+------+------+------
text-1 |  6859|  3858|  4077|  4004|  4029|  3726|  5084|  4963|  6133|  3946|  3794|  3794|  3595|  3857|  3751
text-2 | 21746|  7341|  7766|  7549|  7470|  6525| 12202|  8178| 11223|  6878|  6584|  6584|  6047|  6642|  6175
text-3 | 28352| 10252| 11156| 10360| 10246|  8847| 17537| 11304| 15258|  9383|  8990|  8990|  8241|  8955|  8365
text-4 | 39159| 20406| 22246| 20892| 20752| 17334|   (2)| 23343| 30728| 19247| 17830|   (1)| 16451| 17534| 16584
text-5 | 25149| 10443| 11225| 10618| 10515|  9088| 15998| 11754| 16359|  9712|  9293|  9292|  8554|  9236|  8627

(1) Original file is bigger 32KBs, PackFire tiny doesn't compress
(2) Original file is bigger 38KBs, Cheese Cruncher doesn't compress

NOTES:
.- The CPC native crunchers are "Cheese Cruncher v2.2", "CPCT v3.1" and "Crown Cruncher v1.4".
.- All the files sizes are without Amsdos headers.
.- PFIREt is PackFire tiny mode and PFIRE is PackFire normal mode.
.- There is not CPC decruncher for ZIP, 7Z and PackFire normal mode.

It's not a surprise that the cross-platform crunchers (all based in LZ or Huffman algorythms, technology more known in the internet era) are better (not faster although aplib has a faster decruncher) than the native crunchers (normally RLE and similar technology well known in the 80s).

redbox

This is great work Syx.


Packfire rulez!  Just need to optimise and it's a real Exomizer beater.


What were you doing with Renegade?

TFM

#24
Thank's a lot SyX - great work!!!  :) :) :)

So... the best PC cruncher is "just" 25% better than the best CPC cruncher. Actually not really much. But once a while it will be needed :-)))
TFM of FutureSoft
Also visit the CPC and Plus users favorite OS: FutureOS - The Revolution on CPC6128 and 6128Plus

Powered by SMFPacks Menu Editor Mod