Turrican (128K)

[Jean-Marie]

While I'm not a fan of this new palette, if many people request it then Vox populi, vox Dei.
Now, there are also people who will prefer the regular one. So maybe a poll should be taken?

[eto]

So maybe a poll should be taken?

or just an option when starting the game? Not sure if that's feasible.

[Jean-Marie]

or just an option when starting the game?

Yeah, I guess I could include alternate Levels A,B & C with a new palette, since only the first 3 levels are concerned. I will check this out; for the moment I'm struggling to find 100 bytes free to make the Spikes work with Axelay's optimized scrolling!

[dodogildo]

or just an option when starting the game? Not sure if that's feasible.

That would be great, for the new palette.

[Jean-Marie]

The spikes are working too. Now, I'll see if I can tidy up the code, and next week I'll try to write the changes in the Level A file, which should be very challenging.

[lightforce6128]

next week I'll try to write the changes in the Level A file, which should be very challenging.

Hopefully it will not be challenging. For compression you can use any language, and modern computers will process the data in any language (also slow scripting languages) more than fast enough.

Whenever you find a sequence of the same byte A of length N, replace it with these four bytes: A,N,#A5,#CC. Only apply this if the sequence is longer than 4 bytes. That's all. This is not the best compression, but a fast one that will work acceptably on sprites and level maps with much background.

Example:

"1,2,3,4,  a,a,a,a,a,a  ,5,6,7,8"  gets  "1,2,3,4,  a,6,#A5,#CC  ,5,6,7,8" (spaces only for better readability)

Hopefully the used escape sequence #A5,#CC is not used anywhere in the data to compress. If it is, either the escape sequence can be changed to something else, or it has to be encoded as sequence of length 1 (what will cost 3 bytes):

Example:
"1,2,3,  #A5,#CC  ,4,5,6"  gets  "1,2,3,  #A5,1,#A5,#CC,  #CC  ,4,5,6" (replace the #A5 with a sequence of length 1)


Two questions remain:

• Where should the file be loaded? Usually at the begin of the buffer that will contain the uncompressed data later.
• And which routine says the decompressor where the compressed data is located? Needed are begin and end of the data.

[Jean-Marie]

I think the real difficulties will come with the levels where Parallax is used. Daren used animated tiles to create the effect, where background tiles are "slided" in the Tile Map. This is quite ingenious.
Level I (4.1) looks like a real nightmare with a slew of animated tiles (not only for the parallax).
It's quite easy to carry out LDI/LDD instructions on some linear memory, but with Axelay's optimization, the tiles offsets become shuffled, and that makes thing more complex and lengthy 

 

Code Select Expand

;;TURRICAN.LVI

org &1b78
ld ix,&1d07
ld b,&0f
ld a,(&c7f3)
ld c,a
l1b82:
ld l,(ix)
ld h,(ix+1)
ld a,l
cp &20
jp nc,l1ba6
bit 0,c
jp z,l1b94
inc h
l1b94:
bit 1,c
jp z,l1b9a
dec l
l1b9a:
bit 2,c
jp z,l1ba0
dec h
l1ba0:
bit 3,c
jp z,l1ba6
inc l
l1ba6:
ld (ix),l
ld (ix+1),h
inc ixl
inc ixl
djnz l1b82
ld a,(&c7f3)
rra
jp nc,l1bfc
ld de,&1920
ld hl,&6cb0
REPEAT 8:ldi:ENDM
ld de,&6cb0
call @LDIR-24-24
ld hl,&1920
REPEAT 8:ldi:ENDM
ex de,hl
ld e,&20
call @LDIR-8-8
ld de,&6cd0
call @LDIR-24-24
ld hl,&1920
call @LDIR-8-8
l1bfc:
ld a,(&c7f3)
bit 1,a
jp z,l1c28
ld ix,&6cb0
ld b,&10
l1c0a:
ld e,(ix)
ld d,(ix+1)
ld l,(ix+&20)
ld h,(ix+&21)
ld (ix),h
ld (ix+1),e
ld (ix+&20),d
ld (ix+&21),l
inc ixl
inc ixl
djnz l1c0a
l1c28:
ld a,(&c7f3)
bit 2,a
jp z,l1c72
ld de,&1920
ld hl,&6cc8
REPEAT 8:ldi:ENDM
ld de,&6ccf
ld l,&c7
call @LDDR-24-24
ld e,&b0
ld hl,&1920
REPEAT 8:ldi:ENDM
ex de,hl
ld e,&20
ld l,&e8
call @LDIR-8-8
ld de,&6cef
ld l,&e7
call @LDDR-24-24
ld e,&d0
ld hl,&1920
call @LDIR-8-8
l1c72:
ld a,(&c7f3)
bit 3,a
jp z,l1c9e
ld ix,&6cb0
ld b,&10
l1c80:
ld e,(ix)
ld d,(ix+1)
ld l,(ix+&20)
ld h,(ix+&21)
ld (ix),d
ld (ix+1),l
ld (ix+&20),h
ld (ix+&21),e
inc ixl
inc ixl
djnz l1c80
l1c9e:
ld a,(&dea0)
ld hl,&3e15
rra
jp c,l1cac
ld l,&55
l1cac:
ld de,&6c00
call @LDIR-64-64
ld ix,(&6c7e)
ld iy,(&6c7c)
ld e,&7f
ld hl,&6c7b
call @LDDR-60-60
ld (&6c42),iy
ld (&6c40),ix
ld hl,&6be0
ld de,&1920
call @LDIR-16-16
ld a,(&dea0)
rra
ret c
ld de,&6be0
call l1cef
ld hl,&1920
l1cef:
ld b,&08
l1cf1:
ld c,(hl)
inc l
ld a,(hl)
inc l
exx
ld b,&87
ld c,a
ld a,(bc)
exx
ld (de),a
inc e
ld a,c
exx
ld c,a
ld a,(bc)
exx
ld (de),a
inc e
djnz l1cf1
ret
REPEAT 9:rst 0:ENDM

ORG &9D80
REPEAT 95:ldi:ENDM
@LDIR:
ret

ORG &9E40
REPEAT 95:ldd:ENDM
@LDDR:
ret


[lightforce6128]

I think the real difficulties will come with the levels where Parallax is used. Daren used animated tiles to create the effect, where background tiles are "slided" in the Tile Map. This is quite ingenious. Level I (4.1) looks like a real nightmare with a slew of animated tiles (not only for the parallax).

When I first saw this, I was really impressed. Paralax scrolling was not really common for the CPC. Until recently I thought it would use some kind of optical illusion, where the tiles contain the same background, but shifted differently then the tile offset. But this could not be used here, because it would interfere with reusing tiles throughout a level.

I did some further analysis and now have a better understanding about how the drawing works. There are three levels of image generation, not two!

• The map of e.g. size 137x51 contains indices of blocks (not of tiles!).
• There are 256 blocks, each containing 4x4 tile indices.
• Finally there are 256 tiles, each containing 4x8 pixels.

With this trick the game achieves a compression ratio of 118:1 (or reduces the size by 99.2%). The drawn map has a size of 2192x1632 or 3.6 megapixel, while the three data structures only need 14.8 kilobytes.

By looking at tiles and blocks of level I (4.1) I found there are several blocks showing paralax background. But there are only four tiles (index 203 to 206). The whole paralax scrolling magic is done only in these four tiles!

[lightforce6128]

For better visualization I append the map, blocks, and tiles of level A (1.1). I suppose this is not a big spoiler because almost everybody will know level 1.1 more than good enough.

[Jean-Marie]

I've managed to have the first level written on disc. It includes Axelay's optimized routines, with corrections for the animated tiles. 
I had it working by removing the compression process altogether. The drawback is that each level files will be 26 Kb :/
I worked out a small Visual Basic macro to compress data with the RLE algorithm; it's working nicely but but there are still some problems during the decompression conspicuously. I'll try to investigate further.
 

[Axelay]

I've managed to have the first level written on disc. It includes Axelay's optimized routines, with corrections for the animated tiles.
I had it working by removing the compression process altogether. The drawback is that each level files will be 26 Kb :/
I worked out a small Visual Basic macro to compress data with the RLE algorithm; it's working nicely but but there are still some problems during the decompression conspicuously. I'll try to investigate further.
 

I don't have time to look into this at all, so apologies if these suggestions are unworkable or have already been considered, but would it be possible to leave the level data as is and just have an assembly routine in memory that reformats the tile data after loading?  Or if there is no spare memory for that to be permanently in memory, or there are additional things you have needed to add to make it work, like for the animated tiles, could that simply be appended to the existing level data file, so then you only perhaps need add an extra call to some temporary code added to the data load that sorts things out after the original level data load has done it's 'usual' process?

[Jean-Marie]

Your "shuffled" data is working nicely as it is, I was able to adapt the code displaying the waterfall and the exhaust flames without too much overheads (Thanks again BTW!).
For the spikes, I had to append a "clean" copy of the original tiles (non-shuffled) at the end of the level file, indeed, as it was too complicated & lengthy to modify the original code to make it work with the shuffled tiles.
The problem stems from writing the new file on the disc, as Daren used some intricated code to uncompress the Data. But hopefully, I'll find a solution. I just need to have a serious look at the decoding routine.

[lightforce6128]

The problem stems from writing the new file on the disc, as Daren used some intricated code to uncompress the Data. But hopefully, I'll find a solution. I just need to have a serious look at the decoding routine.

Because I already spent some time on this: Can you describe a situation where it does not work?

As far as I remember, there is one access to tile 0 that cleans it after loading, for whatever reason.
Also some other tiles are used to transport other data (e.g. font data) and are reused afterwards for animation purposes.

What is also important for decoding is not the process itself, but where the compressed data is placed before decompression starts. Because you applied several changes, this position will be different from before. Until now I did not find the code location where this position is stored. It surely needs to be altered.

[Jean-Marie]

I'm progressing : I have located a part of the code which caused some troubles, at 1B39h.
It moved the tiles of the spikes & flames to a buffer used for the display. This was messing up with the shuffled data obviously. Actually, this part of the code can be safely removed because I use no buffer for drawing the spikes & flames.
Only the end of the procedure remains mysterious to me (after the palette setting).

Code Select Expand

org &1B39
ld bc,&002c
ld de,&d7cc
ld hl,&d7cb
ld (hl),&0004
ldir
;;ld bc,&0030
;;ld de,&6dc0
;;ld hl,&6930       ;;move spikes tiles to buffer
;;ldir
;;ld bc,&0040
;;ld de,&6d40
;;ld hl,&6cb0         ;;move flames tiles to buffer
;;ldir
ld hl,&1b1f
call &d600             ;;set palette
ld b,&0006
ld ix,&243e
ld de,&0006
ld hl,&24fe
l1b6e:
ld c,&0010
l1b70:
ld (ix+&0000),l
ld (ix+&0001),h
inc ix
inc ix
l1b7a:
ld a,(hl)
inc hl
cp &00d3
jp z,l1b85
add hl,de
jp l1b7a
l1b85:
dec c
jp nz,l1b70
djnz l1b6e
ret

Until now I did not find the code location where this position is stored. It surely needs to be altered.

I found the decoding routine at B000h, and the BC register seems to contains the size of the compressed data. I guess I'll have to modify it since the size is a bit larger with the new shuffled data.

Code Select Expand

org #b000
ld hl,(#b05b)      ;;hl=1BB0
ld de,(#b05d)      ;;de=1AFF
ld bc,(#b05f)      ;;bc=481A (data size?)
inc bc
ldir
ld ix,(#b063)      ;;ix=7FFF
ld hl,(#b065)      ;;hl=6319
ld de,(#b059)      ;;de=A5CC (escape code)
ld bc,(#b05d)      ;;bc=1AFF
lb01d:
ld a,(hl)
cp e
jr z,#b03f
ld (ix+#00),a
dec hl
dec ix
push af
ld a,h
cp b
jr nz,lb030
ld a,l
cp c
jr z,lb033
lb030:
pop af
jr lb01d
lb033:
pop af
ld a,(#b067)
or a
nop
nop
ret

[lightforce6128]

Until now I did not find the code location where this position is stored. It surely needs to be altered.

I found the decoding routine at B000h, and the BC register seems to contains the size of the compressed data. I guess I'll have to modify it since the size is a bit larger with the new shuffled data.

But not only the size. The location is important as well. The starting point is not the begin of the data (what stays the same), but its end, because processing is done backwards. The end of the compressed data is definitely no longer where it was before.

[Xyphoe]

Just popping in to say, I prefer the original palette. (No offence to MacDeath - that is nice too, but it's just personal preference)  

[Jean-Marie]

I got the first level working under its newly compressed form  Many thanks to @lightforce6128 for the hints!

[lightforce6128]

After looking at the drawing of tiles, now I had a look at drawing of sprites. This is by far more complex and distributes over several longer routines. The following BASIC-like pseudo-code gives a coarse overview about what is happening there. The given addresses map to the begin of routines or code blocks.

Code Select Expand

0DA9: copyTilesToScreen()
0E28: FOR EACH sprite IN spriteList DO
0E3A:    IF sprite IS ExtrasSphere THEN
0E60:        copySmallerExtrasSphereSpriteToATemporaryNormalSprite()
----:    END IF
AEF0:    WITH sprite DO
A7C0:        mirrorHorizontally()
9F00:        mirrorVertically()
AF04:        clipHorizontally()
AF48:        clipVertically()
AFC3:        copyNonTransparentBytesToScreen()
----:    END WITH
0EA9: NEXT
0EB2: drawSpriteWithoutClipping(mine)
0EB9: drawSpriteWithoutClipping(player)
0EE8: drawSpriteWithoutClipping(shield)
DE00: FOR EACH explosion in explosionList
DE2A:    getAnimation()
DE11:    drawSpriteWithoutClipping(explosion)
DE14:    updateAnimation()
----: NEXT
0EEE: swapScreenBuffers()
----:
DF02: SUB drawSpriteWithoutClipping(sprite) : ... : END SUB
Also I would like to, I cannot provide a well commented disassembly. This would be by far too much work. But at least some discoveries I would like to share:

• There is not any kind of caching or reuse.
• In each frame, the extras spheres are converted to normal sprites again.
• In each frame, each single sprite (that needs to) is mirrored again.
• The converted sprites with the extras sphere are transparent at more than 50% of the bytes. Nevertheless each transparent byte is built up during the conversion and skipped later while drawing the sprite. In each frame, for each single extras sphere.
• There are smaller and bigger explosions. It seems, the latter one is not used.
• The smaller explosions skip their last animation step due to a programming error.

For the loop for drawing a sprite I can provide a disassembly. I think this can be optimized. Using this routine takes ~3000 NOPs to draw a single sprite. This is 15% of a frame. Drawing 6 sprites eats up a whole frame. Bad things will happen if an extras box is destroyed and releases a bunch of extras spheres.

Code Select Expand

ORG #AFC3

;; This routine draws a single sprite after mirroring and clipping have been applied.
;;
;; B  - clipped height (full height is 24)
;; C  - clipped width (full width is 6)
;; DE' - source address
;; HL' - target address
;; (source_skip-2)  - bytes to skip in source for each row
;; (target_skip-2)  - bytes to advance target for each row
;; (jump_condition) - set to #CA (JP Z) or #C2 (JP NZ)

draw_sprite:

    y_loop:                                                            ;;             ;;
        LD A,C                                                         ;;  1          ;;
        EXX                                                            ;;  1          ;;
            ;;                                                         ;;  |          ;;
            x_loop:                                                    ;;  |          ;; To write one byte, this inner loop takes 17 NOPs.
                EX AF,AF'                                              ;;  |----- 1   ;; Save A.
                    LD A,(DE) : INC DE                                 ;;  |    +-4-+ ;; Load sprite data and update source pointer.
                    OR A : JR Z,$+3                                    ;;  |    3  4  ;; Is byte opaque?
                        LD (HL),A                                      ;;  |    +---2 ;; Write byte.
                    INC HL                                             ;;  |      2   ;; Update target pointer.
                EX AF,AF'                                              ;;  |      1   ;; Restore A.
            DEC A : JP NZ,x_loop                                       ;;  |----- 4   ;;
            ;;                                                         ;;  |          ;;
            EX DE,HL                                                   ;;  1          ;; Swap registers to allow DE=DE+BC.
                LD BC,#0000 :source_skip                               ;;  3          ;; Skip bytes in source data.
                ADD HL,BC                                              ;;  3          ;;
            EX DE,HL                                                   ;;  1          ;;
            ;;                                                         ;;  |          ;;
            LD BC,#07FA :target_skip                                   ;;  3          ;; Go to next row and skip bytes on screen.
            ADD HL,BC                                                  ;;  3          ;;
            ;;                                                         ;;  |          ;;
            BIT 6,H                                                    ;;  2          ;; Was the screen memory left?
            jump_condition: JP Z,$+7                                   ;;  3          ;;
                LD BC,#C040 : ADD HL,BC                                ;; (6/8)       ;; Then correct target pointer.
            ;;                                                         ;;  |          ;;
        EXX                                                            ;;  1          ;;
    DJNZ y_loop                                                        ;;  4-1        ;;
    ;;                                                                 ;; ----  ----- ;;
    ;;                                                                 ;; 26-1  15|18 ;; (6*(15+18)/2+26+6/8)*24-1 = ~3017 NOPs per sprite.

[lightforce6128]

Only the end of the procedure remains mysterious to me (after the palette setting).

I tried to understand what is going on there, but also without success. The additional code somehow builds up an index (a table with addresses) for another, more complex data structure. Maybe this is somehow related to sprites, maybe it is something completely different.

Code Select Expand

ORG #1B39

    prepare_tiles:

        LD HL,#1B1F : CALL set_table_of_firmware_inks                ;; Set palette.
        LD BC,#002C : LD DE,#D7CC : LD HL,#D7CB : LD (HL),#04 : LDIR ;; Fill buffer with vertical stripes (blue-yellow). Why?
        LD BC,#0030 : LD DE,#6DC0 : LD HL,#6930 : LDIR               ;; Create copy of sprite. This overwrites font data "789".
        LD BC,#0040 : LD DE,#6D40 : LD HL,#6CB0 : LDIR               ;; Create copy of big flame. This overwrites font data "_012".

        LD B,#06                                                ;; Counter for outer loop.
        LD IX,#243E                                             ;; Target address. Table reaches below source address (length: 192, 96 entries).
        LD DE,#0006                                             ;; Source step size.
        LD HL,#24FE                                             ;; Source address.
        outer_loop:                                             ;;
            LD C,#10                                            ;; Counter for inner loop.
            inner_loop:                                         ;;
                LD (IX+#01),H : LD (IX+#00),L : INC IX : INC IX ;; Store source address in target entry.
                search_marker_loop:                             ;;
                    LD A,(HL) : INC HL                          ;;
                    CP #D3 : JP Z,marker_found                  ;;
                    ADD HL,DE                                   ;; Go to next source record.
                JP search_marker_loop                           ;;
                marker_found:                                   ;;
            DEC C : JP NZ,inner_loop                            ;;
        DJNZ outer_loop                                         ;;

        ;; Here:
        ;;     IX            - The target pointer has reached the begin of the source at #24FE.
        ;;     HL            - The source pointer has reached the begin of sprite data at #2BD2.
        ;;     (#243E)       - Target entry 0 points to a value #D3.
        ;;     (#243E+n*2)   - Other target entries point to a value behind #D3.
        ;;     ((#243E+n*2)) - a sequence like ( [type] [adr_low] [adr_high] [?] [?] [?] [?] )+

        RET



ORG #D600

    set_table_of_firmware_inks:

        ;; HL - Table of 17 firmware inks (for 16 pens and border).

        LD E,(HL)                           ;; E is loaded, but not used.
        LD D,#10                            ;; Pen counter, 16..1.
        pen_loop:                           ;;
            LD A,D : SUB #10 : NEG : LD C,A ;; Pen, 0..15.
            LD A,(HL)                       ;; Read firmware ink from table.
            CALL set_pen_to_firmware_ink    ;; Set pen to ink.
            INC HL                          ;; Go to next firmware ink.
        DEC D : JP NZ,pen_loop              ;;

        LD C,#10                            ;; Finally select border (pen 16).
        LD A,(HL)                           ;; Read final firmware ink.
        ;;                                  ;; Fall through to subroutine without CALL.

    set_pen_to_firmware_ink:

        ;; A - firmware ink
        ;; C - pen

        PUSH HL                          ;;
            ADD #85 : LD L,A             ;; Calculate low byte of address (+#85).
            ADC #DE : SUB L : LD H,A     ;; Calculate high byte of address (+#DE).
            LD A,(HL)                    ;; Read hardware ink.
        POP HL                           ;;
        AND #1F : OR #40                 ;; Mask ink and add command bit for Gate Array.
        LD B,#7F : OUT (C),C : OUT (C),A ;; Set ink.
        ;;                               ;;
        RET                              ;; Jump back to pen loop or to caller of 'set_table_of_firmware_inks'.

[Jean-Marie]

Thank you for the comments.
In the AFC3 loops,  I think we can get rid of the following instructions:
EX AF,AF'                          ;; Restore A.
DEC A : JP NZ,x_loop

and use the register B' as a loop counter. The BC' register is overwritten just after, as we can see.
So the code would become :

Code Select Expand

ORG &AFC3
y_loop:
LD A,C
EXX
LD B,A                     ;;B'=x_loop counter
x_loop:
LD A,(DE) : INC DE         ;; Load sprite data and update source pointer.
OR A : JR Z,$+3            ;; Is byte opaque?
LD (HL),A                  ;; Write byte.
INC HL                     ;; Update target pointer.
DJNZ x_loop
EX DE,HL                   ;; Swap registers to allow DE=DE+BC.
LD BC,#0000 :source_skip   ;; Skip bytes in source data.
ADD HL,BC
EX DE,HL
LD BC,#07FA :target_skip   ;; Go to next row and skip bytes on screen.
ADD HL,BC
BIT 6,H                    ;; Was the screen memory left?
jump_condition: JP Z,$+7
LD BC,#C040 : ADD HL,BC    ;; Then correct target pointer.
EXX
DJNZ y_loop
I have not tested it yet.

[Jean-Marie]

Consequently, we can also replace : LD BC,#0000 :source_skip                  ;; Skip bytes in source data.
with
LD C,0 :source_skip
since the B' register will always be 0 after the loop, and the C' value will never be greater than 6.

Code Select Expand

ORG &AFC3
y_loop:
LD A,C
EXX
LD B,A                     ;;B'=x_loop counter
x_loop:
LD A,(DE) : INC DE         ;; Load sprite data and update source pointer.
OR A : JR Z,$+3            ;; Is byte opaque?
LD (HL),A                  ;; Write byte.
INC HL                     ;; Update target pointer.
DJNZ x_loop
EX DE,HL                   ;; Swap registers to allow DE=DE+BC.
LD C,0 :source_skip        ;; Skip bytes in source data.
ADD HL,BC
EX DE,HL
LD BC,#07FA :target_skip   ;; Go to next row and skip bytes on screen.
ADD HL,BC
BIT 6,H                    ;; Was the screen memory left?
jump_condition: JP Z,$+7
LD BC,#C040 : ADD HL,BC    ;; Then correct target pointer.
EXX
DJNZ y_loop