"Tokenizer" for A-lot-of-Text-games ..

[SRS]

Transfering BASIC-Adeventures from other platforms often leads to the "low memory" problem on CPC in Basic.

Sometime I do a manual "tokenizing" if I see some big chunk of repeated text in several lines, i.e." You are watching a" I'll reduce to some string ... or "token"

Do we have some software out there that scans let'S say 60 room desriptions and so on for "common text parts", (above 3 ore four chars) and gives a table of replacement ? Like "you do" becomes "T01" ... so we could compress / recylce parst of text "On the fly" maybe with an RSx and using second RAM bank for adventures ?

Hope you get my point ...

Example :

Code: [Select]

"You are standing at the bar inside the tavern. There ..."

gets

Aat the bar inside the tavern. There ..."
With decoding

Code: [Select]

IF LEFT$(hoel$,1)="A" THEN PRINT"You are standing ";:GOTO 998

[Prodatron]

Better use something like
#A
instead of just A
so you can replace phrases anywhere in the sentences. With the help of INSTR and MID it's easy to write a string replace function.
Regarding "common text parts" it's nothing else than using a compression algorithm which is based on a dictionary.

[Docent]

Transfering BASIC-Adeventures from other platforms often leads to the "low memory" problem on CPC in Basic.

Sometime I do a manual "tokenizing" if I see some big chunk of repeated text in several lines, i.e." You are watching a" I'll reduce to some string ... or "token"

Do we have some software out there that scans let'S say 60 room desriptions and so on for "common text parts", (above 3 ore four chars) and gives a table of replacement ? Like "you do" becomes "T01" ... so we could compress / recylce parst of text "On the fly" maybe with an RSx and using second RAM bank for adventures ?

Hope you get my point ...

Example :

Code: [Select]

"You are standing at the bar inside the tavern. There ..."

gets

Aat the bar inside the tavern. There ..."
With decoding

Code: [Select]

IF LEFT$(hoel$,1)="A" THEN PRINT"You are standing ";:GOTO 998

Hey, you've just invented LZW compression algorithm

[SRS]

IF it would be the SRS algorithm

But did not see something like this for single PRINT commands until nowadays.