No problem, these tools can be published as they were before the link on my website went down.
Enjoy!
Decoding cdi image files
- Shikotei
- Burn:Cycle Activated
- Posts: 42
- Joined: Mon Mar 01, 2010 5:01 pm
- Location: Netherlands
- Contact:
Re: Decoding cdi image files
Recently I've been re-doing some decoding attempts at The Apprentice and only getting the CLUT encoded images out (usually 128 colors).
I'm stumped at the compiled sprites (I assume they are) and deviated to different discs to see if I can build from something that's a little less garbled than the sprites on the Apprentice.
In comes Zelda's Adventure.
Most big images are also stored in a CLUT with a 256 palette.
But all sprites are stored differently, and after some tryouts, got the idea that it's pixel shifted.
One of the results is below.
I've also found the map data (overworld and shrines), and know they are stored using CLUT, but I cannot find the correct palette. Does anyone know where to look?
-- edit 30/08/'17 --
Re-uploaded Ganon (moving the forum removed attachments)
I'm stumped at the compiled sprites (I assume they are) and deviated to different discs to see if I can build from something that's a little less garbled than the sprites on the Apprentice.
In comes Zelda's Adventure.
Most big images are also stored in a CLUT with a 256 palette.
But all sprites are stored differently, and after some tryouts, got the idea that it's pixel shifted.
One of the results is below.
I've also found the map data (overworld and shrines), and know they are stored using CLUT, but I cannot find the correct palette. Does anyone know where to look?
-- edit 30/08/'17 --
Re-uploaded Ganon (moving the forum removed attachments)
Last edited by Shikotei on Wed Aug 30, 2017 9:39 am, edited 1 time in total.
- Shikotei
- Burn:Cycle Activated
- Posts: 42
- Joined: Mon Mar 01, 2010 5:01 pm
- Location: Netherlands
- Contact:
Re: Decoding cdi image files
As an example of what sort of map data I get:
![Image](http://www.shikotei.com/tmp/Fire%20Shrine%20region.png)
It's a raw conversion (data = 0x00 to 0xFF) from hex to RGB (R=G=B) (0xFF = RGB[255,255,255]; 0x00 = RGB[0,0,0]).
The above image is a compilation of 3 images horizontal by 2 vertical. The region on the map is the Shrine of Fire as indicated by the entrance center top and lava flow on the right (both are kind of black).
The images are 384x240, suggesting it is CLUT stored and not YUV (or any of its variations) or raw RGB.
I found this information in the 'over.rtf' file.
As far as I can tell, each area (single screen) has several blocks of information:
Firstly, there's a block of map data, (suspected CLUT stored).
This block is interrupted by blobs of (audio) data that are 0x0914 bytes long and repeats after 0x3F8C bytes of map data.
So there's 0x0914 bytes of audio, then 0x3F8C bytes of data, then 0x0914 bytes of audio, etc.
The map data block is always exactly 0x019E78 bytes long (including the audio).
Should the audio data be removed, there's enough bytes to fill a 384x240 pixel image at 1 byte per pixel.
This is my basis for assuming CLUT encoding.
This block is followed by several other blocks of NULL data, sometimes interrupted at the same frequency as the first block.
At some point there's readable data that reads something like "play;.voice.info..tree..cycle", shortly followed by its area type "plain_stream".
This is followed by a long block of audio data, interrupted by audio data.. I know it's weird, but that is what it looks like.
After another block of data with an unknown purpose, there's more NULL data, followed by sprite data.
This data is pixel shifted (using ASM).
The data contains the sprites that can be found in this area. This (so far) includes enemies, treasures, NPCs and purchasable items (including price).
Then there's more NULL data and a block of (I think) script triggers and other behavioral data. This is backed by (for example in the area where you have to play the flute to a snake to get a firewall spell) "APPEAR..PLAYFLUTE."
This is once again followed by more NULL data. Then you find the CLUT itself. This thing is preceded by 0x000100 and usually starts with 0x323232 0x646464 0x969696 0xC8C8C8 0x000000 0x101010.
Then there's NULL data, something about 'offsets' and then continues with audio data interrupted audio for a short while. After this there's essentially NULL data interrupted by audio.
This concludes one 'block' of map data (the format now repeats).
With the above information, I have been able to extract more than 70 screens of map data and crudely assembled some of it. As this data is extremely raw (no color) and only is noisy grey-scale, it is difficult to determine which piece goes where.
For those who want to try to help, here's the required data:
Full block: http://www.shikotei.com/tmp/FullBlockData.dat
Map data: http://www.shikotei.com/tmp/MapData.dat
Map processed: http://www.shikotei.com/tmp/Map.png
Sprite data: http://www.shikotei.com/tmp/SpriteData.dat
Processed sprite data (CLUT encoded): http://www.shikotei.com/tmp/SpriteDataProcessed.dat
Decoded sprite data: http://www.shikotei.com/tmp/Sprite.png
CLUT data: http://www.shikotei.com/tmp/CLUTData.dat
CLUT palette: http://www.shikotei.com/tmp/CLUTPalette.png
--Edit 13/12/'16--
The above files are no longer on my site; I've succeeded in decoding them. See post below.
![Image](http://www.shikotei.com/tmp/Fire%20Shrine%20region.png)
It's a raw conversion (data = 0x00 to 0xFF) from hex to RGB (R=G=B) (0xFF = RGB[255,255,255]; 0x00 = RGB[0,0,0]).
The above image is a compilation of 3 images horizontal by 2 vertical. The region on the map is the Shrine of Fire as indicated by the entrance center top and lava flow on the right (both are kind of black).
The images are 384x240, suggesting it is CLUT stored and not YUV (or any of its variations) or raw RGB.
I found this information in the 'over.rtf' file.
As far as I can tell, each area (single screen) has several blocks of information:
Firstly, there's a block of map data, (suspected CLUT stored).
This block is interrupted by blobs of (audio) data that are 0x0914 bytes long and repeats after 0x3F8C bytes of map data.
So there's 0x0914 bytes of audio, then 0x3F8C bytes of data, then 0x0914 bytes of audio, etc.
The map data block is always exactly 0x019E78 bytes long (including the audio).
Should the audio data be removed, there's enough bytes to fill a 384x240 pixel image at 1 byte per pixel.
This is my basis for assuming CLUT encoding.
This block is followed by several other blocks of NULL data, sometimes interrupted at the same frequency as the first block.
At some point there's readable data that reads something like "play;.voice.info..tree..cycle", shortly followed by its area type "plain_stream".
This is followed by a long block of audio data, interrupted by audio data.. I know it's weird, but that is what it looks like.
After another block of data with an unknown purpose, there's more NULL data, followed by sprite data.
This data is pixel shifted (using ASM).
The data contains the sprites that can be found in this area. This (so far) includes enemies, treasures, NPCs and purchasable items (including price).
Then there's more NULL data and a block of (I think) script triggers and other behavioral data. This is backed by (for example in the area where you have to play the flute to a snake to get a firewall spell) "APPEAR..PLAYFLUTE."
This is once again followed by more NULL data. Then you find the CLUT itself. This thing is preceded by 0x000100 and usually starts with 0x323232 0x646464 0x969696 0xC8C8C8 0x000000 0x101010.
Then there's NULL data, something about 'offsets' and then continues with audio data interrupted audio for a short while. After this there's essentially NULL data interrupted by audio.
This concludes one 'block' of map data (the format now repeats).
With the above information, I have been able to extract more than 70 screens of map data and crudely assembled some of it. As this data is extremely raw (no color) and only is noisy grey-scale, it is difficult to determine which piece goes where.
For those who want to try to help, here's the required data:
Full block: http://www.shikotei.com/tmp/FullBlockData.dat
Map data: http://www.shikotei.com/tmp/MapData.dat
Map processed: http://www.shikotei.com/tmp/Map.png
Sprite data: http://www.shikotei.com/tmp/SpriteData.dat
Processed sprite data (CLUT encoded): http://www.shikotei.com/tmp/SpriteDataProcessed.dat
Decoded sprite data: http://www.shikotei.com/tmp/Sprite.png
CLUT data: http://www.shikotei.com/tmp/CLUTData.dat
CLUT palette: http://www.shikotei.com/tmp/CLUTPalette.png
--Edit 13/12/'16--
The above files are no longer on my site; I've succeeded in decoding them. See post below.
Last edited by Shikotei on Tue Dec 13, 2016 12:28 pm, edited 1 time in total.
- Shikotei
- Burn:Cycle Activated
- Posts: 42
- Joined: Mon Mar 01, 2010 5:01 pm
- Location: Netherlands
- Contact:
Re: Decoding cdi image files
You'll have to forgive the multi-posting, there's simply no replies in-between..
I've made progress decoding the garbled data.
It was encoded in dYUV. This format also has an encoding size of 1 byte per pixel (actually 2 bytes per 2 pixels).
Long story short: this could not have been decoded without the Greenbook. The deltas were quantified using a custom function.
I've decoded many images (pieces of the map) using a working decoder (again home-brew in PHP).
Trouble is though, that I have yet to find the initial YUV values.
For some images they are assumed to be neutral grey (in this case YUV[128,128,128]).
But for images that are in a sequence (say two horizontally fitting pieces), the second piece has different initial values for Y, U and V.
Efforts so far have not yielded working results.
To show an example: the following two images are horizontally sequenced, but only the first image is properly decoded.
![Image](http://shikotei.com/tmp/Map%20Piece%20-%201.png)
The left image, properly decoded
![Image](http://shikotei.com/tmp/Map%20Piece%20-%202.png)
The right image, missing its initial YUV values, they are assumed [128,128,128]
One thing is noticeable though, and that is that the COLOR seems to be correct. This would make it seem like U and V values are always initialized at 128!
The search continues...
--Edit 13/12/'16--
Because this is going to be a small update, I've refrained from adding yet another post.
I've searched for the assumed initial YUV values that fitted the first image (the one I correctly decoded) and found them.
I searched for 0x808080 [128,128,128] repeated a few dozen times.
A datablob of exactly 720 bytes long is present twice closely following the map data itself (there's one blob in between them of unknown purpose, perhaps 'here you can walk'-data?).
Why 720 bytes? Each map is 240 lines by 384 columns. The linefeed (initial values) consists of Y, U, and V information. One byte each.
So that makes 3 bytes per line, for 240 lines -> 720 bytes.
Using this last piece of missing information, all map data was decoded. The overworld map file (over.rtf) contained 305 map pieces.
All but a dozen or so are part of the overworld. There's one additional map for each first area per shrine and one for each pre-boss area.
There's a few map pieces that are used to show the indoor areas (inside the houses of Skotness for example), but also the caves near the beach.
World map
Assembling the map was no easy task; many pieces were scattered in groups and some weren't as logical as they first seemed.
A tiny world map is stored in zelda.rtf in a CLUT like manner: Each pixel represents a map piece/area. And I suspect that the color differences are caused (not only by the CLUT I used) by the difference in background music.
I suspect that the underworld map file (under.rtf) contains the map data for all the shrines.
-- edit 30/08/'17 --
Reuploaded world map (moving forum removed attachments)
I've made progress decoding the garbled data.
It was encoded in dYUV. This format also has an encoding size of 1 byte per pixel (actually 2 bytes per 2 pixels).
Long story short: this could not have been decoded without the Greenbook. The deltas were quantified using a custom function.
I've decoded many images (pieces of the map) using a working decoder (again home-brew in PHP).
Trouble is though, that I have yet to find the initial YUV values.
For some images they are assumed to be neutral grey (in this case YUV[128,128,128]).
But for images that are in a sequence (say two horizontally fitting pieces), the second piece has different initial values for Y, U and V.
Efforts so far have not yielded working results.
To show an example: the following two images are horizontally sequenced, but only the first image is properly decoded.
![Image](http://shikotei.com/tmp/Map%20Piece%20-%201.png)
The left image, properly decoded
![Image](http://shikotei.com/tmp/Map%20Piece%20-%202.png)
The right image, missing its initial YUV values, they are assumed [128,128,128]
One thing is noticeable though, and that is that the COLOR seems to be correct. This would make it seem like U and V values are always initialized at 128!
The search continues...
--Edit 13/12/'16--
Because this is going to be a small update, I've refrained from adding yet another post.
I've searched for the assumed initial YUV values that fitted the first image (the one I correctly decoded) and found them.
I searched for 0x808080 [128,128,128] repeated a few dozen times.
A datablob of exactly 720 bytes long is present twice closely following the map data itself (there's one blob in between them of unknown purpose, perhaps 'here you can walk'-data?).
Why 720 bytes? Each map is 240 lines by 384 columns. The linefeed (initial values) consists of Y, U, and V information. One byte each.
So that makes 3 bytes per line, for 240 lines -> 720 bytes.
Using this last piece of missing information, all map data was decoded. The overworld map file (over.rtf) contained 305 map pieces.
All but a dozen or so are part of the overworld. There's one additional map for each first area per shrine and one for each pre-boss area.
There's a few map pieces that are used to show the indoor areas (inside the houses of Skotness for example), but also the caves near the beach.
World map
Assembling the map was no easy task; many pieces were scattered in groups and some weren't as logical as they first seemed.
A tiny world map is stored in zelda.rtf in a CLUT like manner: Each pixel represents a map piece/area. And I suspect that the color differences are caused (not only by the CLUT I used) by the difference in background music.
I suspect that the underworld map file (under.rtf) contains the map data for all the shrines.
-- edit 30/08/'17 --
Reuploaded world map (moving forum removed attachments)