Convert a hex editor script into a BASH script

Chat about just about anything else

Convert a hex editor script into a BASH script

Postby xibalba on Thu Feb 02, 2012 9:17 pm

I need someone to convert a script for 010 editor (a hex editor, among other things) into a BASH script, because I can't get 010 editor, which is Windows/Mac, to work with Wine.

Here's the script.
Code: Select all
//--------------------------------------
//--- 010 Editor v3.2.2 Script File
//
// File:        Saga Frontier ARC
// Author:      Tauwasser
// Revision:    1.0
// Purpose:     Unpack Saga Frontier ARChive
//--------------------------------------

LittleEndian();

// Get filename
string fileName = GetFileNameW();

int id = GetFileNum();
int i;
int64 address = 0x00;

// Read Number of Pointers
uint NumPointers = ReadInt(address);

address += 0x04;

uint pointers[NumPointers];

// Read Pointers
for (i = 0; i < NumPointers; i++) {

    pointers[i] = ReadInt(address);
    address += 0x04;

}

// Split Files

uchar buf[0x40000];
int size;
string tmpFile;

for (i = 0; i < NumPointers - 1; i++) {

    size = pointers[i+1] - pointers[i];
    ReadBytes(buf, pointers[i], size);

    FileNew();
    SPrintf(tmpFile, "%s_file%u.bin", fileName, i);

    WriteBytes(buf, 0, size);

    FileSave(tmpFile);
    FileClose();
    FileSelect(id);

}


Here are Tauwasser's explanations, which I don't understand. Non-programmer here (I'm just a Global Mod), so, while I'm happy to have something technical to copy and paste for the programmers, don't bother to dumb it down for my benefit, if you feel like explaining what you do.
This is the internal file structure of *.ARC archives. I originally ruled out file 1 as a palette, because it starts off so linearly. However, they have color gradients for everything. So basically:

* File0: Sprite Sheet
* File1: Palette 15bpp BGR (555) LE
Maybe there is a transparency flag in there. Some entries have their highest bit set to 1 instead of zero.
* File2: ?
* File3: Big Sprite (shares palette file 1)
* File4: ?
* File5: ?
* File6: ?
* File7: ?
* File8: ?
* File9: ?

They're not compressed at all. The files are organized like this: [Number of Pointers] ((Number of Pointers) * [Pointer N]) [Data] ... [Data].
Each file starts where the pointer points to and ends where the next pointer points to. Therefore, there are (Number of Pointers -1) files in there. File 0 starts at [Pointer 0] and ends at [Pointer 1] and so on. The last pointer will usually point to just after the last byte of the file.

Files are zero-padded to 4 bytes. All data is Little Endian. All offsets are absolute within each ARC file. All files within the ARChives you presented to me were in chronological order, so (Pointer N) < (Pointer N + 1) for all N.


Here is a 7z compressed archive, which contains five of the *.ARC files, should you feel like testing your script.
http://www.verve-fanworks.com/SMF/index ... ttach=1008

If there's anything else you need, just ask.
Last edited by xenopeek on Sat Feb 04, 2012 10:58 am, edited 1 time in total.
Reason: Change quote block to code block, for script readability
xibalba
Level 2
Level 2
 
Posts: 90
Joined: Wed Jan 05, 2011 6:07 pm

Linux Mint is funded by ads and donations.
 

Re: Convert a hex editor script into a BASH script

Postby xenopeek on Sun Feb 05, 2012 5:46 am

Moved here by moderator

BASH isn't the handiest for this. Suggestion you learn some Python :wink: To run the below script, you need to install Python 3:
Code: Select all
sudo apt-get install python3

The script is rather rudimentary, but it works. Create a file called "unarc.py", put the below into it, then mark it executable (chmod +x unarc.py).
Code: Select all
#!/usr/bin/env python3

import array
import os
import struct
import sys

for filename in sys.argv[1::]:
   f_in = open(filename, mode='rb')
   numPointers = struct.unpack('<I', f_in.read(4))[0]
   print("{}, {:d} pointers:".format(filename, numPointers))
   pointers = []
   for i in range(numPointers):
      pointers.append(struct.unpack('<I', f_in.read(4))[0])
   for i in range(numPointers - 1):
      f_out = open("{}_file{:04d}.bin".format(filename, i), mode='wb')
      size = pointers[i + 1] - pointers[i]
      data = array.array('B')
      print("\t{}_file{:04d}.bin, from 0x{:08X}, length {:d} bytes".format(filename, i, pointers[i], size))
      f_in.seek(pointers[i], os.SEEK_SET)
      data.fromfile(f_in, size)
      data.tofile(f_out)
      f_out.close
   f_in.close

Example of running the script on the files you included:
Code: Select all
vincent@katya ~/Downloads/saga_frontier $ ./unarc.py *.ARC
M000.ARC, 10 pointers:
   M000.ARC_file0000.bin, from 0x00000030, length 32768 bytes
   M000.ARC_file0001.bin, from 0x00008030, length 512 bytes
   M000.ARC_file0002.bin, from 0x00008230, length 2796 bytes
   M000.ARC_file0003.bin, from 0x00008D1C, length 32768 bytes
   M000.ARC_file0004.bin, from 0x00010D1C, length 112 bytes
   M000.ARC_file0005.bin, from 0x00010D8C, length 464 bytes
   M000.ARC_file0006.bin, from 0x00010F5C, length 24 bytes
   M000.ARC_file0007.bin, from 0x00010F74, length 236 bytes
   M000.ARC_file0008.bin, from 0x00011060, length 64 bytes
M001.ARC, 10 pointers:
   M001.ARC_file0000.bin, from 0x00000030, length 32768 bytes
   M001.ARC_file0001.bin, from 0x00008030, length 512 bytes
   M001.ARC_file0002.bin, from 0x00008230, length 828 bytes
   M001.ARC_file0003.bin, from 0x0000856C, length 0 bytes
   M001.ARC_file0004.bin, from 0x0000856C, length 0 bytes
   M001.ARC_file0005.bin, from 0x0000856C, length 404 bytes
   M001.ARC_file0006.bin, from 0x00008700, length 24 bytes
   M001.ARC_file0007.bin, from 0x00008718, length 208 bytes
   M001.ARC_file0008.bin, from 0x000087E8, length 64 bytes
M002.ARC, 10 pointers:
   M002.ARC_file0000.bin, from 0x00000030, length 32768 bytes
   M002.ARC_file0001.bin, from 0x00008030, length 512 bytes
   M002.ARC_file0002.bin, from 0x00008230, length 2416 bytes
   M002.ARC_file0003.bin, from 0x00008BA0, length 32768 bytes
   M002.ARC_file0004.bin, from 0x00010BA0, length 156 bytes
   M002.ARC_file0005.bin, from 0x00010C3C, length 328 bytes
   M002.ARC_file0006.bin, from 0x00010D84, length 24 bytes
   M002.ARC_file0007.bin, from 0x00010D9C, length 236 bytes
   M002.ARC_file0008.bin, from 0x00010E88, length 64 bytes
M003.ARC, 10 pointers:
   M003.ARC_file0000.bin, from 0x00000030, length 32768 bytes
   M003.ARC_file0001.bin, from 0x00008030, length 512 bytes
   M003.ARC_file0002.bin, from 0x00008230, length 2424 bytes
   M003.ARC_file0003.bin, from 0x00008BA8, length 32768 bytes
   M003.ARC_file0004.bin, from 0x00010BA8, length 144 bytes
   M003.ARC_file0005.bin, from 0x00010C38, length 304 bytes
   M003.ARC_file0006.bin, from 0x00010D68, length 24 bytes
   M003.ARC_file0007.bin, from 0x00010D80, length 208 bytes
   M003.ARC_file0008.bin, from 0x00010E50, length 64 bytes
M004.ARC, 10 pointers:
   M004.ARC_file0000.bin, from 0x00000030, length 32768 bytes
   M004.ARC_file0001.bin, from 0x00008030, length 512 bytes
   M004.ARC_file0002.bin, from 0x00008230, length 1404 bytes
   M004.ARC_file0003.bin, from 0x000087AC, length 32768 bytes
   M004.ARC_file0004.bin, from 0x000107AC, length 100 bytes
   M004.ARC_file0005.bin, from 0x00010810, length 360 bytes
   M004.ARC_file0006.bin, from 0x00010978, length 24 bytes
   M004.ARC_file0007.bin, from 0x00010990, length 208 bytes
   M004.ARC_file0008.bin, from 0x00010A60, length 64 bytes
User avatar
xenopeek
Level 21
Level 21
 
Posts: 15443
Joined: Wed Jul 06, 2011 3:58 am
Location: The Netherlands

Re: Convert a hex editor script into a BASH script

Postby xibalba on Mon Feb 06, 2012 8:49 pm

http://board.byuu.org/viewtopic.php?f=10&t=2421&start=0

Sorry, I was feeling lazy and didn't mark this topic as resolved.

And, FYI, there are actually 10 files.
Code: Select all
1  File0: Sprite Sheet
2  File1: Palette 15bpp BGR (555) LE
3  File2: ?
4  File3: Big Sprite (shares palette file 1)
5  File4: ?
6  File5: ?
7  File6: ?
8  File7: ?
9  File8: ?
10 File9: ?


Here is funky***'s solution, in Python:
Code: Select all
from struct import unpack
import sys

#todo rewrite using the io module
datafile = open(sys.argv[1],"rb")
data = datafile.read()
start = 0
num_pointers = unpack("<I",data[start:start+4])
start +=4
pointer_fmt = "<%dI"%num_pointers[0]
pointers = unpack(pointer_fmt,data[start:start+num_pointers[0]*4])
start += num_pointers[0]*4

i = 0
while i < (num_pointers[0]-1):
    splitname = "%s_file%u.bin"%(datafile.name,i)
    splitfile = open(splitname,"wb")

    splitfile.write(data[pointers[i]:pointers[i+1]])
    splitfile.close()
    i+=1

splitname = "%s_file%u.bin"%(datafile.name,i)
splitfile = open(splitname,"wb")
splitfile.write(data[pointers[i]:])
splitfile.close()

Copy mine to a file called:
Code: Select all
saga_split.py

Then do:
Code: Select all
python saga_split.py file.arc
xibalba
Level 2
Level 2
 
Posts: 90
Joined: Wed Jan 05, 2011 6:07 pm


Return to Open chat

Who is online

Users browsing this forum: Google Adsense [Bot] and 4 guests