Hex: Tutorial on mapfiles: Installment #1

This is where the Admins will put tutorials submitted by users.
Post Reply
Grenadiac




Socialist Golden Age Magic Era Eureka
Tsunami Scorched Earth Articulatist 250

Posts: 390
Joined: Fri Sep 05, 2003 1:36 pm
Location: Tucson, AZ USA
Contact:

Hex: Tutorial on mapfiles: Installment #1

Post by Grenadiac »

I've started to work on a detailed discussion on the mapfile format. I will try to release a new section when I get time to write it. For now, enjoy this--
---------------------------------------------------
An explanation of Halo Map Files.

Ok, this tutorial is LONG overdue. In this tutorial, I intend to explain how the halo mapfile format works. I will try to point out the differences between Halo Xbox and HaloPC where I can. My target audience is advanced modders, but intermediate modders will get some good info out of it as well. After reading this, you should get a general idea of how the mapfile works, and a better understanding of why some things are corrupt, or why your rebuild doesn't work. Ok, so on with the info dump...

---------------------------------------------------
Table of Contents

MAPFILE ORGANIZATION
-Header
-Tag Index Header
-Scenario
-Supporting Tags
-Raw Model Data
-Raw Bitmap Data
-Raw Sound Data
-BSP(s)

THE MAPFILE HEADER
-what it does
-xbox and compression, fixed length
-header format

THE TAG INDEX HEADER
-model raw data offsets
-magic
-tag count
-base tag
-tag ids

THE SCENARIO
-header and reflexives
-scenario defines everything in the GAME, everything else is supporting
-scenario spawns
-high level tags
-scenario reference pools
-bsp data

SUPPORTING TAGS
-metadata...what it really is
-major tag types
-recursive types
-tag hierarchy
-raw data

MODEL RAW DATA
-model tags
-how the model raw data works
-submeshes and submesh headers
-LODs and variations
-differences in offsets for xbox and pc
-raw data corruption with model injection and map rebuilding

BSP
-what is a BSP
-how the bsp is found
-most difficult part of reverse engineering halo
-bsp tags having a zero offset
-collision bsp and visible meshes
---------------------------------------------------

DIFFERENCES BETWEEN HALO XBOX and HALOPC
There a great number of differences between xbox and pc maps. Of course, the version number in the header is 5 for xbox, and 7 for PC. People often say that xbox maps are fixed length, so you can't really add anything to them. This is partially correct, but mostly wrong. It is true that cache files are all the exact same size. However, all of the mapfiles in their "cache" format use garbage padding from the end of the useful halo data to the end of the cache file. When halo runs, it decompresses the map files on the original game disk onto the harddrive. The data here is usually several megabytes shorter than the cachefile length, so the rest of that "slack space" from the end of the useful data to the end of the actual cache file is garbage. The length of the decompressed file is defined by the mapfile header.

Another major difference between the PC and Xbox versions is a lot of the floating point data on the xbox version is in an encoded/compressed format. This is to save disk space and memory on the console, which is much more restrictive than on a normal PC capable of running halo. Since the PC has much more free memory, it is more efficient to use up more disk space and avoid converting all those compressed floats (this saves CPU cycles by avoiding the conversion altogether). In fact, this is the major difference between PC models and xbox models. It would probably be possible to someday convert PC models to xbox.

The biggest difference between PC and xbox is that xbox cache files contain ALL of the necessary resource to run a map. On PC, the raw data for sound and bitmaps are moved into separate .map files to save on disk space. This is the major reason that rebuilding with HMT works on PC, but not on xbox. The code to handle the raw data is not perfect, and it is still not perfectly understood.

Next Topic: MAPFILE ORGANIZATION

-Grenadiac
Cool Cow





Posts: 406
Joined: Sun Jan 18, 2004 4:34 pm

Post by Cool Cow »

Wow thats really awsome it will come in handy once I figure out how to work HMT man am I a newb or what lol *thinks milkshape is some type of shake*
Xhzjang





Posts: 46
Joined: Fri Feb 13, 2004 12:06 pm

Post by Xhzjang »

Cool, thanks alot man! In the mean time could i just ask how you get the offset of the meta data from the tags in the map?
User avatar
HunterXI





Posts: 3927
Joined: Sat Nov 22, 2003 11:21 am
Location: Azeroth
Contact:

Post by HunterXI »

*applauds Gren*... now I can just link to this in my sig as proof that most noobs are too stupid to understand...
This post printed on 100% recycled electrons.
kaptainkommie




Wordewatician 500

Posts: 732
Joined: Wed Nov 26, 2003 11:59 pm
Location: Raleigh, NC, USA

Post by kaptainkommie »

HunterXI wrote:*applauds Gren*... now I can just link to this in my sig as proof that most noobs are too stupid to understand...
That doesn't make any sense first of all. Second of all, its completely uneeded. Lately most of all your posts have ATLEAST 1 diss aimed toward newbs in general if not a specific person. Can't you find anything better to do with your time than pretend to be in some leet Anti-Newb organization and flame whenever you can. When you're not flaming newbs, or spamming to increase your post count, you're asking to be a moderator and complaining about how we aren't doing are job well enough.

If you can't provide positive feedback to forum users, I once AGAIN request that you DO NOT REPLY AT ALL.

Good job Gren, I was wondering when this was coming, perhaps after you're completely finished with your tutorials, someone will take your information and set up a page (similiar to pfhorslayer's page) that outlines the details of the .MAP specification.
User avatar
HunterXI





Posts: 3927
Joined: Sat Nov 22, 2003 11:21 am
Location: Azeroth
Contact:

Post by HunterXI »

kaptainkommie wrote:
HunterXI wrote:*applauds Gren*... now I can just link to this in my sig as proof that most noobs are too stupid to understand...
That doesn't make any sense first of all. Second of all, its completely uneeded. Lately most of all your posts have ATLEAST 1 diss aimed toward newbs in general if not a specific person. Can't you find anything better to do with your time than pretend to be in some leet Anti-Newb organization and flame whenever you can. When you're not flaming newbs, or spamming to increase your post count, you're asking to be a moderator and complaining about how we aren't doing are job well enough.

If you can't provide positive feedback to forum users, I once AGAIN request that you DO NOT REPLY AT ALL.

Good job Gren, I was wondering when this was coming, perhaps after you're completely finished with your tutorials, someone will take your information and set up a page (similiar to pfhorslayer's page) that outlines the details of the .MAP specification.
yeah, but you haven't seen my PM box... OMFG was it bad...

and I didnt mean to spam in any way, I still agree that this is very good, and taught me several thing too :)

and Kaptain, wtf are you talking about mods... Ive never asked to be a mod...

and besides, you're asking me not to post if I dont have "anything nice to say"... you just flamed me, dumbass...
This post printed on 100% recycled electrons.
User avatar
MasterNeoChief




Conceptionist

Posts: 2431
Joined: Tue Dec 23, 2003 9:34 am
Location: The Netherlands

Post by MasterNeoChief »

Make this a sticky, cos it already was halfway down the page...
CovenantSpy





Posts: 270
Joined: Sat Jan 24, 2004 7:00 pm

Post by CovenantSpy »

I was reading some of the rules and was wondering if anyone finds my avatar annoying. If a get a lot of bad reasponses I will take it of and put up a new one.
User avatar
MasterNeoChief




Conceptionist

Posts: 2431
Joined: Tue Dec 23, 2003 9:34 am
Location: The Netherlands

Post by MasterNeoChief »

I find it funny in a way.
Solra Bizna




Articulatist 250

Posts: 396
Joined: Sun Feb 22, 2004 4:07 pm
Location: Mu Geminorum VII

Post by Solra Bizna »

kaptainkommie wrote:...someone will take your information and set up a page (similiar to pfhorslayer's page) that outlines the details of the .MAP specification.
Funny, that was exactly what I was planning to do after I finished HHK (server down).
-:sigma.SB
Talin64




Socialist

Posts: 213
Joined: Mon Jan 26, 2004 12:24 pm

Post by Talin64 »

I just wanted a little clarification on the Xbox map rebuilding problem. Corrrect me if I am wrong but you are saying xbox map rebuilding is impossible because the snd header values are not all known and so we don't know how to update them. This also goes with the bitm header but more is known about that header. This makes me think that hmt doesn't try to extract xbox snd meta or bitm meta for rebuilding. Making it impossible to rebuild the map anyways.
So once we figure out the complete header structure we should be able to map rebuild for xbox or at least get a little closer.
Spar13





Posts: 200
Joined: Thu Dec 25, 2003 2:34 pm
Contact:

Post by Spar13 »

this is a good tut
thkz for making it cya Gren
Image
xorange




Articulatist 100

Posts: 134
Joined: Mon Dec 08, 2003 12:46 pm
Location: Santa Rosa, CA

Post by xorange »

Wow! I can't wait for the rest!
Thanks Gren! :D
Post Reply