md4 v4 file spec 2006.01.17
     by gongo

     foreword
     header
     frame
     boneframe
     bonename
     surface
     vertex
     weight
     triangle
     bonerefs
     collapsemap
     limits

     the md4 file format is a skeletal version of id software's md3 mesh model format. as is in the gpl'd quake 3 1.32b source base, the md4 format isn't fully implemented. to further complicate things, up until now, there haven't been any publicly availble build tools for md4, so developing for it has been difficult at best.

     as the quest for skeletal animation in quake 3 began, i set out to learn as much as i could about the other file formats implemented in third-party games developed from the quake 3 engine. of particular interest to me was the wolfenstein series, namely return to castle wolfenstein's mds format and enemy territory's mdm/mdx formats. after many nights spent googling in vain for the mdm build tools, i finally stumbled upon the mds build tools... only to find that the skelout plugin was for 3dsmax r3 only...

     in a last ditch attempt to find a suitable quake 3 based skeletal model format, i googled for sdks for elite force, american mcgee's alice, and heavy metal fakk 2. i found the sdk for heavy metal fakk 2, and noticed that the format is more or less like the mdm/mdx, except it's skb/ska. while this system of having a separate base frame and animation file was intreging to me, i noticed that i'd have to do extensive changes to the md4 code already in place in the quake 3 source, which was something that i wasn't quite prepared to do yet... i have no previous experience with skeletal model formats, hence why i was looking into return to castle wolfenstein's mds format, just for simplicity's sake it would be better to work with just one file first. and once again, the skelout plugin was available for 3dsmax r2 and r3 only...

     frustrated, sleep deprived, and confused, my spirits were down and i was about to abandon the project altogether in favor of skm. i figured that since there's much more info about skm available on quakesrc, and it's still being used and developed, it would probably be the best alternative. as i deleted the fakk 2 tools directory, i noticed a folder named 'source'... i unzipped the sdk again, and took a look at the tools source folder, only to find the full source for skelout and max2skl, the skb/ska build tool. i started digging around in the max2skl source and found that it had some preliminary md4 exporting capabilities... from there, i compiled the skelout plugin with the 3dsmax 6 sdk and instantly had a working skl file exporter.

     as i posted on the quakesrc.org forums looking for more info about the mds, mdm/mdx, and skb/ska, someone suggested that i simply complete the md4 format and tailor it to my own needs. inspired, my heart took courage, and i set out to finish the md4 exporter and finalize the md4 implementation in the quake 3 source. the following is the result of roughly 2 weeks worth of labor, working on this during every spare moment i had.

     ident should match MD4_IDENT ("IDP4") and version should match MD4_VERSION (currently "4") in qfiles.h. the offsets will give you the position of the desired data within the file by in place conversion. for example, (md4BoneName_t *)((byte *)header + header->ofsBones) will give you the first md4BoneName_t in the array of md4BoneName_t numBones in length.

     bounds gives the frame's min and max for the bounding box. localOrigin is just bounds min + bounds max divided by 2. radius will give the distance to bounds min or max from localOrigin, used for sphere culling. bones is an array of md4BoneFrame_t numBones in length.

     matrix is the bone's axis and offset.

     parent gives the index to the parent bone in the main bones list. flags holds any of the bone flags MD4_BONE_FLAG_H (head), MD4_BONE_FLAG_U (upper), MD4_BONE_FLAG_L (lower), or MD4_BONE_FLAG_T (tag). bone flags are still currently under development, but future uses may include animating bone groups by flags or using flagged bones as tags.

     shader and shaderIndex give the name and index of this surface's shader in the main shader list. currently only one shader per surface is supported. lodBias is a placeholder for use in the renderer. the calculated lod is put in lodBias so that when the surface is passed to the renderer, the proper level of detail collapse map will be used. lodBias is 0 by default (full detail). minLod is the maximum level of collapsing that can be done before the mesh starts to substantially lose shape, or in other words, the lowest level of detail. ofsCollapseMap gives the position of the collapse map for this surface. it is an array of ints numVerts in length.

     vertex is this vertex's position, with normal giving the direction vector of the normal. texCoords gives the vertex position on the texture. weights is a list of md4Weight_t numWeights in length. each vertex can have no more than 8 influences or weights.

     boneIndex gives the index to the bone in the array of bone references for the surface, not the main bones list. boneWeight is any value between 0.01 and 1.0. the sum of all bone weights for any given vertex must equal 1.0. offset gives the direction vector of the weight's influence.

     indexes points to the vertex indices from the main vertex list that make up this triangle. indexes is altered for dynamic level of detail via collapse mapping with triangle->indexes[i] = collapsemap[triangle->indexes[i]] to get the index of the vertex this collapses to.

     boneRefs is an array of ints numBoneRefs in length. the values in the array are indices into the main bone list. boneRefs is used in order to know which bones need to be translated and interpolated to get the final frame position for the current surface in the model.

     collapseMap is an array of ints numVerts in length. the values stored in the array are indices into the vertex list for the current surface. the array lists the indices in collapsing order, such that index numVerts is the first to collapse, and index 0 is the last vertex and never collapses. every collapse removes 2 triangles along the edge uv where u is the collapsing vertex and v is the point u collapses to. this allows scaling of level of detail to a resolution of 1 vertex. ideally, one should not collapse the mesh down past the minLod in order to avoid excessive distortion of the mesh's overall shape. for example, to collapse a mesh down to it's minimum resolution, all vertices with index [minLod, numVerts) should be replaced with collapseMap[index] until the value is within [0, minLod), leaving minLod vertices to be rendered. in the process, some triangles will have all 3 vertices collapse together. these triangles need to be removed from the render list to avoid rendering one-dimensional triangles. generally speaking, texture mapping is retained by using the texCoords of the vertex collapsed to. the result is fairly faithful to the original full resolution texture mapping, with an acceptable amount of error (seams in texture mapping become more obvious). implementation in-game would entail scaling the level of detail from minLod at lowest to numVerts at highest, with the varying degrees of detail more or less spaced evenly between the two points.

     bones, frames, and surfaces maximums are per model. triangle and vertex maximums are per surface. note that these limits are similar to those of the md3 format for consistency's sake. i could set the limits higher, but given that the md4 spec technically maxes out at 8192 tris/surf * 32 surfs = 262144 tris, i don't think it's really necessary...