Author Archives: Larval

Solar System Simulation

This project was for our implementation of physics in a real-time application. I attempted to create a realistic simulation of our solar system using a custom implementation of the laws of motion (i.e. not using Unreal’s AddForce()) and a few numerical integration methods to try and keep the simulation stable. The simulation also makes use of Kepler’s orbital elements and the real world values of each of our planet’s orbits to make the simulation accurate. It is not a true N-Body simulation – gravity is only calculated between the Sun and the planets.

Sol, Mercury, Venus, Earth and Mars

The simulation includes the 8 major planets of our solar system (sorry Pluto, you don’t have a good texture I can download anywhere), all of which orbit at (close to) their correct orbital paths, velocities, and periods. The motion of each planet is affected by the gravitational pull of the sun. In reality, each planet would also be affected by the gravity of every other planet, but I omitted this from my simulation because the exponential increase in computation didn’t seem worth it for a minor effect.

You can fly around the solar system using Unreal’s standard fly cam controls, and you can adjust the camera speed using the scroll wheel. Clicking on a planet will reveal several planetary and orbital properties, including the object’s mass and radius, as well as it’s major and minor orbital axes, the current distance from the primary, etc.

Jupiter and its properties. All values excluding mass, radius, and tilt are calculated, not predefined, and align pretty closely to the planet’s real-life orbital values defined here.

It’s not possible – or at least, very impractical – for the simulation to be perfectly accurate due to the astronomical magnitudes of the numbers involved, such as the mass of each planet, the gravitational constant (a very tiny number), and the distances of our solar system – 32-bit floats just can’t deal with the difference in magnitude very well. 64-bit doubles might, but flying for literally hours at light speed within the simulation to reach Neptune would be a poor user experience. Therefore, I’ve had to scale the solar system down to sensible values (planet masses range up to the thousands, not octillions, of kilograms, for example) and increase the strength of gravity so that it’s still felt with such (relatively) small masses. Nevertheless, the calculations hold up, and the solar system behaves like it really does – if you really wanted, you could sit and wait a full year for Earth to complete its orbit at 1x simulation speed. Speaking of which, the time slider at the bottom lets you speed up the simulation, so you can watch the planets zip around the sun up to 10,000,000x faster than they normally would (though beyond 1,000,000x does damage the simulation – more on that later).

The drop-down menu allows you to change between 3 integration methods, which essentially changes how the simulation is calculated (again – more on this later). It’s interesting to see how each method affects the simulation. The reset button restores the simulation to its initial state, placing all the planets back in their default positions.

Earth is the only planet in the simulation with a separate scrolling layer for its clouds.

The 8 planets use textures downloaded from here, and Earth actually uses a separate scrolling cloud layer. The appearance of the sun and stars is done entirely procedurally with Unreal’s materials. The sun’s material is composed of two scrolling gaussian noise textures layered on top of each other, one with a large scale and the other a small scale. This gives the surface those big variations in colour. The self-illumination is also cranked way up to give the sun its bright glow, and on particularly bright spots, the glows really shine and do a pretty decent job of emulating solar flares. The stars work by generating UV cells and choosing a random spot within the cell to become a star, feeding the distance of the random spot from the centre of the cell into its own scale, but really I just followed this tutorial.

The light that each planet receives is also done using a basic lambert lighting material. Unfortunately, placing a point light where the sun is and cranking up its brightness and falloff radius wasn’t good enough – over these large distances, the point light is simply too weak, and cranking up the camera exposure helps to light up the further planets at the cost of making the inner planets blinding (I suppose this is pretty realistic, but again, not a great user experience).

Finally, each planet leaves an orbital trail so they can easily be located and identified at a distance. The trailing affect is a basic ribbon particle which uses a solid gradient material. The size of the ribbon is calculated using a logarithmic function and the distance between each planet and the camera. The logarithmic function basically means the size of the trail doesn’t grow much bigger the further you get away, but when you’re very close, shrinks to almost nothing. This is perfect for this situation – when the camera is far away, the trails all look pretty similar to each other, and when you approach one planet, the ribbon shrinks away and doesn’t obstruct the planet itself.

Deconstructing MechAssault – The MGF file

Like most games, MechAssault uses large resource archives to store all of the assets used for a certain level in the game. Generally speaking, these are catalog files which simply act as containers for several other files. Exploring MechAssault’s root directory, there are plenty of large files with the .mgf extension and file names which match the names for each “level” in the game: multiplayer maps, campaign missions, and levels used for in-game cinematics. There are a few others too: text.mgf, movies.mgf, and common.mgf. These MGF files store mostly the same data between them, as most assets are re-used throughout all levels in the game, though there are still some unique assets per level.

In this blog post, I will discuss how I deconstructed the MGF file format and reveal all the information I have so far. Please note that I don’t know everything about the format, so some information may be incomplete. This is an ongoing project so I will inevitably learn more about the format over time.

Decompressing the files

Initially, most of the MGF files (excluding text.mgf and movies.mgf) are compressed, making it impossible to actually read or decipher anything using a hex editor. Compression essentially works by rearranging the data to pack it much more efficiently and reduce disk space, which ends up scrambling the data we’re interested in. Fortunately, these files are compressed with zlib, a commonly used and free compression library. Because it’s so common, there exist decompression tools, which is where this journey starts.

To decompress the files, there’s a great command-line tool made by Luigi Auriemma called offzip. Luigi has a lot of great tools for these type of reverse engineering projects – if you’re interested in reverse engineering games yourself (especially older ones), you’ll probably find something of use on his website. The following command + arguments works best for this tool: offzip.exe -a -1 filename.mgfwhere -a extracts all the decompressed data, and -1 ensures only one file is output. Using the following batch script is a handy way of decompressing all the files at once:

for %%i in (*.mgf) do offzip.exe -a -1 %%i decompressed\%%i

With the files decompressed, investigation can begin!

Investigating the MGF file

Breaking down the file structure is essentially a long, ongoing task of inspecting the file with a hex editor and looking for patterns, testing values, and overall a lot of trial and error. Without any documentation, this task can be very hard, comparable to finding a needle in a haystack. Throughout this blog post, I will note some useful tips for those who are interested in getting started with projects like this one.

Overview of the MGF file structure

First and foremost, MGF files are binary files, which means all the data is encoded in raw binary format, not human-friendly text. If they were text files, this job would be an awful lot easier, but text files are very inefficient for computers – binary data can just be dumped in to memory and the CPU understands it, but text needs to be parsed, which can take a lot more time. Some files stored within the archive are text files, but I will explore these in later posts.

Another point worth mentioning is that data is stored in little-endian format. MechAssault was an original Xbox game, meaning it ran on Xbox hardware – an Intel x86 CPU. Intel x86 architecture reads data in little-endian format, which means no byte swapping is necessary. Read more about byte endian-ness here.

MGF files are composed of 5 main chunks:

  • Header – Stores offsets and lengths of the following sections
  • File allocation table – Stores information about each file in the archive
  • Directory entry table – Describes each directory node in the archive, folder or file
  • Strings – Null-terminated strings for file and folder names
  • File data – Stores all file data for every file in the archive
Diagram of an MGF file’s structure

Header

Binary files often start with a small chunk known as a header (a chunk is just a block of data for a specific purpose) which provides information about locations of data in the file and how said data may be formatted. This is usually in the form of offsets (pointers to locations in the file) and sizes (length in bytes), which help the program reading the file find the data it needs. Because binary files are not human-friendly, commonly used formats should be accompanied with some documentation that explains how and where data is stored. For example, the Microsoft WAVE soundfile format or Paul Bourke’s data formats. Because MechAssault uses a proprietary game engine, there is no public documentation on the MGF file format.

MGF files are no different – they start with a header that is always 64 bytes long. Below is a table describing the MGF file header:

OffsetLength and data typeDescription
04 bytes, char*MGF file signature – always reads mgf (including space at the end).
41 byte, charVersion – always 02 for MechAssault 1 archives, always 04 for MechAssault 2 archives.
51 byte, charUnknown – always 01
62 bytes, char*Unknown – always reads “ZZ”
84 bytes, intPadding – always 0
124 bytes, unsigned intNumber of files in the archive
164 bytes, unsigned intLength of file entry chunk in bytes
204 bytes, unsigned intOffset of file entry chunk (always 64 because the file entry chunk starts immediately after the header)
244 bytes, unsigned intNumber of directories (files and folders) in the archive
284 bytes, unsigned intLength of directory relationship chunk in bytes
324 bytes, unsigned intOffset of directory relationship chunk
364 bytes, unsigned intLength of directory strings chunk
404 bytes, unsigned intOffset of directory strings chunk
4420 bytesPadding – all zeros

File allocation table

Immediately following the header begins a list of 32-byte structures which provide information about each file in the archive. The number of entries in the list is stored in the header at offset 12. Using the information provided by the header, it is simple to calculate how long each structure is by simply dividing the length of the file entry chunk (offset 16 in the header) by the number of files. There is also a noticeable pattern in the data itself which repeats every 32 bytes. File entries are only 32 bytes long for MechAssault 2 archives – MechAssault 1 archives are 28 bytes long and the fields are rearranged.

Below is a table describing the file entry structure in a MechAssault 2 archive:

OffsetLength and data typeDescription
04 bytes, unsigned intIndex of directory entry that links to this file
44 bytes, unsigned intHash of file data
84 bytes, unsigned intHash of full file path
124 bytes, unsigned intFile length
164 bytes, unsigned intFile length (again)
204 bytes, time_tUNIX timestamp – last modified date
244 bytes, unsigned intOffset to the file’s data
284 bytesUnknown, always 0x00F71200. Only in MA2 archives.

The most difficult piece of data to identify were the file hashes. Originally I thought this was just a 64 bit hash of the file data, but after disassembling the game code, I discovered that the latter 4 bytes were actually a custom hash of the file path.

Tips:

  • When looking for offsets, first of all make sure that the value is less than the total size of the file (seems obvious) and greater than the offset where the suspected value is stored (rarely are offsets going to point backwards in the file). Then, use your hex editors “go to” function to go to the offset. Inspect the data – is there a sudden change in the pattern? If there is, you’ve probably got an offset.
  • When looking for lengths stored near known offsets, simply add the suspected length value to the offset and use go to again – another change in the data’s pattern? You may have found the end of that block of data.
  • 4 byte integers are very common for simple data fields in binary files, even if the values stored in them never use all 4 of those bytes. This is probably because 4 byte integers fit snuggly in to most modern CPU registers. If you’re finding chunks of data scattered evenly with groups of 1, 2 or 3 0s in your hex editor, it’s probably a bunch of 4 byte integers.

Directory entry table

After the file entries, the next chunk of data is a list of structures that describe the hierarchical directory structure of the archive, including definitions of folders and files, as well as indexes which point to each directory’s parent. There are also offsets which point to strings stored in the following chunk, identifying the directory names. These structures are 24 bytes long, composed of 6 32-bit integers. I know, “directory relationship table” isn’t a great name, but I can’t think of anything else.

Below is a table describing an entry in the directory relationship table:

OffsetLength and data typeDescription
04 bytes, intUnknown
44 bytes, intParent index
84 bytes, intFirst child index
124 bytes, intNext sibling index
164 bytes, intOffset to file/folder name
204 bytes, intIndex of file entry in file table, if this entry is a file

This section remains the most mysterious part of the MGF file so far as I have yet to understand what some of the fields are. Earlier I mentioned there were gaps in the index fields of the file entries. It is here where these gaps are explained – the gaps occur because they are the indexes of folders, and the file entry table does not store folder entries. Each entry here stores the index of its parent. Because the parent index never refers to an index that can be found in the list of file entries, this is how I deduced that these were parent indices.

Strings

This section is very simple – it is just a large chunk full of null-terminated strings, referred to by the previous chunk. All strings in this section are the names of every file and folder in the archive. The first string is always “MGF ” and the second is always a backslash, which is the root directory of every archive.

File data

The final chunk of the MGF file format contains all of the actual file data for every file in the archive, pointed to by the file entries in the file entry table chunk. There are many types of files contained in the archives, identifiable with the extensions that can be found in the relevant file names. Plenty of them are plain text files, though there are still many binary files for assets such as textures, vertex buffers, level data, and so on.

Conclusion

With the MGF file format (mostly) deconstructed, it will now be much easier to inspect the individual files stored within them to better understand the game’s engine and how it uses these assets. In future blog posts, I will explore more of the files in depth and add the ability to preview the assets in MGF Explorer.

Generic Zombie Game

Generic Zombie Game is a simple top-down zombie survival shooter built with C++ and SFML. The gameplay consists of surviving against indefinitely spawning zombies with a choice of 3 weapons – a pistol, assault rifle and a knife. The game also features a simple day and night cycle, and you can press Q to equip the flashlight during night time. Be careful though, you can’t defend yourself with the flashlight!

This project is intended as an implementation of linear algebra used in 2D graphics.


Gameplay

The player moves around the world using the WASD keys and uses the mouse to both aim and fire. The player’s sprite also points towards the direction of the cursor with the following steps:

  • Convert the mouse cursor’s screen space position in to world space
  • Calculate the vector from the player’s world position to the cursor’s world position (cursorWorld – playerWorld)
  • Calculate the angle of this vector using arc-tangent and apply this angle as a rotation on the player sprite

The same vector is also used to position the camera, giving the appearance of the camera “following” the cursor.

Visuals

The game makes use of animated sprites for the player and zombies. These work by loading spritesheets (an image with all frames of the animation stored in one) alongside a text file which provides information on where each frame of the animation is placed and how large they are. These values are loaded in to an array and a sub-section of the associated sprite sheet is rendered to the screen depending on the current time and frame.

The day and night cycle – as well as the flashlight effect – works using SFML’s blend mode wrapper around OpenGL blending:

  • Draw a solid rectangle to a different render target that covers the entire screen
  • Set the alpha channel of the rectangle to some value based on sin(runtime * small_number) to simulate sun rise and set
  • Draw the flashlight sprite (which is just a white spot with transparency) over the rectangle using Additive blending
  • Convert the result to a texture and draw it over the gameplay screen with Multiply blending

Alien Swarm mod – Star Wars: Republic Commando player models

Port of delta squad player models from Star Wars: Republic Commando to Alien Swarm. For this, I used umodel to export the player models and textures from SW:RC, and the WallWorm 3DS Max plugin to export player models/skeletons from Alien Swarm. I then skinned the RC models to the Alien Swarm skeleton and recompiled the AS player models using WallWorm again. VTFEdit was used to convert the textures from SW:RC to Source engine textures.

You can visit the mod’s Steam workshop submission here: https://steamcommunity.com/sharedfiles/filedetails/?id=935685159

Relic Hunters Zero

Relic Hunters Zero Clone is, well, a clone of Relic Hunters Zero… kind of. It uses the same assets, at least. This was a university group project – my responsibility was to develop the player and weapons for the game, and also manage/guide the project’s development as the product owner. It is a complete game which features 4 distinct levels, multiple enemy types, multiple weapon types, pick-ups, lives, and even multiple screens – title screen, main menu, options, credits, and more!


The Player

The player can move, shoot, and even dash in any direction to escape any sticky situation. You may also equip up to 2 of any weapon in the game and swap them at any time with weapons found within supply crates in the game world. You have 100 health and 3 lives to get through all 4 levels of the game, and health can be restored with health packs dropped by enemies. I also implemented the collision response with the map, preventing the player from entering walls.

The Weapons

There are 3 types of weapon in the game: the Pistol, Rifle and Shotgun. Each weapon has its own unique firing behaviour properties such as damage, ammo, and sounds. Weapons have limited ammo which can be refilled by obtaining ammo pick-ups dropped by enemies throughout the game. Each weapon has a unique magazine size and amount of reserve ammo, and they can be reloaded at any time.

Phobos Settlement

Phobos Settlement is a 3D graphical application that serves to demonstrate various 3D mathematics and techniques. The scene takes place on the surface of Phobos – one of Mars’ two irregular moons – using a skybox generated with SpaceEngine and multiple NASA 3D models.


Procedural Animation

The scene is constructed using Hieroglyph’s hierarchical scene graph, animated using simple procedural animation. The satellite dishes in particular exploit the hierarchical properties to “point at” an object above – the dish is attached to the stand, which is attached to the base. The stand’s yaw rotations towards the object are inherited by the dish, which then performs an additional pitch to lock on to the target.

The entire scene is lit with a single rotating point light using the blinn-phong reflection model. The ground makes use of a normal map to create the illusion of depth on the craters.

The GUI

On the right hand side of the screen, there is a custom 2D GUI panel with buttons, sliders and a checkbox to adjust some camera and scene properties. The “Launch!” button starts a countdown sequence which will cause the shuttle to close its latches and take-off!

The timescale can be adjusted to slow down or completely stop all animation in the scene. The checkbox can be toggled to reveal additional scene information on the left side of the screen.

MechAssault MGF Explorer

MechAssault MGF Explorer is a GUI tool written in C++/wxWidgets which can open resource files used in MechAssault and MechAssault 2: Lone Wolf’s proprietary game engine. This is an ongoing project that will develop over time as I understand more about the file format and all of the resources within them. It has been developed with MechAssault 2 in mind, but can open archives used in the previous game as well.

This tool is heavily inspired by Adjutant, a program that can open .map files used in all the Halo games, index the files, and allow you to view textures, models, sounds, and even maps and animations. MGF Explorer can currently view textures, models, and plain text files in much the same way, though is (currently) far from the kind of feature level of Adjutant.

I also have a blog for this project, check it out!


MechAssault 2: Lone Wolf (and its predecessor) are original Xbox games published by Microsoft in the early 2000s. They were developed by Day 1 Studios, formerly known as Meyer/Glass Interactive and now owned by Wargaming.net.

The games were developed with Day 1’s proprietary engine of which there is no public/leaked information about (I still don’t even know if the engine has a name!). As such, this project has been a long-term exercise in reverse engineering/data mining to understand the engine’s inner workings and the assets used within the games.

This project has been an incredible learning experience as an aspiring game developer/software engineer; I have learned how to solve difficult problems involving undocumented raw binary data using hex editing tools, writing small CLI programs, and even learning how GUI applications are built and structured using the Model-View-Controller design pattern.

MechAssault stores its textures in a proprietary file format with the .tif extension (not the same as TIFF!). These files can store 2D textures, 3D textures, texture arrays and mip maps, and support BGRA8888, BGRA4444, BGRA5551, RGB565 and greyscale textures, as well as DXT1, 3 and 5 compression.

MGF Explorer can open and preview almost all texture files correctly – some are encoded using the morton Z-order curve and have yet to be decoded.

MGF Explorer can also preview 3D models used in the game. Models are stored in .mgmodel files, which are just simple XML documents that define the materials, meshes, and node hierarchy (using XML syntax) for the model.

After selecting an .mgmodel file from the file tree, an OpenGL viewport is revealed and the textured model will appear. This viewport also supports a flying camera, controllable by holding the right mouse button (over the viewport) and using WASD, space and ctrl. Additionally, a list of all the meshes that compose the model, along with the number of vertices, vertex stride and flags is shown. Selecting one of these meshes from the list will highlight it yellow in the viewport (helps with tracking down erroneous vertex/index buffers).