Reverse Engineering Earth 2150, Part 5: PAR Compiler
As of this evening, I now have a working PAR file compiler as well as a decompiler. Source files in the usual place.
Most of the work up to this point has been learning about the file structure. The same knowledge that lets me decompile the file will let me recompile it from the generated sources, so I was able to turn out the first prototype compiler in an evening.
That’s not to say it was plain sailing. Oh, no, computers never make things that easy.
In principle, the compiler should produce a file that is bit-for-bit identical to the original one that was decompiled. Mine doesn’t quite do that because of the whole parameters.csv debacle I mentioned in [part 4]({% post_url 2021-06-07-reverse-engineering-earth-2150-part-4 %}), so some of the compiled entities will be slightly out of order. But it will produce a PAR file that is the same size as the original, and it turns out the game doesn’t care about the exact order of the entities anyway.
Troubleshooting
As I’m sure you can imagine, I wasn’t terribly happy when the first version I produced was several kilobytes too small. This is a 331kB file, so a few k is going to be noticeable, and missing data is bad. I ran it back through the decompiler and all the entities I expected to be there were there, so it had to be something within the entity structure that was missing.
So out comes the hex editor once again. I open the original and my recompiled PAR file side-by-side in compare mode and ask it to find me the first difference. That turned out to be in the very first entity. You may [recall]({% post_url 2021-06-01-reverse-engineering-earth-2150 %}) that the entity structure includes, as its fourth element, the number of fields that entity has. You may further recall that builders have 81 fields. Well, the first entity was a builder, and it was only reporting that it had 67 fields. That’s 14 missing.
All the strings I expected seemed to be there, but as I checked them, I noticed something missing. Most strings in the original PAR file had an FFFFFFFF field after them, but I’d forgotten to add them back in after stripping them out of the CSVs. With that in mind, I recalled that only the mesh name in field 1 lacked this “padding”, so I had the compiler add it back in for every string after field 1 and re-ran it.
But of course it wasn’t going to be that simple. The compiled PAR file was now several hundred bytes too large. That meant that I’d added padding to a few dozen strings that shouldn’t have them. I got fifteen minutes into coding exceptions for when it should be added when I realized I was doing this the hard way, by trial and error.
Part of my decompiler necessarily has to classify entity types and put them in their own files. That means, in turn, that every CSV file contains entities with the same field structure. Well, except parameters.csv, which is a mixed bag. So, as part of its operation, I had it check which strings had a padding field after them, and make a note of their numeric index in the field list, then show me all those lists together at the end.
Looking again at which fields had this padding and which did not, I noticed a pattern. Most (but not all) of the ones that didn’t were mesh names. Meshes are stored elsewhere and don’t have entries in the PAR file. So, what if…
Turns out that FFFFFFFF field isn’t padding at all. It’s a marker to indicate that the string before it is a reference to another entity. The game probably uses this internally to drop in pointers or entity references into their structures after loading.
That meaning isn’t actually important, though. I just need to know which fields in which classes of entity get a marker field after them, and I have those lists now. I just have to load the appropriate list in with each CSV file and pass it to my PAR writer, which I then had add the markers in the right places.
Compiled once more, and…
Results!
…I think that’s a success.
I decided my next step was to load it into the game and see if everything still worked. I initially thought I’d need to pack it back into a .wd file, but after talking about my plans on the discord groups I mentioned last time, it was pointed out to me that the game will happily read the file if I just stick it in as Parameters\EARTH2150.wd in the game directory.
So I did, loaded up, started a new game, nothing crashed. So far, so good.
Time to try changing something. I went into vehicles.csv and changed the LC’s basic vehicle, the Lunar, to use a different vehicle’s model and have 555 health instead of its usual 240. Nice, easy number to look for.
Recompile, load up again, open the vehicle designer screen, and…
Next Steps
So, I now have what I originally set out for: the ability to modify units in game. There are two things I’d like to do now: finish my documentation so that others can write their own tools if desired, or at least understand mine; and fix some limitations these basic scripts have.
And oh, they have limitations. parameters.csv is still a mess and I really need to properly separate out its contents. Entity types are hard-coded to the structures I painstakingly worked out, but I suspect it may be possible to mod in other structures that my scripts just can’t handle. Bit-masked fields show up as 2048 and 8192 and such instead of anything useful. And there’s one particular set of entities that uses 32-bit floating point values instead of integers, and they just show up as nonsense in the CSV.
But hey, considering I only started this project ten days ago, I think I’ve done very well.
Posts in this series:
{% for post in site.tags[‘Reverse Engineering Earth 2150’] reversed %}
- {{post.title}}{%endfor%}