12 replies to this topic

[chipgw]

A while back I started working on what I intended to be an open source remake of the SAGE engine capable of running BFME games in place of the original game exe. The intended advantages of this over the original exe would include supporting Linux and macOS, out of the box online multiplayer (no need for LAN emulation), and better support for newer systems. I laid some groundwork (which you can check out on GitHub), but I didn't get all that far before getting busy with school and work.

 

Now, writing a game engine from scratch is a huge task, so in my recent desire to get back to it (thanks to an email from someone who found the project on GitHub) I've been considering using Unreal Engine 4 so that I don't have to write the entire thing myself. However, this approach would have other drawbacks (loading original game assets would be more difficult, for example) and would be a change in direction from my initial goal for the project. (More a fan remake like Black Mesa or KOTOR Apeiron than an engine remake like OpenMW)

 

Here's the pros and cons I see so far:

UE4:

++Robust engine, graphics, and network code

+Looks great

+Already compatible with a wide variety of hardware and platforms

--May need/want to remake models, animations, and maps

-Not sure, but making and distributing content in addition to code may be more risky legally (AFAIK worst case is a C&D letter, but still...)

 

From scratch:

+Should be easier to support existing mods and custom maps

+Not having to work around an existing content system that isn't directly compatible with SAGE assets and isn't really made to be extensible.

--Engine code from scratch is a huge task. Huge.

--Robust multiplayer support is beyond my ability

 

Both would still rely on the original sounds, INI files, and at least some of the textures, thus requiring a copy of the original game to play.

 

Which one I decide to go with will depend on the kind of people who are able to contribute. Asset creators would only really be useful for a remake in Unreal Engine, C++ coders would be good for both, but more so for a pure engine remake. (especially anyone experienced with network code, I could do graphics and core game logic by myself given enough time, but networking isn't something I know anything about...)

 

We all have lives, whichever way this project goes it will a very long term thing. It will be quite some time (likely years) before there's anything to release even as a prototype/alpha.

 

TL;DR: Any programmers and/or modelers who'd be interested and able to join me in my crazy desire to remake BFME games? (2 would be the first target, but all of them would be nice...)

 

Also, sorry if this is a bad/wrong place to post this...

[Plokite_Wolf]

You can connect with the guy who's making Thyme, the open-source reimplementation of C&C: Generals/Zero Hour, which was originally made by the same team on basically the same SAGE engine.

[chipgw]

You can connect with the guy who's making Thyme, the open-source reimplementation of C&C: Generals/Zero Hour, which was originally made by the same team on basically the same SAGE engine.

He actually contacted me a couple weeks ago with a question about the .big file format. It sounded like his project was heading in a different direction (integrating with C&C's original code), but I guess that's just different means to the same end. Maybe I should look into it more...

[Pixel]

I’m quite surprised no one’s brought this up before. Sure, the SAGE engine is fantastic but it’s limited these days. A few weeks back, I started on something similar. I never really intend to release it. It’s more of a personal project to see what’s possible, rather than anything else.

 

Personally, I chose Unreal Engine 4. Mostly, because I have previous experience with it and Blueprints allows for rapid-prototyping. There really aren’t that many hurdles to overcome to make it happen, at least, in the short-term.

 

Essentially, for models, there’s really nothing stopping them from being brought right into Unreal. Import it into Max and export it out into the correct format (in this instance, fbx). They’ll work – but they won’t be animated.

That’s where it becomes slightly more tricky. Unreal only allows for a single root bone to be present. BFME and SAGE don’t conform to that standard. Yet, it’s not a problem that’s particularly hard to solve. Simply put, it requires modifying the skeletons and animations themselves. It’s more than able to be done (and I’ve attempted it successfully).

 

There’s some definite benefits that a more modern engine (like UE4) would bring with it and even the possibility of large-scale battles never thought possible of in BFME. But, it’s mostly a pipe dream unless a team attempts it, of course. It’d take more than one or two people, in my opinion (the sheer amount of assets that need to be ported over alone would take time).

 

For me, I spent about a week working on a port of BFME1’s Moria level from the campaign. I’d never worked with the Moria assets during my modding days, and quite frankly, they were terrible. As such, I spent around two days breaking them down into individual parts and unwrapped them from scratch (fun!).

 

Next up, I started porting over the actual characters I’d need for this purpose. That includes the fellowship, trolls, goblins and the balrog. I have most of them done, and simply need to finish up a few animations before I bring them in. After which, I’m planning on incorporating AI and the like. There's very few props on the Moria map at the moment, but I have them ready to bring in next. The lighting and mapping overall, could use some work, as well - I'll be the first to admit. I'm not a mapper, and lighting is far from a strong point of mine. Here’s what I have running in UE4 at the moment, though:

 

 

Edited by Pixel, 24 April 2017 - 08:54 PM.

[Kwen]

This is very cool.

[chipgw]

I’m quite surprised no one’s brought this up before. Sure, the SAGE engine is fantastic but it’s limited these days. A few weeks back, I started on something similar. I never really intend to release it. It’s more of a personal project to see what’s possible, rather than anything else.

 

Personally, I chose Unreal Engine 4. Mostly, because I have previous experience with it and Blueprints allows for rapid-prototyping. There really aren’t that many hurdles to overcome to make it happen, at least, in the short-term.

 

Essentially, for models, there’s really nothing stopping them from being brought right into Unreal. Import it into Max and export it out into the correct format (in this instance, fbx). They’ll work – but they won’t be animated.

That’s where it becomes slightly more tricky. Unreal only allows for a single root bone to be present. BFME and SAGE don’t conform to that standard. Yet, it’s not a problem that’s particularly hard to solve. Simply put, it requires modifying the skeletons and animations themselves. It’s more than able to be done (and I’ve attempted it successfully).

 

There’s some definite benefits that a more modern engine (like UE4) would bring with it and even the possibility of large-scale battles never thought possible of in BFME. But, it’s mostly a pipe dream unless a team attempts it, of course. It’d take more than one or two people, in my opinion (the sheer amount of assets that need to be ported over alone would take time).

 

For me, I spent about a week working on a port of BFME1’s Moria level from the campaign. I’d never worked with the Moria assets during my modding days, and quite frankly, they were terrible. As such, I spent around two days breaking them down into individual parts and unwrapped them from scratch (fun!).

 

Next up, I started porting over the actual characters I’d need for this purpose. That includes the fellowship, trolls, goblins and the balrog. I have most of them done, and simply need to finish up a few animations before I bring them in. After which, I’m planning on incorporating AI and the like. There's very few props on the Moria map at the moment, but I have them ready to bring in next. The lighting and mapping overall, could use some work, as well - I'll be the first to admit. I'm not a mapper, and lighting is far from a strong point of mine. Here’s what I have running in UE4 at the moment, though:

 

 

That's pretty neat. I could probably get it looking nice as far as lighting goes, but unfortunately porting assets by hand like that isn't something legally possible AFAIK. My understanding is everything would either need to be imported at runtime or created from scratch...

 

Between myself and the other person we decided it would probably be best to put work into Thyme. Once that project is finished enough that it longer requires code from the C&C Generals game.dat it should be relatively simple to make a BFME version of it. (that is, simpler than starting from nothing)

[Bashkuga]

I can only say good luck guys! I'm looking forward to what you will make out of this .

[NDC]

Hey chipgw!

Sounds like a very exciting and much needed undertaking! I wish you the best of luck!
I think there is one other project heading the same way:

https://github.com/feliwir
https://github.com/feliwir/openSage

Have you seen that?

By the way, I have some knowledge of how BFME animation files are structured - please let me know if I can be of any assistance.

[Blade]

He actually contacted me a couple weeks ago with a question about the .big file format. It sounded like his project was heading in a different direction (integrating with C&C's original code), but I guess that's just different means to the same end. Maybe I should look into it more...

 

 

 

There are different approaches to recreating a game engine and they tend to have slightly different outcomes.

 

Thyme is using a bottom up approach similar to that used successfully by OpenRCT2, using the original binary as a scaffold and basically replace the original classes and functions one at a time with your own until you have a working engine that doesn't depend on the original any more. In terms of how that works, its very similar to how Ares for RA2, TT Scripts for Renegade or GenTool for Zero Hour work, only with the explicit goal of reimplementing everything rather than just what is needed to hook in new features (TT scripts in particular appears to have reimplemented a very large percentage of Renegade making it a useful reference for the W3D parts of SAGE code).

 

The benefits of doing a reimplementation this way is that you get a working game on day 1 and the end result is very close to the original in terms of behaviour. The downside is that you are very constrained in terms of class layouts and function parameters, they must match the layout used in the original exactly. Its also difficult to make a cross platform version as you rely on functionality in the original binary until everything is reimplemented.

 

All the other projects that I've looked at including the ones mentioned so far are using a top down approach, implementing file parsers based on what is currently known about the formats and building an engine that kind of behaves like the original game around that from scratch. Examples of this kind of recreation would be warmux for worms or OpenRA as a reimplementation of the early C&C games.

 

The benefits of this is that its easier to rebrand the project if copyright holders come knocking, just dump any original assets you are using and roll your own and you are your own game since you won't have touched much (if any) of the original code. Its also much more like the experience you would need to write games and engines as a game dev is that is what you aspire towards so some aspiring developers might find that more attractive that essentially following someone elses blueprint. The downside is that you end up with something that really isn't related at all to the original games engine wise, the assets just form a skin on top of your own game. Its also takes much longer to get something playable which makes it harder to keep pushing on with.

[Blade]

Hey chipgw!

Sounds like a very exciting and much needed undertaking! I wish you the best of luck!
I think there is one other project heading the same way:

https://github.com/feliwir
https://github.com/feliwir/openSage

Have you seen that?

By the way, I have some knowledge of how BFME animation files are structured - please let me know if I can be of any assistance.

 

You should join the #thyme irc channel on freenode, feliwir is often online there and was looking at the animation formats, particularly the newer formats used in BFME2 if you have any information on them. I think currently he's trying to understand and reimplement the apt format used for the UI in BFME and later SAGE games.

[NDC]

Hey chipgw!

Sounds like a very exciting and much needed undertaking! I wish you the best of luck!
I think there is one other project heading the same way:

https://github.com/feliwir
https://github.com/feliwir/openSage

Have you seen that?

By the way, I have some knowledge of how BFME animation files are structured - please let me know if I can be of any assistance.

 
You should join the #thyme irc channel on freenode, feliwir is often online there and was looking at the animation formats, particularly the newer formats used in BFME2 if you have any information on them. I think currently he's trying to understand and reimplement the apt format used for the UI in BFME and later SAGE games.

I'll do that, thanks for the heads-up, Blade!

I think the BFME2 format has not been decoded directly yet, but I've found a way to extract animation channels from it, and then convert those into their better known BFME1 counterparts (here's the dedicated thread). Potentially, once all animations are converted, there should be no need to decipher it in the first place.
I know it sounds a bit awkward, but that's the best idea I could come up with so far.

Edited by NDC, 19 September 2017 - 02:54 PM.

[NDC]

There are different approaches to recreating a game engine and they tend to have slightly different outcomes.
 
Thyme is using a bottom up approach similar to that used successfully by OpenRCT2, using the original binary as a scaffold and basically replace the original classes and functions one at a time with your own until you have a working engine that doesn't depend on the original any more. In terms of how that works, its very similar to how Ares for RA2, TT Scripts for Renegade or GenTool for Zero Hour work, only with the explicit goal of reimplementing everything rather than just what is needed to hook in new features (TT scripts in particular appears to have reimplemented a very large percentage of Renegade making it a useful reference for the W3D parts of SAGE code).

This sounds so exciting, Blade! Do you know some books/resources I can study to get a better grasp of this process? I have some programming background, so hopefully it won't be learning completely from scratch!

[Blade]

It depends on which aspect you want to understand. For the actual dll injection where you make the the original binary load your new dll, there are a couple of ways of doing it. You can patch the original with an extra dll import (I believe OpenRCT2 did this), but then you need to ship a patched original binary. You can make your dll pretend to be a dll already loaded by the original and then forward the function calls it makes to the actual original as well as patching in your own code when the dll loads (I believe GenTool does this). Finally you can have some kind of loader process that starts the target exe, pauses it immediately and then patches in a dll load command in memory before resuming the exe.

 

This final option is what Thyme uses, a small launcher program is also compiled along side the dll to do the startup and force the load of the dll. To get this information I basically did a lot of googling and consolidated information from different sources. Fortunately the work is done now, so in general any contributors to thyme won't need to know this stuff.

 

Regarding the actual reimplementation, mostly what you need to know are C++ to write the implementations and some disassembler and ASM knowledge to examine the original binary to find and study the original functions. If you were to purchase or otherwise acquire a commercial version of IDA it can also generate pseudo C code to make understanding a function easier.

 

Once you know the address in the binary that the original function lives at and you have your reimplemented version, there are functions in thyme that run when the dll loads that you can add to that take the original address and a function pointer and patch the original function to jump straight into your new function at run time so when the original code tries to call say the open file function, it will actually jump into your open file function.

 

There are also special functions that go the other way. Say there is a complex function you haven't figured out yet that gets called everywhere, you could use its address to create a function pointer to it to call it where its needed in your own code. Similarly you can access global variables that are shared between functions in the same way, by making a pointer or reference to them.

 

I would suggest reading some tutorials on writing code in x86 assembly to get an idea of how a program is structured after it has been compiled and probably grab at least the free version of IDA and load the ZH binary. When you load it, it will probably only reliably find winmain, I've done a fair amount of work at my end identifying functions in the game which makes it look a little less daunting and confusing. For BFME games, I recommend you load no-cd versions that have had safedisc stripped out, otherwise disassemblers have a hard time working out what is what due to how the protection worked. Its always the game.dat file that is the actual exe BTW, bfme.exe or whatever its called is just a loader. I suspect the unofficial patches that ship new game.dat files to not required the mini cd images are no-cd exes that have had the safedisc removed, so they will probably be fine.