Alternative ways to fix the performance

Meta
1

I’ve tried to profile the game to look for places to improve the performance (Increasing game performance) and I’ve managed to do some things by adding multithreading for part of the microbe operations etc., it’s still not exactly good. And we had to put in the entity limit into the game, which can be lowered to a pretty ridiculously low number. And even then the normal value isn’t exactly a good performance even on a fast system. With me pretty much out of ideas having tested disabling various parts of the game and doing multiple types of profiling, and it seems no one else so far has been able to fix the performance either, let me present 2 radically different ideas I have.

Before presenting the ideas I just want to preface that these aren’t really anything I could work on soon, and both of these would be huge changes with very major ramifications. One of them might even turn us back towards the old situation where we really had a difficulty of finding any programmers willing to work on the game. The reason these would take so long as they would require a lot of time to redo quite large parts of the game, and would forever in the future need people to be at least a bit familiar as to how that system is specifically done as we wouldn’t be using the normal Godot Node approach.

I think that’s enough preface so the two ideas are: separating the Godot rendering (maybe not even using Nodes but Godot’s graphics component directly) from the C# gameplay logic which we could make very multithreaded, and the second idea is to move the gameplay code into C++. I’ll cover the two ideas more in-depth next.

For splitting the rendering from the game logic (this’ll basically be required in the C++ game logic as well, so I’ll cover this first), the idea is that it kind of seems like the Godot scene tree is just too slow for what we want to do (though maybe something like multimesh and custom rendering of microbe parts might be a lighter weight alternative that fixes this, but still would require a lot of refactoring) so we’ll get around that. Basically what we would do is have a Godot Node that’s responsible for running the game logic with multiple background threads (we’d need to do our own bullet physics integration but I already wrote that once for Leviathan so I’m sure I’d be able to write one again) and then only use Godot to draw the cells that would be on screen (in the most efficient way possible, we might need to directly call into the Godot rendering system). This should save a ton of time Godot needs to spend internally to keep the scene tree updated. I think the main problem might be that we have such big hierarchies of nodes making up the microbe entities. So rendering those as multimeshes or some other instancing approach, and not using the Godot scene tree at all for cells that are offscreen should give us a massive performance boost.

So obviously it would be a pretty huge change to the game code to switch around how the microbe (and other entity) logic is simulated and how Godot is used to draw stuff. But at least we can use all our existing C# data classes so only the game entities and gameplay systems would need to be redone to work on the new abstract entities that are not Godot objects. There’s also a small chance that the part rendering with Godot is too slow (or needs to be written in C++) or hard to make. So in the long term this wouldn’t be that much of a problem but would require a month or two of my time (probably) to make it all work, and a little bit of an explanation document regarding the architecture.

Onto the much more involved approach, which is the conversion to C++. Basically the idea here is the same as above basically but in addition we rewrite most of the game’s code in C++, we could still keep some GUI code and components in C#. This has the major downside that all programmers will need to have C++ compilers installed and as such with native code libraries etc. (we might need a native bullet library anyway for the approach one so this might already partly be a problem there as well). We can partly alleviate this by having precompiled libraries and game logic so that anyone not working on the core game logic, wouldn’t need a C++ compiler, just ruby to run the download scripts that would allow downloading the right precompiled files. Though, maybe the core gameplay library would also be a bit of a hassle to have be precompiled, though I suppose we already have devbuilds so merging the devbuild system with storing precompiled game logic binaries might be doable. Also the libraries will be platform specific so for making releases we’d need (or I’d need) to spend more effort on the way the game is packaged to make it at least a bit more streamlined than manually needing to compile and upload the releases from all platforms.

The requirement of writing game code in C++ could mean that we wouldn’t have as many available programmers anymore as writing C++ correctly is a lot harder than C#, so we’d need to actually target much more skilled programmers who could help us. Lower quality C++ with mistakes in it can cause really hard to find bugs and random memory corruption crashes, C# protects us from those. So C++ needs more skill to get right without bugs. Though I fear we would have much fewer programmers, it might not actually be the case, if we assume that before our trouble was that we were just less known. Using Godot for rendering the game might be enough of a name drop that more people would be willing to work on Thrive based on that.

Due to the way more extra complexity of this approach, I’d roughly double or triple the time, so up to 6 months this would take of my time. However, this would be the ultimate performance we can get without also ditching Godot as the rendering library, because we’d be using C++ for basically all of the time critical code so mostly our skills would be the limit, and I guess in a few cases Godot GUI Node bugs and limits could still bite us. Yeah, in summary this’d be basically doing one more full conversion of the game to a different programming language and decoupling a lot of it from Godot. I don’t consider this much of a benefit anymore (considering how much time I’ve spent on engine conversions already), but if this was completed, it’d be pretty easy for us to basically switch to any rendering library we liked, in case Godot was ever considered not the best choice, though I kind of doubt that unless we go with a really basic rendering library, we’d just be switching some bugs for other bugs.

As an added bonus either of these approaches should fix the physics related crashes, or at least make it so that the physics crashes would be our fault if they keep happening.

Godot 4.1 (4.0 doesn’t promise to have OpenGL fallback, only vulkan, so we can’t really consider using that) might change the situation with their own physics and presumably improved 3D scene handling. Which might help us a lot without us needing to really do anything.

I don’t think I can come up with any concrete conclusions currently, I just wanted to write all my thoughts about this in a single place. And hear what other people think about the tradeoffs. Also please ask for clarifications if you have questions, those will also help me clarify the ideas to myself that are jumbled in my head.

6 Likes
2

Those are colossally daunting tasks, indeed I fear the end result will be how it’s like in the Leviathan era (presumably even harder to maintain). Will we lose access to the Godot editor with these approach? I think this is very important because I personally very much depend on the 2D editor for an intertwined workflow of prototyping and later creating complex UI, Godot’s editor has been very useful on this front.

Regardless, I’m leaning towards a more safer approach to stick using Godot as a whole, especially with 4.0 on the seemingly-near horizon there’s a chance this work might even become redundant. Moreover, Godot 4.0 seems to have OpenGL support merged already, at least OpenGL 3.3 from the looks of it. Even though it’s not a maximum compatibility it still offers some compatibility with lower-end devices, I’ve also heard some kind of GLES2 support allegedly will be included with very basic 3D which I think is even adequate for our uses.

There’s even GDNative (or its successor GDExtension in 4.0) for C++ usage without compiling or modifying the engine. Now I haven’t look much into this and its specifics but from the sounds of it, I think this will help immensely if we ever want to write some part of the code in C++.

Overall, I’m just worried that all this effort will be wasted if it turns out that later Godot versions prove to perform better at a lower cost of work and maintenance.

1 Like
3

I mean we don’t use the Godot editor for editing the microbe scene much, right? I still imagine we can have individual pieces of graphics as mostly separate Godot scenes. And of course all the GUI scenes and GUI components can stay as they are.

Which is exactly why I don’t plan on changing the GUI workflow, much. If anything we’ll need to decouple the GUI a bit more from the game logic if we go the C++ route. The communication in that case would be more that the GUI code would export many more variables that the C++ side would set to manage the GUI state.

Well even Godot documentation admits that their scene tree is not usable for all use cases (they are probably talking about 3D graphics). So we’d still be using Godot entirely, just that our 3D graphics need to go to their lower level rendering systems for performance reasons.

That’s what I would use because recompiling the engine would be a pain. But still those gdnative libs need to be recompiled for each platform each time the game logic would change.

In fact I’ve already written a small school assignment game in C++ using Godot and gdnative:

Other than the really lacking Godot documentation for how to use it from C++, it wasn’t that difficult. Though I would imagine that people who aren’t familiar with Godot would have a difficult time picking up C++ development in Godot directly, much easier to do that if you know how Godot works first and can map the code you want to write to C++ and try to find how to do the equivalent Godot operations.

Which is a very fair thing, but seeing as Godot 4.0 release date is becoming a bit of a meme, we might still be waiting for a long time. Also the Godot developers don’t seem to want to merge my PR into Godot even though it’s been ready for months (it’s targeting the 4.x branch) Add Breakpad for crash dump generation by hhyyrylainen · Pull Request #56014 · godotengine/godot · GitHub

4

The way I see it is, if these are very low level and important scripts, it’s unlikely novice members were ever going to work on it effectively anyways. (Example: the cloud code, the membrane code, etc)

We have some really good people nowadays so maybe putting some critical parts into C++ would not be the biggest barrier to our more senior members. I know you and myself both understand C++ pretty well, and members like FuJa for example, already have enough general programming experience where they probably wouldn’t struggle too bad.

5

The problem with move to C++ would be that, we can’t have part of the code really in C# and other in C++. So we need to pick which side should have the Species class and then have to pull a ton of code to that side.

Critical parts of clouds and membrane generation would definitely be possible to move to C++, but the way I see things those do not explain our performance problems, so doing that won’t fix anything. Though it will give more baseline FPS on slower systems, which while nice would still bring some of the C++ difficulties related to precompiling the binaries and making releases, while still not fixing the game performance with a lot of entities.

So one additional benefit of the second approach would be that improving the cloud and membrane generation performance would be very easy and natural to do at the same time.

6

Considering that the second approach builds off of the first approach, is it possible to first implement the first approach, and then after 3 months if we see it did not improve the performance enough, we escalate into the second more intense approach?

Also, how much research can we do beforehand into the potential effectiveness of these approaches? If they could take potentially 3-9 months to fully implement, I think several weeks of research just to ensure their effectiveness would be worth it.

7

That is certainly possible. And at least doing the approach 1 first will validate if the performance improvements are there to be gained.

I can’t for certain say whether rewriting a bunch of the C# code and then converting as is to C++ is easier than converting to C++ while refactoring the entity handling approach in one go. One of those may or may not be less effort than the other one.

Well doing approach 1 would kind of already validate whether approach 2 has a chance of fixing the performance.

Also I opened a thread recently that would validate the idea of using C++ components for speed, so if this is done first: Using C++ components in the game it will set the stage for more C++ stuff and also give us results as to whether the kind of speed up we want from using C++ is actually achievable or not.

For just getting started I see just two choices:

  • Someone edits the game to take out the entities from the Godot scene tree and still render them somehow and runs logic for them to check if the part which I think is slow (letting Godot handle the scene tree and collisions in there), and sees how fast that is
  • The custom physics integration and running the entities separately just needs to be done in a quick and dirty way that doesn’t necessarily have all features working and might crash often, but as long as it is stable enough we’ll get some performance numbers with that
8

I suddenly realized one more thing regarding this based on the recent bit of prototyping done using Godot networking, which is that if we add multiplayer we cannot use Godot’s inbuilt Node replication mechanisms for that. So instead we need to do entity data updating and local prediction code ourselves, maybe even use all network connectivity outside Godot. For example we could use a RakNet fork like GitHub - SLikeSoft/SLikeNet: SLikeNet™ is an Open Source/Free Software cross-platform network engine written in C++ and specifially designed for games (and applications which have comparable requirements on a network engine like games) building upon the discontinued RakNet network engine which had more than 13 years of active development. as the networking library as I’m not keen on rewriting that from scratch as I think the entity replication and prediction will be difficult enough on its own.

9

Seems like a good library, and as the feature overview claimed, looks like it has some feature parity with Godot 4’s networking. I assume we’ll only go with this if the goal for C++ rewrite is solidified or using this just in case we’d do the rewrite in the future? Otherwise I think the networking provided by Godot already is probably good enough for our needs and convenience.

10

Yeah, I think the networking provided by Godot is good enough if we use a Godot based gameplay logic. I just wanted to bring up this additional hurdle that if we add multiplayer that people enjoy playing we need to reimplement the multiplayer as well if we entirely change the game architecture.

1 Like