System dependencies and parallelism

Programming
1

I staeted a small prototype of the systems and gamestates where you can tell what systems does each system depend on (as in, what systems they expect to have updated before they are allowed to update).
Then the gamestate can use that information to manage the systems.
I think if we do something like that on an actual engine we can have an automatic paralellism between systems, which could be good for performance.
This is the prototype. Beware, i needed a lot of templates to make it work so it has more angle brackets than xml :confused:
Also, it needs the BOOST libraries.

2

What I’ve been thinking of doing for Leviathan is grouping systems so that each group is ran in parallel. That way each system that needs to run before some else is placed in the same group with a smaller run number so it is sorted to be before the system that depends on it.

3

That works too but i like the idea of explicit dependencies because it’s clearer and easier to understand (and possibly enforce so that people don’t shoot themselves in the foot) and more efficient (if a system A has to run after B and C then B and C can run in parallel if they’re independent).

4

I improved the design of the gamestate (now it has less inheritance nonsense) and finished the parallelism.

5

I’ve now fully reviewed the code and it’s pretty cool. Especially impressed how you managed to get it to work fully with templates for numbering all the systems and collecting the dependencies.

But it would need some tweaking to throw it into the engine_refactor. It should be done in the ruby file to make compilation faster (don’t need templates when the systems have been pre-sorted based on the dependencies) and better fit into the current system. Currently the system ordering (world class generation) is quite a mess, but I hope to split it into a few files and structure it a bit better. But here’s the file and the part where the system order is sorted: https://bitbucket.org/hhyyrylainen/leviathan/src/81a7b35dc011f8b90be6b4f748af1b993fe83d4c/Helpers/FileGen.rb?at=develop&fileviewer=file-view-default#FileGen.rb-828

6

I made an incomplete prototype of code generation of the world (in the latest master branch) using macros, which i like more than the ruby script because they’re a part of the language (instead of a whole new language) and they can read from c++ files (which mean you can write stuff in the system’s header file), although it could still require some code generation if the engine needs to do anything too fancy… (plus, as macros, they can look kinda ugly)
the prototype
to use it you just replace the generated files with this files and disable the ruby generator.

7

That is pretty nice. But there are a few things the macros don’t handle and changing the components / systems running parameters would need changes to both of the files (the list of components and the Tick method). But these are the bigger issues:

  • _ReleaseAllComponents doesn’t use the macros.
  • Also the MembraneComponent destroy must call “Release” on the component (so there are two cases of components, which may be very hard to handle with templates)
  • “// Component types of parent type” don’t get automatically done (the ruby script automatically fills the component types of the parent class)
  • The huge added mess needs to be handwritten. I’m not looking forward to finding bugs in code like this:
if(!addedAgentCloudComponent.empty() || !addedPosition.empty() || !addedMembraneComponent.empty() || !addedCompoundAbsorberComponent.empty()){
    _CompoundAbsorberSystem.CreateNodes(
        addedAgentCloudComponent, addedPosition, addedMembraneComponent, addedCompoundAbsorberComponent,
        ComponentAgentCloudComponent, ComponentPosition, ComponentMembraneComponent, ComponentCompoundAbsorberComponent);
}

So I’d still say that it will result in less bugs and headaches to use an actual scripting language for the code generation rather than try to use macros that can’t handle the special cases that easily.

8

Gigantic necro but (as i mentioned on the discord) i made a prototype that can update system in multiple threads with dependency management at runtime. It also allows for systems to be added individually so you could have worlds inheriting from eachother and everything should work ok.
It would still require code generation in order to deduce the dependencies, generate the boilerplate and pass the parameters to each system.

1 Like