Thrives current AI seems meh to me. Out of curiosity, I was wondering how real microbes behaved. Well, we don’t know that well. But it has been noted that microbes are covered in sensory receptors and can detect both touches as well as a variety of chemicals in the environment. It was also noted that in some ways despite not having neurons they sometimes behave like simple neural networks.
I propose we replace the current microbe AI with a single layer NN that is trained with a genetic algorithm ( via Auto Evo ).
The AI can also mutate new sensory receptors. We would need some way to create a penalty for having too many receptors without giving the player an unfair advantage (having sight while the AI doesn’t is already a pretty big one) and also taking into account that on player species the player won’t need them but their AI counterparts of the same species will.
Allowing microbes to add pheromones is an important step toward emergent social evolution. Combined with the receptors and basic NN it could allow for rudimentary intelligence.
If you think you can pull off a neural network AI go for it, but I have been of the opinion that neural networks aren’t a good fit for Thrive for the following reasons, with some problems being more fundamental than others:
NNs take 100s of generations to grow into something useful via genetic algorithms, sometimes taking hours of computation time, while auto-evo is expected to complete during play, within a 30 second benchmark, on any players computer.
NNs are very good at deriving simple behaviors from base principles, but even in Microbe stage Thrive AI is expected to engage in behaviors that, while complex, are almost identical. Every single species that evolves in the game should be seeking compound clouds in almost the exact same way, and every toxin predator should be doing the exact same thing with it.
NNs don’t mesh with the existing behavior sliders in any obvious way, unless you wanted those to be the pathways of some fixed NN, which is generally not what we’re talking about with genetic algorithms.
There’s no way (that I know of) to get player behavior involved in the NN training, since the player can’t play thousands of iterations a minute like the computer needs to, meaning that these AI are training for a world where the player doesn’t exist, so you shouldn’t expect it to “learn” from your actions.
Genetic algorithms rely on simulation of the scenario that it’s being trained for, which the current auto-evo has no way of doing, so that would need to be built from scratch as well.
I’ve never seen a NN trained to the point of behavior as complex as something like an insect, so we would either need to drop this style of AI for later stages, or push the science of Machine Learning to new heights to make the later parts of the game.
Finally, quite frankly, I’ve seen the best fan responses come from interesting AI behavior, not nessicarily “smart” behavior. I think that people want to be surprised by the AI more than they want to be beaten by it, and I would be concerned that a pure machine learning approach would result in homogenous AI that does the same optimal thing every game.
I’d love to be proven wrong, but I think there’s a reason that machine learning hasn’t thrown out all other styles of video game AI just yet. Neural nets do some awesome stuff, but when you really think about the results of those experiments you see on Youtube, the AI is mostly coming up with solutions you could have come up with, and sometimes coded, in a similar timeframe that the algorithm took to derive it.
I see your points and some are quite valid. I’ll address your points as well as I can.
Auto Evo would need an overhaul. I personally think it does already but that’s another topic for discussion. This is a very simple NN (no hidden layers), which addresses the problem to a degree. I also think we should preload a semi-functioning network so they are not super dumb at the start.
I think the preloaded network should help with this, and I also think the biggest advantage here is the emergent behaviors.
This is valid. I think this system would need to go or at least be heavily refactored. Maybe in a way where the player can customize sensory receptors as well as add a bias to the NN.
This is also valid. There are a few methods I can think of that might be able to deal with this but no guarantees. Why this is a setback, I personally don’t see it as a deal-breaker.
See point 1
I think in the later stages it should still exist, but it will likely have to be supplemented with hard-coded AI behaviors.
Personally in my experience you have the backward. Players do love interesting behavior, and that tends to be exactly what this kind of AI algorithm provides. It tends to be very diverse, and with other random systems like auto-evo allowing for a world of emergent behaviors, it is quite the opposite of a hard-coded AI that always stays the same.
On this last point, the point of a system is not about optimizing play strategies or doing something quicker or smarter than us. It’s about the evolution of behaviors in a game about evolution.
I like going back and forth on theory as it helps iron out any kinks. If I can get the idea to a point where it looks solid, I plan to make a prototype to see if it will work.
Alright, so the matter of what players like isn’t something we can sort out, so if this idea is viable it’s worth building out and putting in front of playtesters. Ergo, I will can my final point.
My concerns with other systems needing overhaul is ultimately a beurocratic question, and a matter of determination. The reality is that too big of a refactor just won’t happen in a software project that’s already underway, but it’s a judgement call how big that is and I don’t have an argument to convince you other than, “dude, trust me”
You’ve responded to my concerns involving details to the simulation and later stages with more specific and tactical ideas, and to me that sounds like a great candidate for a prototype. Personally, I don’t think that a one-layer neural net is going to handle behaviors as simple as deciding if another microbe is a good prey candidate, but I certainly can’t prove that on paper. If you have specific questions for me I’m happy to work with you on them, but I’m not sure where to go with this besides, “prove me wrong”
I think the only way I could personally theorize about this (but I’m not the only dev here) would be to constrain the problem to make an AI using NNs that solves a smaller sub-problem that NPC microbes need to do, then figure out how a new auto-evo could grow or load a NN to that point, then expand it to new cases from there. I, personally, might be convinced of the issues around all the overhauls that need to be done, but first I need to be surprised about what a neural net can really do.
Honestly, I don’t think you could have given a better response. If I find the time, I might start working on that prototype. If anyone else has any concerns or ideas I would still love to hear them though
As you stated, microbes don’t have neurons. So a neural network to simulate their behavior seems entirely unnecessary. NNs tend to very wasteful in energy (computations) and that’s why microbes don’t have one in the first place. Now, you state that:
It was also noted that in some ways despite not having neurons they sometimes behave like simple neural networks.
I’d really like to know your source for this, for starters, since otherwise I can only make blind guesses. But even assuming you could model this as such, I see two pitfalls.
First, neural networks are famously obscure: how would “genetic behavior”, as manipulated by the player, relate to them? How can we explain this to the player ? On a meta standpoint, how do we get future developpers to debug, balance or adapt something as obscure as a NN? Symbolic behavioural models seems much more appealing in this regard, since we can build them incrementally, and more importantly, control them. Remember that Thrive is also adapting to scientific knowledge, and we’re not safe from discovering that a model can not cover the whole range of behaviors, and so we’ll have to adapt it. Which seems more feasable with a symbolic model.
Aside from clarity, the other issue is efficiency vs organicity. NNs may build up very effective behaviours… even perhaps a bit too much. As I said in my introduction, neural networks come at a cost in vivo. This is why bacteria rely on other processes, and may sometimes fall into biases/limitations specifically driven by mechanical constraints. How would we deal with that in a NN?
Overall I feel this idea of a NN-determined behaviour is at odds with Thrive sciency aspects (science is not only about replicating phenomenons, also about understanding them). This is why, even with a snapshot of a functional behaviour-deciding NN, I may actually prefer keeping a symbolic-model-driven behaviour, for both clarity and robustness. And the current models also deserve some love as well, I’d say.
But as I said, I do not know your source on this behaviour modelling by NNs, and reading it might lift some of my current worries. In the end, the decisions to try out new prototypes is yours anyways, and it may give us some ideas whatever the result is.
The current run and tumble, as far as I remember, has been okayed by the theory team to be basically be how microbes behave in real life. So if we deviate from that it needs very strong argumentation as to why we are making the game less realistic. Of course a valid concern is that if the run and tumble behaviour is too “dumb” for microbes to really survive in the game environment, then it is something we need to address
While much less talked about recently, players really seem to be enthusiastic about getting to design how their species behaves. The behaviour tab had been waited for many years and I think quite many players find it enjoyable to pick behaviours for their species. With this in mind with a NN it wouldn’t be possible for players to do this anymore. So basically if we go with a NN we are cutting out a pretty major gameplay feature out of the game that currently exists, and there are plans to make it even more advanced (Behavior Editor Implementation - #13 by Buckly).
Here are some things I will point out that hopefully eases some concerns:
The AI algorithm proposed is a Genetic Algorithm. It allows behaviors to evolve in the same way that everything else in the cell does.
A NN like the one I’m proposing has no hidden layers. It basically works like cause and effect response to stimuli, but one that can be tweaked to suit the cell (Which is actually too simple for some behavior as some been have shown changing strategy IRL, but encompass how most behave.)
With a network like this, if we want we should be able behavior sliders by having them edit the output layers biases. That way you could make the cell by default prefer some actions.
Hmm, I definitely don’t know more about this than the theroy team, but me and some others have noted them locking on and running stright at stuff instead of the bumbling about you see in real life?
Run and tumble is only used when the AI is just looking for compounds. When they focus on a target (a chunk or a cell) they will chase and perform much smarter actions. This was done so that the AI is more interesting to play against. Before we added run and tumble the AI cells just randomly moved while looking for compounds. And that was greatly improved by run and tumble.
I can’t find my original source again, but I found this ( https://www.frontiersin.org/articles/10.3389/fmicb.2015.00264/full ) other paper that summarizes what I have learned quite well (despite being quite dense). The use of NN in this case is talking about a biologic one, but done right a computer one should work too.
I say that as long as the parameters of the network make the network produce 6 categories of behaviour like in our tab, and combinations thereof, go for it.
