Stellaris Dev Diary #172 - Reworking the AI

[Moah]

"The Contingency will systematically try to stop the biggest threat to the galaxy, until nothing remains" - you mean what...they kill themselves on arrival?

That would be a hilarious event chain to have. Introduce an ID4 type of virus where they start making that calculation

 

[Moah]

And of course: which AI do you consider more competent at the moment - Stellaris or EU4?
Thanks!

AlphaGo Zero.

 

[MatRopert]

Don't get me wrong, I think the work you've done is absolutely solid. But really you had to make experiments to reach this conclusion? Doomstacking is the only sensible strategy in Stellaris, even if the general public on this forums consider this statement to be an offence against the community.

Well that's the thing, how fun is it to play against an AI that will always curb stomp you? It's sometimes easier to make an AI that is efficient rather than entertaining (take tic-tac-toe as an extreme, it's trivial to make one that cannot loose).
That's why we'll be monitoring the feedback...

I'd like to suggest this info is available by default if you select the fleet of an AI you're allied with.

That UI is a bit too unpolished to be used as a game feature, but we'll keep in mind that you guys would like to see more of what your AI allies are thinking at a given moment.

 

[sidestep]

Hm, another thought just occurred to me: the focus surplus, at least in minerals, isn't something that depends on the stage of the game, but rather on the number of colonies you have (and are actively developing). Would it be possible to take this into account, or is that too computationally-expensive to do?

Edit: also, many thanks for sticking around in the thread, it cannot be repeated often enough - greatly appreciated

The plans could for sure be more complex and take these things into account, but that would be future improvements. Most things are not too computationally expensive, it's mostly just the jobs that are difficult to compute beforehand.

 

[MatRopert]

Is every AI "minister" behaving the same whatever their ethics, civics and personality?)

They use the same ministers but even those will vary in behaviour depending on some factors.
For example the military AI will commit between 80 and 100% of its fleet power to offense depending on aggressiveness.

What is really deflating (not entertaining) is having the AI send its forces piecemeal into a grinder I have prepared because it couldn't simply combine all its reources into an effective stack that could actually do some damage.

It shouldn't happen as often (if at all) now. They'll always try to regroup first and if they can't the entire group should disband and fleets reassigned to other targets.

For the military AI, I hope it is taking distance into account when making decisions about what target to attack. It would randomly path out 4 year trips to attack space stations in the rear of the enemy territory, which is not optimal.

Targets are prioritized by how deep they are in enemy territory. It's unlikely that they'll go for something that is 4 systems deep before they occupied systems on the way.

@sidestep, @MatRopert thank you for those great dev diaries and for sticking around to answer questions! Will you be continue to work on the Stellaris AI, or do you have to move on to other projects?

Moving to another project? Hell, I just got here!
But @sidestep did move to Imperator already, et tu brute? .

 

[sidestep]

Since I am actually thinking it through before writing something extensive, here are my thoughts on early focuses.





Alloys focus 50 for early game? Are you implying fighting DEs by default?
Habitat costs 3000 alloys and 200 influence. Optimistically you have +5 monthly influence by that stage, so it 40 months to get the influence. 50 net alloys is a nice figure if you are spamming habitats or actively fighting somebody, but since when habitats are early game? 20 alloys net would be fine. 30 is for rushing somebody's capital by 2020. Even for habitats stage, 50 is not needed, by the time you get it, you can have like 9k saved. If we are talking standard settings no magic levels of skill, habitats appear like 70 years into the game. 70*12*10 is a bit less than 9k, but a bit. Make it 20 and that's enough.

Food focus 10? Encourage planetary growth costs 1000 food per planet for 120 months. 12 (actually, it's 1000/120=8.3) net food for each planet is the focus.

Focus minerals 30? Buildings costs are in the hundreds. Most common 2nd building is the robots assembly which has a price tag of 500. Though since default AI is probably not going to replace starting buildings, 30 may be enough.

Consumer goods nobody cares about, unless you are planning to use them for planetary decision or for bribes, which AI probably is not going to do. Leave it at 5. Unless AI is going to sell them.

Early game energy is like the most important thing, you need 300 to clear blocker that grants extra pop, you need 200 per scientist. If we are talking about strong AI, it is looking to get something like 5 surveying ships. If we are talking about 20 alloys net, then science ship price takes 5 months. 200/5=40 is the upper threshold ballpark figure of early energy focus.

So, something that's important to remember is that focus goals are supplementary to the income goals, you need to look at the base income goals as well since they determine a lot of the AI score, but also that goals are just that goals. The early game AI might not even reach 50 alloys, but having that goal higher means that they focus it early ( which is important ) but also that its weighted higher relative to the other goals. If you're lacking a lot of mineral income then the AI will build mineral production districts even if the alloys focus goal isn't fulfilled, the focus goal is important because it means that the AI will always look at alloys as something somewhat important. I also don't think that 50 alloy income is very high tbh, if you're playing wide early then not being able to build ships and starbases because of lack of alloys is gonna suck.

Since focus goals cannot be dynamically set by number of planets the food income focus is there mostly to act as a deficit buffer and make sure that the AI starts thinking about food seriously before it actually hits the negatives. But if we could set it as a multiple of planets that would be neat.

As for the minerals you need to once again look at the base income goal too, the focus goal of 30 acts a lot as a "minimum" income threshold to make sure that the alloy focus goal does not give alloys to high of a score before you have some kind base mineral income. The AI is gonna build a lot more mineral production districts than the focus goal dictates since the base income goal is 200, by far the highest income goal in the early game, giving minerals a very high relative score.

The consumer goods focus goal is much like the food one, to make sure that there is a small buffer before going into deficit. If every time the AI built a Science Lab ( or another building requiring consumer goods upkeep ) it would go into deficit then it would just be acting way to reactively. Having a small consumer goods buffer just makes it more stable. Also, yes, the AI will sell consumer goods if it wants/has to.

Lastly energy is the easiest resource for the AI to acquire outside of planet construction, it can build mining stations to get more but also sells resources to keep a pretty nice energy stockpile for such things as blockers and leaders. This means that once again that the energy focus goal is more of a buffer than anything else. During our tests the AI had much less issues with energy than anything else, especially since it produced at lot of alloys that it could sell if necessary

 

[sidestep]

Can you not relativly easy know and compare what can be produced if every job is worked - devastation and what is produced now.

Well, "relatively" is the operative word here. Finding available jobs and calculating theoretically how much resources they would produce if worked is relatively easy, the biggest problem I think is that you would also have to make sure that:
1. There are pops in your empire that could work those jobs, that they can be moved there and how fast they could be moved ( and what would happen to the planet they're moved from? )
2. That any pops you move there will actually take those jobs ( expensive calculations )
3. Pops can't be moved somewhere else that is better ( more expensive calculations )

 

[sidestep]

Just curious: do the economic plans also allow specifying how much naval cap to shoot for, and how many ships to build? The examples don't list that, but it feels like it fits in the purview of an economic "minister" responsible for ship construction.

Unfortunately no, that is already dictated by another system/minister. The idea was that it should be part of the plan, but there wasn't any time to rip the old system out and replace it
Also, a side effect of the AI economy being better is that the old ship-building system actually works better so, that's something

 

[sidestep]

How much do the ministers "talk" to each other? E.g. if the economic minister decides we need to increase mineral output by 80 but our local resources are mostly tapped out, will it ask the war minister to set its sights on that mineral-rich region the neighbors have? Will the war minister tell the economic minister "we can't take them without a 20% expansion of the fleet please build me more battleships"?

Well, this is a VERY interesting topic and could probably be an entire DD too. One of the most common ways the AI ministers communicate and coordinate in Stellaris is via the abstract concept of "strategies". In short the AI will look at it's goals and the space around it and create strategies that the different ministers can "pick up" and pursue in different ways ( you can for example find AI script that checks if you have a certain strategy and if so increase a certain budget to accommodate that ). They communicate in other ways too but AI communication is difficult because you also don't want them to be too directly interconnected, since they're modules that can be interchanged you want them to be able to work independently and without dependencies on other ministers. I'm simplifying this quite a lot here but hopefully it will give you some idea

 

[sidestep]

Why don't you do it? It seems giving different plan to different civics should be something part of the base game and that would enhance the game experience. Is it something planned for the next few patches once the plan mechanism is considered to be working well?

Haha, well, as with everything else in game development it comes down to time and prioritization. I'm not a scripter so for me to design and script a lot of great plans would take time, time that I could spend more efficiently coding and improving the base system. It's easier to add script to this at a later time so more focus was put into the code. I agree that it would be nice to have all of these things, which is why there is support for it, but at this point there just wasn't enough time. I do hope that it will be added later as the system grows and improves, but it's also very hard to know exactly where to put effort and resources before the system has been tested "live".

 

[sidestep]

Quiz question: what happens if a planet is full of slaves who can't do specialist jobs, there are 10 free specialist slots (because the former owner built those), and you have a mineral shortage. Will your AI build mining districts, or will it think there are enough jobs available, and completely cease developing the planet?

In general, while no doubt your AI is an improvement over what was there before, unfortunately I'm not as excited as everyone else. Just having the AI work towards a static number of surplus income, is too basic. To name food and minerals, the surplus you should strive towards depends entirely on how many planets you own. For empires that can use the Encourage Growth decision, that's 8.33 food surplus per planet. The minerals surplus per planet should be between 10 or 20, depending on the stage of the game and how fast your population grows. The required energy surplus is a bit more complex, as it depends on how much your empire sprawl increases your campaign costs, and whether or not you want to terraform worlds. The desired alloy surplus depends on whether you're living in a friendly or hostile corner of the galaxy. And let's not forget the new approach makes incompatible any two mods that add a new resource.

As a consequence these new AI plans will force modders to create an infinite number of permutations to account for all possible situations. They're not modder friendly! It would have been better if you could instead set the surplus goal for each resource individually. I would have envisioned something like this in 00_strategic_resources.txt

minerals = {
    tradable = yes
    market_amount = 100
    market_price = 100
    max = 15000
 
    deficit_modifier = minerals_deficit #found in static modifiers

    ai_weight = {
        weight = 1
    }

    ai_wants = {
        whatever
    }

    ai_wants_surplus = {
        base = 30
    
        modifier = {
            add = 10
            num_owned_planets >= 2
        }
        modifier = {
            add = 10
            num_owned_planets >= 3
        }
        modifier = {
            factor = 1.1
            condition_whatever = yes
        }
    }
}

Good quiz question! Honestly, I don't know. But if this is an issue I have no doubt that our QA ( or the community ) will find it and it will become a JIRA for the team to fix.
As for your concerns on the simplicity of the scriptable plans, they are valid, the system is not very advanced in that regard and is for sure not perfect. However, as with all systems in our games they tend to grown and improve over time. I had a strict deadline to work towards and so I had to make difficult decisions where to improve and expand the system. Would a more advanced plan system have made the economic AI that much better than improving something else? Hard to say, but the basic system in place is already a big improvement over the old AI that had no plans at all. Improving the plan system is one of several things that I wanted to do but just didn't have time to, but that does not mean that this is necessarily the final version of it. After seeing it "live" after release perhaps its deemed necessary to improve it further, or perhaps something else is identified as more important. I have no idea, but game development is a constant give and take and I have no doubt that the team will work to make the game even better no matter what!

 

[sidestep]

1. Does the AI scoring system multiply base building weights, or ignore them entirely until the plan is complete and the base building weights take over? If so is it multiplying just the base weight, or the base weight factored by any scripted modifiers to it?

2. One thing I dont see here is the possibility to weight defenses (where does the AI want to build fortresses for example) How does this work? Since there is no define for AI_DEFENSE_SCORE_MULT or the like, how could we teach the AI that a defensive building might be a good idea if it doesnt produce resourses that the plan is otherwise looking for?



Thank you for all this hard work, Stellaris is going to feel like a whole new game.

The scoring system does not use the base building weights, they are only used when the plan system is not used. The base weight used by the plan system is calculated entirely separate from the building weights.

Defensive buildings such as fortresses is handled by another system, that's why defenses does not have a weight here

 

[sidestep]

@sidestep

Great diary, looking forward to see the fleet management changes!

As for the economy plan, will we be able to have fine control over what buildings will be built durin the execution? For example if I for whatever reason want AI to build robot assemblies, but I do not want them to build gene clinics, will that be acomplishable (it seems that both get placed into the same pop growth category so probably will get the same weight modifiers)?




Is there any way to see the base weights for buildings? I assume that beyond the hood every plan will do the similar weight comparison the way it is doing now, I would be happy if there is a way to, for example, forbid AI from building a housing building until there are free district slots. Would that be possible? Surprised that the AI isn't simply trying to keep housing and amenities above 0, is it possible to make the AI do that? Is it possible to make it so police states will build more precincts or will I have to fallback to an old system to acomplish that?

A lot of interesting points here. First of all you are completely correct, gene clinics and assemblies are part of the same category and will get the same weight. The AI still however follows the allow = {} trigger fields when building so if you disallow gene clinics in that trigger then it won't build them. This goes the same for housing, if you script in the allow = {} trigger that housing is not allowed before all other districts are filled up the AI will follow that.

You can see the final weights in the debug tooltip but no the base unfortunately, that breakdown could be make more verbose but it just became unreadable at that point.
And the AI does indeed try to keep housing, amenities and crime above zero, that's always done and does not need to be scripted into any plan. The scoring of these things work similarly as the resource buildings, i.e. if you are missing lots of housing then that will be heavily prioritized.

 

[sidestep]

Yes you did. Thank you very much for the reply, this looks really impressive and I hope you will find time to implement planet specific plans some day. Even if it is just something for modders to implement it would be a great way to reduce micro.


Here is another question: You mentioned that you got the basic numbers for the AI plans from QA. Since you have a couple synths running Stellar is over night for testing anyway, have you considered tuning those numbers with an evolutionary algorithm?

There is no algorithm in place, but I am however logging a lot of stats and numbers so that I can back up any changes in the system with actual data. If the AI performs better or worse after a change it will be reflected in numbers with nice graphs etc

 

[sidestep]

This diary gives me hope.

But I always miss being able to automate my empire when it becomes large, the micromanagement kills the fun, so:

-Can we turn on this plan/AI for our own empires too? Or are they restricted to the computer AIs?

Right now it's AI only. But you can always turn on human_ai and just watch it run, hehe

 

[sidestep]

Does the AI take branch offices into consideration for "The Plan"?

On the topic of economic AI, I have never seen AI Megacorps build a branch office into the 4th slot.

Branch offices are handled by a separate system already, I did not have time to port it over I'm afraid.

 

[sidestep]

When it comes to the economy, would the AI try to specialize in a planet they have have or nay? It would make sense and produce more when a planet does. Or an empire-wide specialization like building pop growth rates on all planets.

Resouces are cool to focus on and all, but if they don't work to specialize, most players will outgrow them in this matter.

Humans are always gonna win over the AI in these matters, we're just smarter. The goal was never to make an AI that was "better than human", that's a fools errand unless you make the AI cheat like crazy. However, they do indirectly take into account specialized planets since they try to build things on planets where they would produce the most. So if you have a mineral rich planet that could very well turn out to be highly specialized. But you are unlikely to see the AI specialize heavily like humans do, that was outside the scope of this rework. The goal was to make the AI economy more robust resource wise, because they were just too likely to crash and burn their economy before.

 

[sidestep]

While the English language was born in Europe and the European way of using it tends to be better, I will die on this hill:

The plural of "Math" is "Math".

Haha, what can I say? I'm swedish, so I hope I get a pass ^^
( civilization should be spelled with a 'z' though, because it's a beautiful letter...no bias )

 

[sidestep]

The AI overhaul certainly sounds great.

I do wonder, are the Plans(TM) modularly moddable though? As in, mods wont overwrite each other when messing with the Plan, but can add their own targets for strategic resources and have it all work together. Instead of, in an overwrite scenario, mod A overwrites mod B, so the AI will properly target resource A but be unaware resource B exists at all and ignore it.

They are not modular, but that would be really nice. It was one of the improvements that was just never gotten to.

 

[sidestep]

Well, it was a good read. Better than most DDs by far, especially because follow-up questions and concerns are being addressed.
Also, glad you finally found some time to finish necessary debug tools and steamline some AI code (which should have been there at release, to be honest >.>) - hopefully, this will make further AI adjustments faster and easier.


One thing I did not understand though: do plans adjust for difficulty? As I understand it, when AI income is doubled, the "normal" plans stop being an effective motivation. While AIs could, theoretically, switch to the next plan, the pop requirements will, likely, keep AIs back.

Yeah, having them just switch over to "the next" plan is something I would like, but if they have a really solid economy then falling back to the old way of building until they hit the next plan isn't too bad in my experience.