20 replies to this topic

[VidiVici]

Hi,

I dunno if this has been asked before. But is it possible for the AI to "save" data?

I am asking this because I want the cpu to do some machine learning.

As learning without memory is impossible I want to know this before I start putting time in this.


Thx.

Edited by VidiVici, 11 August 2007 - 09:37 AM.

[thudo]

Thanks VidIici for offering.. I'll let one of our elite AI devs answer.. Btw, what programming skills do you bring to the table?

[VidiVici]

Nice...

I am a Java Programmer and am currently doing a master in machine learning.

If the AI allows to save data it will be worthed for me to learn SCAR scripting etc...

I have been using your skirmish AI almost from the start but only after reading the interview I the saw potential (modularity) to try and add another layer to the current AI using the machine learning tools.

Hence the question if I can save some parameters from one game to the next.

Edited by VidiVici, 11 August 2007 - 01:41 PM.

[ArkhanTheBlack]

Yes, it's possible to store and load data from files. But you shouldn't do that too extensively since it can cost a lot of AI performance, like the DowPro lag issues have prooved.
Though I have my doubts if the AL self learning will be so effective. I think special AI info scripts for certain maps would be more efficient. But who knows, I don't know what your idea is so far, so it's hard to comment.

[VidiVici]

Nice, that's already a first hurdle passed, duly noted: I should avoid reading and writing files as much as possible.

This is still an information round as the AI for a game like DoW is very complex I am trying to see what is achievable. The first step for any learning to occur I need to be able to implement "select-evaluate-save" cycles over multiple games.

The concept initially is very simple.
The AI would evaluate the success of predefined strategies and save the data. In the subsequent games this data would be used to select the more successful ones more often.
That's where the reading and writing comes into play, not the toughest part but vital for learning to make sense.

Later the AI would try self-made strategies and improve on them using more complex Machine Learning techniques.

Now I can go and look more closely on how the AI ticks.
Thx.

Edited by VidiVici, 11 August 2007 - 10:29 PM.

[runab0ut]

The concept initially is very simple.
The AI would evaluate the success of predefined strategies and save the data. In the subsequent games this data would be used to select the more successful ones more often.
That's where the reading and writing comes into play, not the toughest part but vital for learning to make sense.

Later the AI would try self-made strategies and improve on them using more complex Machine Learning techniques.

Just throwing an idea out: would saving a few selected points in a map be feasible for use of the learning ai you're working on?

Say mapping the map for AI to get data and process it. For example... a certain chokepoint in the map causes more infantry deaths with/or turret spam. The AI would then selectively not march thru that area until he builds a counter.

Or even select a certain strategic point or relic point to capture first (as a priority) or an area of a map to avoid.

[VidiVici]

I am just starting to try and figure out what can be done I am a long way from being able to implement something like that.

What you raise is also not really "learning" per se. The cpu can figure this out during the course of a single game.

What I want to do is have the cpu getting better at beating you by figuring out your weaknesses, being less predictable and try stuff not implemented from the start.

[thudo]

Be careful - Relic might abduct you for having the best AI ever developed for their forthcoming DoW2 or another THQ project. What you are attempting only paid devs will do in many years to come as AI development is NOT in the forefront of tech like eye-candy and physics are these days.

Someday, however.. someday Skynet will have its DAY!! <shakes hand to sky>

Until then.. the work you, Arkhan, Larkin, myself, and others try to do to elevate the importance of good AI will be the precursors of our next kind of opponents.

[troubadour]

Well i am just back from holidays and a new beta version is done and we talk about AI learning wooo !!!

I think - as Arkhan already suggested - that saving a file per map about what build order AI used and if they succeeded or not could be a 1st step before 'learning'
This file could be generated at the end of a skirmish game without a big impact on performance at least no lag issue
A way forward could be improving information stored in these files such as SP, CP or Relic point to cap first or to avoid

Just my 2 euros cents - i guess you have your own plans.. just see it as a brainstorming session

[runab0ut]

I am just starting to try and figure out what can be done I am a long way from being able to implement something like that.

What you raise is also not really "learning" per se. The cpu can figure this out during the course of a single game.

What I want to do is have the cpu getting better at beating you by figuring out your weaknesses, being less predictable and try stuff not implemented from the start.

Not really 'learning', but yeah... it could be done by hand. Maybe the AI team can get an idea running with this.

Just a flat file that the ai could read and know the map positions or something beforehand and then base its decisions/strategies to what every routines it goes when in a game.

Players might mark a certain point in the file as "this point is critical in this map" then either the AI will strive to take it quickly or protect it as its marked as 'priority' as the game goes.

[VidiVici]

Hi again, as you have noticed I kinda dumped the project these past weeks as I didn't really feel like putting my time in a potentially dead game.

Now with the announcement of SS I am ready to pick it up again.
I have the Corsix tools and have taken an glance at the AI code, sleek and well structured btw.

I have found where I will start to try and implement the learning: BuildBaseStrategy spending rates.
Correct me if I'm wrong but they seem to be currently static and race dependent and set by an educated guess.

	-- Limit tactics
	-- NOTE: These #'s are horrifically dependent on balance changes, we should
	-- scan the costs of all structures / add-ons / researches that let us tier up 
	-- and figure out a min from that
	self.spend_now_req = 400
	self.spend_later_req = 600
	self.spend_now_pow = 75
	self.spend_later_pow = 150

As I don't really understand the full flow of the AI so I would like some advice:
1) Where do you think the revaluation of the spending pattern would be most efficient?
2) What functions are called at tier change and end of game?

The idea is to let the AI to learn the most efficient spending pattern though Evolutionary computation.
Later changes in environmental factors will trigger different spending patterns which in tern will be optimized according to the specific needs.
To have a spending pattern depending on the tier, the income, the army size, etc...

PS:
Does it make sense to tweak this?
Where is it used?
On what base were these parameters assigned?

Edited by VidiVici, 30 September 2007 - 12:49 PM.

[thudo]

I would concentrate more on the AI learning the ThreatOnPath system so it doesn't send units or builders down a dangerous path on a map but instead goes another route to surprise the enemy.

Having the AI spend more wisely really isn't the issue - its the pathing, PERIOD.

All RTSes have AI's where the "logical pathing" is honestly not logical and the AI loves to needlessly suicides itself. Can this be addressed? Thats Faaaaaaaaaaaaaaaarrrrrrrrrrrrrrr more critical to RTSes then spending.

[VidiVici]

Heh, address a big issue from the start might be a bit ambitious
I am looking for a simple project to familiarize myself with AI modding.

But I'll take a look at ThreadOnPath.

PS: Where is ThreadOnPath?

Edited by VidiVici, 30 September 2007 - 03:22 PM.

[thudo]

Its broken.. Relic has long since been asked to fix it. You can find the calls in the cpu_manager.ai but essentially it crashes the game.

So essentially these are the 2 MAIN ISSUES:

1) ThreatOnPath
2) Path Distance

I've re-starting dialogue with Relic to include these major code fixes for the 3rd expansion in March'08 which honestly should give them more than enough time to resolve it especially since we have FLYERS now in the game.

[ArkhanTheBlack]

self.spend_now_req = 400
self.spend_later_req = 600
self.spend_now_pow = 75
self.spend_later_pow = 150

Those values don't exist in our mod. Are you using the original AI files from Relic?

Anyway, the resource spending rate in our mod is preciseley calculated dependent on the current build item in the build programs. The algorithm for that is superior to any AI learning approach since the perfect result would be exactly this algorithm (Actually it's not that difficult...).

AI learning is better used for something we can't just calculate with an elegant formula. For example, the attack and defending behaviour of the opponent would be very interesting. If you can find a way to store map dependent data, you could store the position where a human player usually attacks first. This could be used to predict a good defending position.

[VidiVici]

@Arkhan:

Indeed it seems those values are from the original AI. My bad....
I'm still quite new at this can tell me how I can get the map name in game?
If I can't get that info I won't be able to try and save that.

@thudo
Correct me if I'm wrong but it looked like the function used to check for enemy at a certain distance of the airPath.

I have done a first tweak for TheatOnPath:
It is now looking if the enemy is in a pie-segment [ <) ] along the path.


Edit Moved to a new topic

Edited by VidiVici, 02 October 2007 - 12:41 PM.

[troubadour]

@Thud & Vidivici
Count me in if you need beta tester.

[VidiVici]

Thanks to offer but the tweaks are minor atm and don't realy need a lot of beta testing

Edited by VidiVici, 02 October 2007 - 12:38 PM.

[RobertW7928]

oh adaptive ai would be the next step to gaming experience oh and it should be great to u guys doing anything to improve or add to ai mod?

Edited by RobertW7928, 26 November 2007 - 10:34 AM.

[thudo]

Nothing new is being added to the existing DoW Advanced AI v2.50 *UNTIL SOULSTORM* so we are currently in "feature freeze" until the 3rd expansion.