9 replies to this topic

[Corsix]

So, I said that I wasn't happy with the LUA load times. I've been trying various vectors of attack, and I've found one that works very well. I've done two benchmarks of the old LUA code and the new LUA code:
Loading of the EBPS for the Eldar warp gen
Old system: 8 seconds
New system: <2 seconds

Burn to RGD all of the SBPS folder
Old system: 176 seconds (2 min 56 secs)
New system: 41 seconds

This means that the new code takes just a quarter of the time of the old code, and hopefully I can make it even faster

[Finaldeath]

Out of programming passion, and interest, what optimisations have you been toying with?

Also; I will find the time to test the application more soon.

[Corsix]

The problem is finding a fast implementation of Inherit, Reference and InheritMeta. The initial implemention was:

Load the requested file from H/D
Make a copy of the GameData/MetaData table and return it

The problem here is the time it took to load a LUA file from disk/SGA and process it. The next implementation was:

Check to see if the file is in the cache
If not then:
	load it from H/D and add to the cache
end
Make a copy of the GameData/MetaData table and return it

Thus the I/O bottleneck was fixed. Now the problem was doing a copy of a table from one LUA file to another. The implementation that was in beta 0.3:

Check to see if the file is in the cache
If not then:
	load it from H/D and add to the cache
	Take a copy of the GameData and MetaData tables and put them in faster to access structures
end
Quickly return a copy of the GameData/MetaData table

This improved times a little bit, but it was hardly noticable. The implementation I've just done is:

Check to see if the file is in the cache
If not then:
	load it from H/D and add to the cache
end
Return a special object

This special object is the key to the speed, as it avoids the entire table copy alltogether. Previously, every time you did a Reference/Inherit/InheritMeta, the entire table would be copied, which took time. This new object holds a pointer to the original table and works like so:

OnItemGet
	Get item from table
	If this item is a table then
		Return a special object to access it
	end
	return item

OnItemSet
	Make a one level deep copy of the table and set the pointer to this one
	Set the item into this new table

This object means that entire tables are no longer copied, but you can still set things in tables without affecting the original.

Thats the evolution in a very small nutshell.

Edited by Corsix, 11 May 2006 - 07:12 PM.

[thudo]

Damn! 1/4 the time?!?!?! Lets all pray both 1.5 and DC will hold to the same coding/script structure so these can be maintained! Outstanding work there Corsix!

[Finaldeath]

Um, thud, I'm pretty damn sure they won't just uproot the entire games structure for one expansion pack.

And corsix, nice overview, glad you got to know about pointers - although I don't currently code in anything but PHP lately, I remember using C lists and pointers, wee, fun

[Corsix]

Thud: DoW uses RGD and not LUA. I'm pretty sure they use something like this already as my original code was slow in comparison.

FD: I wasn't originally aware that LUA supported what I wanted to do; LUA is a stragne language at the C API level

[Corsix]

Hmm; I've been attacking the batch conversion times, and I just got 4 seconds for the entire SBPS folder. (Something seems wrong; It shouldn't be that fast...)

Edit: Yeah, something was wrong. It works now though: SBPS folder: 6 seconds. DoW:WA entire attrib folder - 59 seconds.

Edited by Corsix, 13 May 2006 - 05:40 PM.

[Finaldeath]

Can it be set to load "in steps"? If it is going to be a half-decent AE replacement, or at least, manual replacement, it'd be great to have certain parts delayed opening, since 6 seconds for one part is pretty damn good

we'll see, I still need to beta test it!

[Corsix]

The burn times are referring to when you right click on a folder or LUA and select "burn all LUAs in this folder" or "Burn this lua". Application load times remain unchanged.

[Corsix]

I've put up this improved version as beta 0.3.1 by the way, so check the beta download page.