Neverwinter Nights Forum News

Remove all ads!

(Originally posted by chevalier)

Here are today's Neverwinter Nights forum highlights. Please take into account that these are only single parts of various threads and should not be taken out of context. Bear in mind also that the posts presented here are copied as-is, and that any bad spelling and grammar does not get corrected on our end.

Georg Zoeller, Designer

Data storage:

Quote: For variable storage NWNX is also an option. It wouldn't be too hard to write a NWNX plugin to store variables using a balanced binary tree or a hashtable and serialize this data when the DLL is loaded/unloaded. Sure, it means encoding everything to a string, but this is still far more efficient than dealing with the builtin BWDB. I've done this as part of a rather large extension to NWN for our PW and it works quite well.

NWNX is a nice tool for storing data that is not constantly required - i.e. tracking stats (kills, death, house ownership, guild affiliations) - especially the latter ones are required very seldom and there is no need to store them constantly on an object if you can just load them on demand from the database (mysql prefered, ms access odbc is slow). I'd suggest wrapping all Set/Get variable calls into a seperate function for your module so you can run profiling on how often variables are accessed - and decide which ones have to be moved into the database and which not. Variables on items are especially dangerous for huge modules since there will be a lot of items around after a while (I'd noticed this with HCR a lot). In a huge module, you shouldn't do things like tracking the burntime of a torch on the torch itself, I'd try to use a different approach like torches with charges that need to be activated (i.e. create a set of matches item, remove the item property light from the torch by default and add it as temporary item property when the matches are used on the torch).

NWN problems:

Quote: Georg..i was not saying that the allegory you posted was at all inflamitory. I was reading another topic right before this one, where the dev got a little too defensive. I was just asking that we *all* make an effort to be more courteous...that being said, I thank you for your postings..it is a breath of fresh air to get some meaningful responses from the devs. I realize that you guys are probably busy with stuff, but every once in a while, throw us dogs some cookies!

The Ugly
The recent bad atmosphere on this board threads was mainly caused by a single person who agitated his friends/players into coming here and bump the threads he created with the sole intention of offending people and creating flame wars. That problem has been permanently fixed and any other attempt to disrupt our community will be dealt with swiftly.

Please understand this:

Some of the problems posted by this person on this board were valid and had not been reported yet, others were known and already in our database. However the person did not send a single valid bug report (or included to many swear words and got it deleted by the filter) but instead posted hostile and inflammatory topics on this boards - this is not the way to get the things that are troubleing you fixed (since the topics were written in a very hostile language, nobody here cared about verifying their validity). The people here at BioWare are very passionate about their work and we care about our products (you will find very few other companies out there that supports their products over such a long time). We are however only humans, and when confronted with open hostility, childish behavior or straight insults our commitment to care for a persons problems will be impacted. Frankly I couldn't care less for the problems reported by someone who calls me a "dirty wh*re or c*** s****er" on his website. Now on to the cookies (taste varies by perception)

I) CPU 30% on empty server -
Yes, in our database, priority rated as low due to the fact that gameplay is not affected in any way, after all no player is on the server.

There seems to be some misconceptions about the importance of CPU load here:

The server has a default cpu load of 99%. It sleep()s every few frames in order to allow other applications to execute, effectivly dropping the cpu load to the number you see when calling top. Probably we could call sleep() more often when nobody is on the server in order to drop the cpu load generated by the server, but as said, this is low priority as it does not do anything to improve gameplay. CPU load 30% on an empty server does not mean that the game is wasting 30% CPU on doing nothing. Again, I agree that it would be nice if the server would degrade CPU load down to 10% when nobody is playing in order to allow other applications to execute more frequently.

II) CPU load increasing after 30's player - The issue is known and has been put on hold, reason below. The issue itself looks like the game is exceeding some limits in it's memory management or caching systems, probably causing it be less efficient and more cpu expensive with it's tasks. Since setup required to reproduce this problem is time and resource intensive, work on it has been put on hold until the below managed changes to the game's resource management systems have been finished.

III) High CPU load in general
First - 90% CPU load is not a problem per se - it becomes a problem when game performance is significantly decreased at the same time. There are a couple of problems we are aware of that cause high CPU load, especially with a very high number of players. A couple of them seem to be related to certain intel chipsets and could not be reproduced reliably here to fix them. Other reasons have been mentioned earlier in this thread (placeables and pathfinding, heartbeats, etc). The game's network messaging protocol is highly compressed (NWN has a very low bandwith requirement compared to other games), however this is taxing to a servers CPU with a lot of players, especially if some those players have mediocre or bad connections.

A couple of potential problems have been fixed internally, some can't be fixed (i.e. CPU not powerful enough to support 64 players) the others are on hold, reason again below.

The Good
The Live Team is currently investigating a sigificant update to the games underlying resource management and caching system. Preliminary tests have shown a tremendous increase in game performance (around 40% frame rate for some areas) and they expect a significant performance increase for the dedicated server, especially for machines with a lot of RAM (which could be able to keep most of the game resources in memory all the time). Currently there are a couple of unresolved issues with this change - if they manage to get those ironed out in time, this update could be available with patch 1.63, if not, the change will have to wait for an upcoming patch. If this change goes through, there is a decent chance that it might solve many of the issues mentioned in this forum, so investigation on those issues has been put on halt for a while.

Finally I want to add this:

NWN is not designed to run 100+ megabytes modules with several hundred areas in them, everyone doing this is asking for trouble. We disregard and delete any complaints to nwbugs@bioware.com that report performance problems with those setups unless they can provide instructions on how to reproduce a problem on a much smaller scope.

We have made clear from the start that running persistent worlds with this size, is not the scope NWN was created for and, while we are not artifically create hurdles for people who want to try this, we can not concentrate our resources on fixing problems arising from those setups. A module with several hundred areas and hundreds of megabyte size is operating far beyond NWNs specifications, the overhead for object management and event handling alone will be literally eating up your processing power and memory, even before a single creature has run their AI. Even with the most careful design and a scaled down creature AI, the game will never reach the performance some people expect it to have. As mentioned before, there is a reason why we distribute our official campaign in several smaller modules instead of making one huge one - NWN can't handle it. People running these setups can somethimes experience a 50-60% or more cpu load on an empty server or higher loads with a lot of player.

Please do not misunderstand the above statement as an attack on the server admin community in general, it is not - many people here run servers in a smaller scope and are experiencing problems with the multiplayer aspect of the game - and those are valid bugs. An example would be lag with 20 or 40 players on a sufficiently powerful server with enough bandwith. That said, we are interested to know what issues pose the greatest problems for both the regular server admin community and the PW community. Maybe it would be nice if I could get a list the top 10 problems that are affecting you the most (I know, everyone has their own opinion, but we should be able to get some kind of trend), the reason why they are problematic and if possible, steps to reproduce them. This can also include feature requests (like the extended banlist). While I can not promise anything, I can at least make sure that the list gets on the Desk of the Live Team producer

Further:

Quote: ... How many variables is too many? And is it relative to the area, object or module-wide? I use them a LOT. (and I mean, A LOT!)

That, unfortunately heavily depends on your machine, the module size, your available memory and number of players as well as the objects those variables are attached to and the data type of those variables. If you don't experience any problems, you probably don't need to worry, but there are a couple of ways to approach this. You could create a couple of scripts to test the limits if you feel like it.

I.e. create a script that creates a number of new variables of a given type on any player in your module, maybe even every creature and the module itself. Then start benchmarking your module (preferably with players available) by slowly increasing the number of variables until you experience noticeable performance decrease. The timegraph 2 console command is very helpful to track disk activity and other problems. A large number of variables can pose multiple problems:

a) memory creep that can lead to increased disk swapping

b) management overhead - the time required to access variables can increase when there is a significant number of variables stored on an obect

c) object size - any object in the game that gets variables attached will grow in size - it will take more time to load it from the disk (increased module load time) and cost more to hold it in memory

The previous discussion here was around point b, but a and c are valid concerns as well - but my experience is that other things (lots of heartbeat scripts, screwed up pathfinding, delaycommand chains, events like OnHitCastSpell and giant conversations with a million starting conditions) are far more common reasons for people's performance problems.

If you want discuss ways to increase your modules performance, feel free to open a new thread, I'm going to participate with whatever input I can offer on the topic.

Further:

a) Memory leaks - I've seen a couple of fix messages regarding memory leaks from Craig, so we are definitly making some progress here. It's pretty hard to catch those pesky things, especially if your lines of code are measured in millions, so I won't promise that there will be no memory leaks anymore - but it's definitly getting better

b) Bandwith - No the the changes the Live Team is looking at would not affect the compression on the game's protocol or the general bandwith usage, expect it to remain around the same it is now. As far as I understand it those changes deal with more efficient resource managment and are, among other things, significantly reducing the number of disk operations in NWN.

If you are experiencing combat lag, there are a couple of other things you can look at

a) Scripted weapons and armor - the OnHitCastSpell property is nice, but it's also very expensive. A fighter using his OnHitCastSpell - Fireball axe to execute a whirlwind attack against a large number of enemies wearing OnHitCastFireball armor can easily bring your server to a screeching halt . Caching those scripts can help, but not significantly

b) Players pathfind too, so keep placeables off the seams especially in combat heavy ares

c) Remove all unneccessary event from creatures that are not supposed to be in combat (i.e. OnPerception).

d) Some DMs have reported that there has been a lot of lag lately due to people trying to exploit a bug that is related to fast switching of certain items in order to get more actions - this can cause a significant amount of events and network messages to be triggered and will degrade the performance for everyone on the server. If you think this might be the case, a DelayCommand(0.0,ClearAllActions(TRUE) in the OnEquip script, limited to PCs only will cure this until we release a fix for the problem (through it will make everyone who changes an item in combat flatfooted, might be realistic, but not necessarily nice - limiting this behavior to people DMs find suspicious and flag with a variable might be more sensible).

Further:

One more thing - we are very excited about the changes that the Live Team is evaluating to speed up the engine, but please do not get your hopes too high about them in the next patch - we will only deploy them if all the open issues we currently have with that have been taken care of, it's a fairly major change and we don't want to rush anything - and I don't want to get crucified by the community for "promising the solution to all problems with the next patch" (to quote a PM).

Patch 1.63: Patch 1.63 is at least 3 weeks off - at least means that it can be anything upwards from three weeks, it does not mean 4, 5 or 6 weeks, it means just more than three (excuse my sarcastic explainations here, there's good reason for them, unfortunately). There is a chance for a beta soon (soon as in "before the official patch comes out"). There will very likely be an extended beta period, as the patch a lot of changes in it, some of them in very scary places in the code where usually nobody dares to look at. Patch 1.63 will be a great addition to NWN. So, please, wait with "where's the patch" threads for at least another month

XP: Uh, the scripting formula used for monster XP awards is quite complicated, and through based on the DND 3E system, there are a lot of factors playing into it (party size, henchmen, level of followers, etc) - While it might theoretically be possible to recreate the formula used in the engine, calculation OnDeath would be very inefficient and probably cause performance problems on your machine. There are a couple of alternate XP systems on the Vault, you might want to poke around those a bit.