Server Deployments Week 18- Recap

On Tuesday April 29th, the Main channel received the server maintenance project that was on the Magnum RC in week 17, comprising a fix for and a crash mode fix.

On Wednesday April 30th, the Magnum RC had the server-side Sunshine / AIS v3 code re-enabled (this code requires the use of the Sunshine RC viewer), and all three RCs were updated with the bug fixes deployed to the Main channel.

SL Viewer

There have been no updates to the RC viewers in the release channel during week 18 and no further releases, either RC or project viewers, so the SL viewer releases remain as per the last update to my Current Viewer Releases page.

Group Chat Optimisation

The code Simon Linden has been working on to improve group chat was deployed to a single group chat server, where it has been running for all groups starting with group_id “b”. Commenting on the work at the Server Beta meeting on Thursday May 1st, Maestro Linden said:

Simon’s been looking at the performance of that group chat server, and it seems to be running fine. So there are plans to update the rest of the group chat servers to the new version early next week. We won’t go so far as to say that group chat has been totally fixed, though – Simon has identified some other changes which could improve performance further.

A recent fix was made to IM sessions to correct the issue where it is possible to see “typing…” in an IM window when the other person isn’t actually typing (see STORM-1975), and questions were raised on whether this fix might be adding a load to group chat sessions, as the viewer-side code appeared to send the message during group chat sessions as well as person-to-person IMs. However, both Simon and Maestro Linden indicated that the notifications are simply ignored by the chat servers during a group chat session, so no additional load is created, although Maestro admitted it would be nice if the viewer didn’t send meaningless messages.

Aside from the back-end load, the biggest issue which occurs in group chat is when someone using it changes regions. When this happens, the chat service has to figure out where you are.

“The region you are on, your viewer and the back-end database all know where you are, and keep updated very fast. The chat servers, however, aren’t kept in perfect sync because that would be very hard to do with 50000+ people moving around who are all in 42 groups,” Simon said, in explaining the problem. He added, “It [the chat service] doesn’t track missing messages … but if it can’t send one to your agent, it then has to ask where you really are and then it sends there,” all of which takes time, delaying the receipt of group chat messages.

In describing the changes made, Simon concluded:

The new code we have out now is a bit more efficient, but more importantly it has more metrics and it showed me that the performance problems are in a few other areas … it turns out the updates to keep the list of people in the group chat updated are really significant. It gets worse, of course, in large groups – more people coming and going, and more people who need the updates.

Group Ban Lists

It appears a server-side deployment of the code require to manage the new group ban list functionality is drawing close. There is still work to be done viewer-side, but recent testing on Aditi resulted in a number of JIRAs being filed, and the associated server bugs have been stomped on by Baker Linden, with help from the likes of Caleb Linden.