It’s time for the November update to our performance improvements roadmap. Before giving the update, we wanted to shed some more light on the recent issues related to the new Activity Finder so that you all have a better understanding of why these issues are happening. Note that completely replacing the old Activity Finder was something we had to do. It was not able to be easily updated and could not handle the load we see on the live servers.



Before launching the new Activity Finder, the team tried to put the system to as much internal load testing as possible by using automated bots. Even though bots cannot truly simulate player behavior, they are a good tool for identifying basic load problems, and we were able to get significantly more testing bots into the system than players on the live servers and still have the system run smoothly. This gave us confidence to push it to PTS for Update 24. On PTS, new problems were discovered and fixed, but we never saw player concurrencies on PTS as compared to Live. Once the system hit Live, well, we all know that what happened wasn’t good.



The core issue we’ve been having with the new Activity Finder is that it shares many of the same system resources as the rest of the game. This means that when the Activity Finder gets overloaded, everything else on the server starts to bog down – which is why players experience long loading times, timeouts/disconnects, and sluggish server response at these times. It’s been a challenge for us to identify and fix these issues as each time we go through a cycle of overload, we fix that particular problem, but then discover a new one as new concurrency levels are hit. It is very important to note that almost all of the server issues encountered over the last month were caused by the Activity Finder – it is not a hardware issue or a more widespread problem. The reason why this seems to happen the most on the PC EU server is because it’s our most populated server where people generally all play during the same core hours.



So, when the most recent Undaunted event started, we started to see server performance issues, even though we had already found and fixed Activity Finder problems during the Witches Festival event. We found and fixed issue after issue during the Undaunted event, but new ones kept cropping up. So we postponed the event and are taking our time now to ensure that we resolve this problem once and for all. We are testing a new build internally, and we fully expect to have this issue resolved.



I, along with everyone else, appreciate the irony of stating just last year that we wouldn’t have another in-game event focused on the Activity Finder until we were “100% sure” it would work. With the new Activity Finder, we were a lot more confident than with the old one. Obviously, we were wrong.



Even though it is a small consolation, I do want to make sure everyone understands that the issues you have been seeing are caused by one system going haywire and not a deep systemic or hardware problem. This Activity Finder issue has been plaguing us for over a year now and it is long past time that we fix it.



We all appreciate everyone’s patience on this, and hopefully we are approaching the end of this process. We haven’t exactly covered ourselves in glory--we know it, and we are doing everything we can to make it better.



Creative Director - The Elder Scrolls Online

Facebook | Twitter | Google+ | Tumblr | Pinterest | YouTube Rich LambertCreative Director -