Server Deployments Week 44 – Recap

As always, please refer to the server deployment thread for the latest information and updates.

On Tuesday, October 28th, CDN support was deployed across the Main channel, meaning that the entire grid now utilises the Highwinds CDN for texture and mesh fetching. As the 130 regions deployed to the Snack channel were all originally from the Main channel, they have been reabsorbed into that channel, and Snack has once again be dissolved

On Wednesday, October 29th, all three RC channels received the same server maintenance project. which includes some minor improvements, which were described by Maestro Linden at the Server Beta meeting on Thursday, October 30th, as, “just some code cleanup and some extra optional debug logging.”

SL Viewer

On Wednesday, October 29th, the Lab promoted the HTTP Pipelining RC viewer (version 3.7.19.295700) to the de facto release viewer. As I’ve noted in previous blog posts, this viewer:

Can issue multiple asset fetches on a connection without waiting for responses to earlier requests. This reduces the impact of a user’s physical location on scene loading and generally improves the experience for everyone

Includes significant improvements to inventory folder and item fetches, which can markedly decrease the time taken for inventory to load.

All other SL viewer remain as per my Current Viewer Releases page.

Issues and Problems

Mesh Loading Issues

There have been mixed reports of mesh loading issues, which may or may be linked to some long-standing issues with mesh attachments. During the Server Beta meeting on Thursday, October 30th, the issues of mesh items which should be cached by the viewer being slow to load, don’t show up at all or show up with a low LOD, requiring a relog. Interestingly, the meshes can appear correctly when using one account, but not with another, either though both accounts are drawing from the same viewer cache.

Whirly Fizzle noted that there is a potential (and equally weird) fix for odd mesh loading issues, commenting:

There’s a weird mesh loading issue (not new) that’s “fixed” by deleting a certain per account file. I don’t even know why it should even work but it does. Sometimes a certain mesh gets stuck and will not render for a certain account. Relog doesn’t fix it & neither does a cache clear. You have to delete texture_list_last.xml in the per account settings files.

Texture Load Issues

There have been a couple of reports of texture loading issues which are still under investigation.

In the first, there have been some reports that textures are getting stuck when “half loaded”, so that they remain blurred or may fail to render. It’s not clear how widespread this issue is, or whether it might be related to BUG-6382, although this is also unclear.

In the second, one content creator has reported customers are suddenly having issues with a HUD-based system for changing the colour of their shoes. When used, the textures rez very slowly, or the shoes get stuck as grey for up to 5 minutes at a time, or get stuck at a 1st level load until the wearer teleports or relogs. This is apparently a completely new behaviour with the shoes in question, and it appears to be exacerbated when using the HTTP viewer.

AMD Catalyst™ 14.9.2 Beta Woes

The latest AMD Catalyst™ 14.9.2 Beta drivers will not render rigged mesh unless hardware skinning is disabled. The current workaround is to either disable hardware skinning (which is not ideal when it comes to rendering mesh in general, and ALM features like materials will not render) or to remain on / roll-back to the 14.9.1 beta drivers if the problem is encountered. See BUG-7653. As was mentioned in the Server Beta UG meeting, this could be particularly confusing to new users running AMD GPUs and who are using the Lab’s default mesh avatars.

E-mail Communications Issues for In-world Objects

A further JIRA has been filed in respect of e-mail communications with in-world objects hanging for no readily apparent reason (see: BUG-7554). similar issues have been reported in the past (although this one has a publicly viewable report), and Caleb Linden is currently poking at things.

UK Log-in Issues

Some users in the UK experienced issues attempting to log-in following the Main channel restarts (failing to connect to a server, connections timing-out when requesting region capabilities, et.). Some fingers were pointed at the London CDN node being the cause, but as Maestro Linden pointed out:

The symptoms (failed region connects, high ping times) don’t seem to match up with the CDN at all. All the CDN is providing at this point is meshes, textures, and map tiles. If the CDN were totally offline, the worst you should see is a world of grey with no meshes loading, assuming your viewer hadn’t previously cached the assets in that area. Given that the issue seemed to be UK-specific (at least from the comments in this thread), I suspect there was regional network issue, somewhere on the path between the UK to Phoenix (where sims are currently hosted).

This issue did appear to clear itself up, and there have been no further reports of problems.

Other Items

The Future of Max Bandwidth

The Max Bandwidth setting in the viewer which controls UDP traffic between the simulator and the viewer. This is actually capped server-side at 3,000kbps, although some users have persistently set their viewer much higher (up to 10,000kbps), which has in the past led to issues for users as a result of it encouraging traffic congestion, etc. A JIRA proposal to cap the slider in the viewer to match that of the server-side limit (see SH-3149) was raised in 2012, but never acted upon.

However, as the Max Bandwidth setting is only applicable to UDP data, with the wider use of the CDN / HTTP for delivery asset data to the viewer, it will, over time, become less important a setting. Even so, given that setting the value too high can lead to problems other than with possible congestion (see VWR-29029), advice on adjusting it to a reasonable setting, such as suggested by the Firestorm team should still be heeded (make sure you test against the Phoenix, AZ link when following the instructions, as that is the Lab’s simulator DC).