Introduction

We have been developing a large project in ASP.NET 3.5 along with Microsoft AJAX for the last few months and when we started testing in the test environment, we came across a large number of ScriptResource.axd entries making the HTTP calls to download the JavaScript to make it available for the page. Here we had about 80/90 lines keeping the HTTP handler busy, so in this case if we had 100 concurrent users, we would have had about 800/900 concurrent calls to HTTP. In order to optimize the performance/scalability and reducing the HTTP calls, we found out a way to get those calls minimized and thought of sharing with the community who could have the same performance issue.

The Solution

The solution we present here combines multiple JavaScript file declarations in your HTML into a single declaration, meaning our 80/90 JavaScript file references are combined into 1.

Script Profiler

There is a very cool utility contained in this project which gets you a list of all scripts being used on the page. So, before running the project, just set enableProfiler to true and you will get the list of scripts on the page. Those scripts must be placed under optimizerSection tag, so it will help ScriptManager to find the script and combine it.

< optimizerSection enable =" true" enableScriptCompression =" true" enableHtmlCompression =" true" enableScriptMinification =" true" enableHtmlMinification =" true" enableProfiler =" false" > < add key =" 1" name =" MicrosoftAjax.js" assembly =" System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" path =" " / > < add key =" 2" name =" MicrosoftAjaxWebForms.js" assembly =" System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" path =" " / > < add key =" 3" name =" " assembly =" " path =" ~/Scripts/Script01.js" / > < add key =" 4" name =" " assembly =" " path =" ~/Scripts/Script02.js" / > < add key =" 5" name =" " assembly =" " path =" ~/Scripts/Script03.js" / > < add key =" 6" name =" AspNetPerformanceOptimizer.Controls.Control01Client.js" assembly =" AspNetPerformanceOptimizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" path =" " / > < add key =" 7" name =" AspNetPerformanceOptimizer.Controls.Control02Client.js" assembly =" AspNetPerformanceOptimizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" path =" " / > < add key =" 8" name =" AspNetPerformanceOptimizer.Controls.Control03Client.js" assembly =" AspNetPerformanceOptimizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" path =" " / > < add key =" 9" name =" AspNetPerformanceOptimizer.Controls.Control04Client.js" assembly =" AspNetPerformanceOptimizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" path =" " / > < add key =" 10" name =" AspNetPerformanceOptimizer.Controls.Control05Client.js" assembly =" AspNetPerformanceOptimizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" path =" " / > < add key =" 11" name =" AspNetPerformanceOptimizer.Controls.Control06Client.js" assembly =" AspNetPerformanceOptimizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" path =" " / > < add key =" 12" name =" AspNetPerformanceOptimizer.Controls.Control07Client.js" assembly =" AspNetPerformanceOptimizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" path =" " / > < add key =" 13" name =" AspNetPerformanceOptimizer.Controls.Control08Client.js" assembly =" AspNetPerformanceOptimizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" path =" " / > < add key =" 14" name =" AspNetPerformanceOptimizer.Controls.Control09Client.js" assembly =" AspNetPerformanceOptimizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" path =" " / > < add key =" 15" name =" AspNetPerformanceOptimizer.Controls.Control10Client.js" assembly =" AspNetPerformanceOptimizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" path =" " / > < add key =" 16" name =" AspNetPerformanceOptimizer.Controls.Control11Client.js" assembly =" AspNetPerformanceOptimizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" path =" " / > < add key =" 17" name =" AspNetPerformanceOptimizer.Controls.Control12Client.js" assembly =" AspNetPerformanceOptimizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" path =" " / > < add key =" 18" name =" AspNetPerformanceOptimizer.Controls.Control13Client.js" assembly =" AspNetPerformanceOptimizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" path =" " / > < add key =" 19" name =" AspNetPerformanceOptimizer.Controls.Control14Client.js" assembly =" AspNetPerformanceOptimizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" path =" " / > < add key =" 20" name =" AspNetPerformanceOptimizer.Controls.Control15Client.js" assembly =" AspNetPerformanceOptimizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" path =" " / > < add key =" 21" name =" AspNetPerformanceOptimizer.Controls.Control16Client.js" assembly =" AspNetPerformanceOptimizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" path =" " / > < add key =" 22" name =" AspNetPerformanceOptimizer.Controls.Control17Client.js" assembly =" AspNetPerformanceOptimizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" path =" " / > < add key =" 23" name =" AspNetPerformanceOptimizer.Controls.Control18Client.js" assembly =" AspNetPerformanceOptimizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" path =" " / > < add key =" 24" name =" AspNetPerformanceOptimizer.Controls.Control19Client.js" assembly =" AspNetPerformanceOptimizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" path =" " / > < /optimizerSection >

Configuration

We have a file called ScriptCombinerSection.cs which loads the necessary configuration information from the web.config file and makes it available using a static class called OptimizerConfig .

public class OptimizerConfig { protected static Dictionary<string, /> _scripts; protected static bool _enable; protected static bool _enableProfiler; protected static bool _enableScriptCompression; protected static bool _enableHtmlCompression; protected static bool _enableScriptMinification; protected static bool _enableHtmlMinification; static OptimizerConfig() { _scripts = new Dictionary<string, />(); OptimizerSection sec = null ; try { sec = (OptimizerSection) System.Configuration.ConfigurationManager.GetSection " optimizerSection" ); foreach (ScriptElement i in sec.Scripts) { _scripts.Add(i.Key, i); } _enable = sec.Enable; _enableProfiler = sec.EnableProfiler; _enableScriptCompression = sec.EnableScriptCompression; _enableHtmlCompression = sec.EnableHtmlCompression; _enableScriptMinification = sec.EnableScriptMinification; _enableHtmlMinification = sec.EnableHtmlMinification; } catch { } } public static ScriptElement GetScriptByKey( string key) { ScriptElement objElement = null ; try { objElement = _scripts[key]; } catch { } return objElement; } public static ScriptElement GetScriptByResource( string name, string assembly ) { ScriptElement objElement = null ; foreach (KeyValuePair<string, /> element in _scripts) { if (element.Value.Name == name && element.Value.Assembly == assembly ) { objElement = element.Value; break ; } } return objElement; } public static ScriptElement GetScriptByPath( string path) { ScriptElement objElement = null ; foreach (KeyValuePair<string, /> element in _scripts) { if (element.Value.Path == path) { objElement = element.Value; break ; } } return objElement; } public static ScriptElement GetScriptByName( string name) { ScriptElement objElement = null ; foreach (KeyValuePair<string, /> element in _scripts) { if (element.Value.Name == name) { objElement = element.Value; break ; } } return objElement; } public static bool Enable { get { return _enable; } } public static bool EnableProfiler { get { return _enableProfiler; } } public static bool EnableScriptCompression { get { return _enableScriptCompression; } } public static bool EnableHtmlCompression { get { return _enableHtmlCompression; } } public static bool EnableScriptMinification { get { return _enableScriptMinification; } } public static bool EnableHtmlMinification { get { return _enableHtmlMinification; } } }

Optimization Types

We can divide the overall performance optimization into five different pieces:

Script Combiner: Combines all scriptresource.axd calls into a single call.

Combines all scriptresource.axd calls into a single call. Script Compressor: Compresses all client side scripts based on the browser capability including gzip/deflate.

Compresses all client side scripts based on the browser capability including gzip/deflate. Script Minifier: Removes comments, indentations, and line breaks.

Removes comments, indentations, and line breaks. HTML Compressor: Compress all HTML markup based on the browser capability including gzip/deflate.

Compress all HTML markup based on the browser capability including gzip/deflate. HTML Minification: Writes complete HTML into a single line and minifies it at possible level (under construction).

Script Combiner and Compressor

As mentioned earlier that the basic purpose of developing the script combiner was to minimize the HTTP calls and get the complete client side script in one shot. In order to get it working, we will need to override the ScriptManager class and three of its methods. One of the main methods which must be overridden is OnResolveScriptReference . Whenever each script gets resolved, we get it here and replace it with the script information provided by the web.config. If we enable script profile, the Render method writes the list of profiled scripts on the browser.

public class OptimizeScriptManager : ScriptManager { private const string HANDLER_PATH = " ~/ClientScriptCombiner.aspx?keys=" ; private const string BLOCKED_HANDLER_PATH = HANDLER_PATH + " -1" ; private Dictionary<string, /> _scripts = new Dictionary<string, />(); private List<ScriptReference> _profilerScripts = null ; protected override void OnInit(EventArgs e) { base .OnInit(e); if (OptimizerConfig.EnableProfiler) _profilerScripts = new List<ScriptReference>(); } protected override void OnResolveScriptReference(ScriptReferenceEventArgs e) { try { base .OnResolveScriptReference(e); #region Profiling scripts if (OptimizerConfig.EnableProfiler) { bool isFound = false ; foreach (ScriptReference reference in _profilerScripts) { if (reference.Assembly == e.Script.Assembly && reference.Name == e.Script.Name && reference.Path == e.Script.Path) { isFound = true ; break ; } } if (!isFound) { ScriptReference objScrRef = new ScriptReference(e.Script.Name, e.Script.Assembly); if (!string.IsNullOrEmpty(e.Script.Name) && string .IsNullOrEmpty(e.Script.Assembly)) { objScrRef.Assembly = " System.Web.Extensions, Version=3.5.0.0," + " Culture=neutral, PublicKeyToken=31bf3856ad364e35" ; } objScrRef.Path = e.Script.Path; objScrRef.IgnoreScriptPath = e.Script.IgnoreScriptPath; objScrRef.NotifyScriptLoaded = e.Script.NotifyScriptLoaded; objScrRef.ResourceUICultures = e.Script.ResourceUICultures; objScrRef.ScriptMode = e.Script.ScriptMode; _profilerScripts.Add(objScrRef); objScrRef = null ; } } #endregion #region Combining Client Scripts bool isAssemblyBased = ((e.Script.Assembly.Length > 0 ) ? true : false ); bool isPathBased = ((e.Script.Path.Length > 0 ) ? true : false ); bool isNameBased = ((e.Script.Path.Length == 0 && e.Script.Assembly.Length == 0 && e.Script.Name.Length > 0 ) ? true : false ); if (OptimizerConfig.Enable && (isAssemblyBased || isPathBased || isNameBased)) { ScriptElement element = null ; try { if (isAssemblyBased) element = OptimizerConfig.GetScriptByResource(e.Script.Name, e.Script.Assembly); else if (isPathBased) { element = OptimizerConfig.GetScriptByPath(e.Script.Path); if ( null != element) { if (!OptimizerHelper.IsValidExtension(element, " .js" )) { element = null ; } else if (!OptimizerHelper.IsAbsolutePathExists(element)) { string absolutePath = OptimizerHelper.GetAbsolutePath(element); element = null ; } } } else if (isNameBased) element = OptimizerConfig.GetScriptByName(e.Script.Name); } catch (Exception exc) { element = null ; } if (element != null ) { if (!_scripts.ContainsKey(element.Key)) { try { _scripts.Add(element.Key, e.Script); e.Script.Assembly = string .Empty; e.Script.Name = string .Empty; StringBuilder objStrBuilder = new StringBuilder(); objStrBuilder.Append(HANDLER_PATH); foreach (KeyValuePair<string, /> script in _scripts) { objStrBuilder.Append(script.Key + " ." ); } string strPath = objStrBuilder.ToString(); objStrBuilder = null ; foreach (KeyValuePair<string, /> script in _scripts) { script.Value.Path = strPath; } } catch { } } else { e.Script.Assembly = string .Empty; e.Script.Name = string .Empty; e.Script.Path = BLOCKED_HANDLER_PATH; } } } #endregion } catch (Exception ex) { this .Page.Response.Write(ex.ToString().Replace( "

" , " <br>" )); } } protected override void Render(HtmlTextWriter writer) { try { #region Writing profiled scripts on the browser if (OptimizerConfig.EnableProfiler && _profilerScripts != null ) { StringBuilder builder = new StringBuilder(); int index = 1 ; foreach (ScriptReference script in _profilerScripts) { builder.Append( " <add key=\"" + index++ + " \" name=\"" + script.Name + " \" assembly=\"" + script.Assembly + " \" path=\"" + script.Path + " \" /><br>" ); } writer.WriteLine( " <pre>" ); writer.WriteLine(builder.ToString()); writer.WriteLine( " </pre>" ); builder = null ; } #endregion } catch (Exception ex) { this .Page.Response.Write(ex.ToString().Replace( "

" , " <br>" )); } finally { base .Render(writer); } } }

Once all scripts have been resolved, it will build the URL with all keys which are required by the page:

< script src =" ClientScriptCombiner.aspx?keys= 1.2.3.4.5.6.7.8.9.10.11.12.13.14.15.16.17.18.19.20.21.22.23.24." type =" text/javascript" > < / script >

Here keys parameter in query string represents the script numbers separated by dot (.). These script numbers are specified in web.config, so while resolving the script, it picks the number against the matching string and builds the URL. Once the URL has been built and written to the browser, the handler will get called and pick each number to extract the client script from the assemblies or files. The StringBuilder is used to collect the stream of scripts and write it on the browser. When the handler gets called and finishes combining all scripts, it determines the capability of the browser, either it supports the compression or not. If it does, e.g. gzip, it will create the instance of the GZipStream class and compress the scripts, otherwise it writes as is.

public void ProcessRequest(HttpContext context) { bool shouldProcessRequest = true ; string [] scriptKeys = null ; string keys = context.Server.UrlDecode(context.Request.Params[ " keys" ]); string scriptResourcePath = String .Empty; ScriptManager objScriptManager = new ScriptManager(); StringBuilder scriptBuilder = new StringBuilder(); IHttpHandler handler = new ScriptResourceHandler(); if ( String .IsNullOrEmpty(keys) || keys.Equals( " -1" )) shouldProcessRequest = false ; if (shouldProcessRequest) { scriptKeys = keys.Split( ' .' ); scriptResourcePath = string .Format( " {0}{1}{2}{3}{4}{5}{6}{7}" , context.Request.Url.Scheme, " ://" , context.Request.Url.Host, " :" , context.Request.Url.Port, " /" , context.Request.ApplicationPath, " /ScriptResource.axd" ); foreach ( string key in scriptKeys) { ScriptElement element = OptimizerConfig.GetScriptByKey(key); if (element == null ) continue ; #region Generating resource URL dynamically and creating WebRequest object to extract stream of script if (element != null ) { ScriptReference reference = null ; bool isPathBased = false ; if (element.Path.Length > 0 ) { reference = new ScriptReference(element.Path); isPathBased = true ; } else if (element.Assembly.Length > 0 && element.Name.Length > 0 ) { reference = new ScriptReference(element.Name, element.Assembly); } try { OptimizeScriptReference openReference = new OptimizeScriptReference(reference); string url = string .Empty; if (!isPathBased) { url = context.Request.Url.OriginalString.Replace (context.Request.RawUrl, " " ) + openReference.GetUrl(objScriptManager); var queryStringIndex = url.IndexOf( ' ?' ); var queryString = url.Substring(queryStringIndex + 1 ); var request = new HttpRequest( " scriptresource.axd" , scriptResourcePath, queryString); using (StringWriter textWriter = new StringWriter(scriptBuilder)) { HttpResponse response = new HttpResponse(textWriter); HttpContext ctx = new HttpContext(request, response); handler.ProcessRequest(ctx); } } else { string absolutePath = OptimizerHelper.GetAbsolutePath(element); if (OptimizerHelper.IsAbsolutePathExists(absolutePath)) { using (StreamReader objJsReader = new StreamReader(absolutePath, true )) { scriptBuilder.Append(objJsReader.ReadToEnd()); } } } scriptBuilder.AppendLine(); } catch (Exception ex) { } } #endregion } } objScriptManager = null ; #region Writing combine output scripts to the Response.OutputStream context.Response.Clear(); context.Response.ContentType = " application/x-javascript" ; try { SetResponseCache(context.Response); scriptBuilder.AppendLine(); string combinedScripts = scriptBuilder.ToString(); if (shouldProcessRequest) { if (OptimizerConfig.EnableScriptMinification) { combinedScripts = JsMinifier.GetMinifiedCode(combinedScripts); } string encodingTypes = string .Empty; string compressionType = " none" ; if (OptimizerConfig.EnableScriptCompression) { encodingTypes = context.Request.Headers[ " Accept-Encoding" ]; if (!string.IsNullOrEmpty(encodingTypes)) { encodingTypes = encodingTypes.ToLower(); if (context.Request.Browser.Browser == " IE" ) { if (context.Request.Browser.MajorVersion < 6 ) compressionType = " none" ; else if (context.Request.Browser.MajorVersion == 6 && !string.IsNullOrEmpty(context.Request.ServerVariables [ " HTTP_USER_AGENT" ]) && context.Request.ServerVariables [ " HTTP_USER_AGENT" ].Contains( " EV1" )) compressionType = " none" ; } if ((encodingTypes.Contains( " gzip" ) || encodingTypes.Contains( " x-gzip" ) || encodingTypes.Contains( " *" ))) compressionType = " gzip" ; else if (encodingTypes.Contains( " deflate" )) compressionType = " deflate" ; } } else { compressionType = " none" ; } if (compressionType == " gzip" ) { using (MemoryStream stream = new MemoryStream()) { using (StreamWriter writer = new StreamWriter( new GZipStream(stream, CompressionMode.Compress), Encoding.UTF8)) { writer.Write(combinedScripts); } byte [] buffer = stream.ToArray(); context.Response.AddHeader( " Content-encoding" , " gzip" ); context.Response.OutputStream.Write(buffer, 0 , buffer.Length); } } else if (compressionType == " deflate" ) { using (MemoryStream stream = new MemoryStream()) { using (StreamWriter writer = new StreamWriter ( new DeflateStream(stream, CompressionMode.Compress), Encoding.UTF8)) { writer.Write(combinedScripts); } byte [] buffer = stream.ToArray(); context.Response.AddHeader( " Content-encoding" , " deflate" ); context.Response.OutputStream.Write(buffer, 0 , buffer.Length); } } else { context.Response.AddHeader( " Content-Length" , combinedScripts.Length.ToString()); context.Response.Write(combinedScripts); } } scriptBuilder = null ; } catch (Exception ex) { context.Response.Write(ex.ToString().Replace( "

" , " <br>" )); } #endregion }

Script Minifier

We are using jsmin i.e. courtesy of Douglas Crockford.

HTML Compressor, HTML Minifier

In order to compress and minify the HTML markups, we have come up with the new streaming class called HtmlCompressStream inheriting from Stream .

public class HtmlCompressStream : Stream { public enum CompressionType { None = 0 , GZip = 1 , Deflate = 2 }; private Stream _stream; public HtmlCompressStream (Stream stream, CompressionMode mode, CompressionType type) { switch (type) { case CompressionType.GZip: _stream = new GZipStream(stream, mode); break ; case CompressionType.Deflate: _stream = new DeflateStream(stream, mode); break ; default : _stream = new StreamWriter(stream).BaseStream; break ; } } public Stream BaseStream { get { return _stream; } } public override bool CanRead { get { return _stream.CanRead; } } public override bool CanSeek { get { return _stream.CanSeek; } } public override bool CanWrite { get { return _stream.CanWrite; } } public override long Length { get { return _stream.Length; } } public override long Position { get { return _stream.Position; } set { _stream.Position = value ; } } public override IAsyncResult BeginRead( byte [] array, int offset, int count, AsyncCallback asyncCallback, object asyncState) { return _stream.BeginRead(array, offset, count, asyncCallback, asyncState); } public override IAsyncResult BeginWrite( byte [] array, int offset, int count, AsyncCallback asyncCallback, object asyncState) { return _stream.BeginWrite(array, offset, count, asyncCallback, asyncCallback); } protected override void Dispose( bool disposing) { _stream.Dispose(); } public override int EndRead(IAsyncResult asyncResult) { return _stream.EndRead(asyncResult); } public override void EndWrite(IAsyncResult asyncResult) { _stream.EndWrite(asyncResult); } public override void Flush() { _stream.Flush(); } public override int Read( byte [] array, int offset, int count) { return _stream.Read(array, offset, count); } public override long Seek( long offset, SeekOrigin origin) { return _stream.Seek(offset, origin); } public override void SetLength( long value ) { _stream.SetLength( value ); } public override void Write( byte [] array, int offset, int count) { if (OptimizerConfig.EnableHtmlMinification) { _stream.Write(array, offset, count); } else { _stream.Write(array, offset, count); } } }

Based on the browser’s capability, we create the compress stream object and let the writer write the stuff. Writing this class has a couple of advantages, one is to enable compression and the second is to minify HTML markup at the time of writing. So, when the writer wants to emit stream on the browser, it invokes the Write method which writes the stream of bytes to the browser. We assign the instance of this class to the Response.Filter property at the time of page’s request, i.e. written in the Application_BeginRequest event of Global.asax.

public class Global : System.Web.HttpApplication { protected void Application_BeginRequest( object sender, EventArgs e) { HttpRequest request = this .Request; HttpResponse response = this .Response; if (request.RawUrl.IndexOf( " .aspx" ) > -1 && string .IsNullOrEmpty(request.Params[ " keys" ])) { if (OptimizerConfig.EnableHtmlCompression && !(request.Browser.IsBrowser( " IE" ) && request.Browser.MajorVersion < = 6 )) { string acceptEncoding = request.Headers[ " Accept-Encoding" ]; if (!string.IsNullOrEmpty(acceptEncoding)) { acceptEncoding = acceptEncoding.ToLower(CultureInfo.InvariantCulture); if (acceptEncoding.Contains( " gzip" )) { new GZipStream(response.Filter, CompressionMode.Compress); response.Filter = new HtmlCompressStream(response.Filter, CompressionMode.Compress, HtmlCompressStream.CompressionType.GZip); response.AddHeader( " Content-encoding" , " gzip" ); } else if (acceptEncoding.Contains( " deflate" )) { response.Filter = new HtmlCompressStream(response.Filter, CompressionMode.Compress, HtmlCompressStream.CompressionType.Deflate); response.AddHeader( " Content-encoding" , " deflate" ); } } } else { response.Filter = new HtmlCompressStream(response.Filter, CompressionMode.Compress, HtmlCompressStream.CompressionType.None); } } } }

Note: The HTML minification is still under construction and would be done very soon. We’re looking forward to having a great HTML minifier.

Using the Sample Project

There are some Ajax client controls and *.js files created to run and test the sample. You will need to rebuild the project and then access default.aspx. If you want to test it with AjaxControlToolkit, download the latest version of control toolkit and reference it in the project. Drag and drop some controls from the Toolbox, enable the profiler, and run the project. It will write the list of scripts on the page; just copy those scripts and put it in the web.config file under optimizerSection. That’s all we need to do to run the project.

Useful Results

BEFORE (1.55 seconds for 238kb)

AFTER (62 milliseconds for 75kb)

Conclusion

We always find some kind of trade-off between processing and network latency, this article is entirely focusing on network latency; not on the processing cost because it has been assumed that this implementation is done on high end processing servers. Finally, if you're using ASP.NET with AJAX client controls, make sure your website is providing best performance at all levels of scalability.