Some complain OpenOffice.org is slow and bloated. With each release there may be dozens of performance improvements, but there are also new features, some of which may slow things down. This the natural balance in software development, but in the end, what is the net effect on performance from one version to the next?

The benchmark

We need a good benchmark to produce good data, but what do we measure? Let's assume the most common operations are starting OpenOffice.org, opening a Writer document, scrolling from top to bottom using the down arrow, exporting the document, and closing both the document and OpenOffice.org.

This benchmark automatically performs these 5 operations while taking precise measurements. The tests are repeated because of inevitable noise in the results. The benchmark performs 10 passes where each pass begins with a reboot to test cold start performance. Cold starts are slower because hard drive information is not in cache memory. Each pass consists of 10 iterations, of which most iterations measure warm start performance. Warm starts are faster because the operating system has cached the hard drive information in memory. You may notice OpenOffice.org is faster the second time you open it. In this benchmark, testing each version of OpenOffice.org yields 500 measurements. With 11 versions, that is 5500 measurements.

The specific Writer document tested is the ODF_text_reference_v1_1.odt. All tests are done against unmodified vanilla builds from the OpenOffice.org web site.

Test environment

The modest test machine is about three years old. Of course, newer machines should perform better.

Linux distribution: Fedora 7

Kernel: Linux 2.6.23.15-80.fc7

Java: Sun 1.6.0_03

CPU: AMD Athlon XP 3000+

RAM: 768 MB, DDR 333 (PC 2700)

HDD: Maxtor 6Y080L0, 7200 RPM, 80 GB

Filesystem: ext3

Default settings in OpenOffice.org (no optimizations)

The results

OpenOffice.org startup performance is critical to the overall perception that the application is quick. The startup process involves many steps including loading the application into memory, resolving the linking of symbols, initializing fonts, reading configuration files, initializing the Java runtime, checking printers, checking for software updates, and importing any documents. At different points the startup process may stress the hard disk, CPU, memory, and network interface.

A 2003 document called OpenOffice.org "Q" Product Concept described these goals for OpenOffice.org 2.0 performance:

SO/OOo will improve its performance in four areas that are especially important to customers. The two most visible areas of improvement will be decreasing the Startup Time and Document Open/Save Time.

With startup time, they certainly succeeded as you can see in this boxplot depicting application startup times after a cold start, meaning directly after a reboot of the operating system without the benefit of caching. The single jump from OpenOffice.org version 1.1.5 to 2.0.0 slashed application startup time a dramatic 43%. (In all charts on this page, smaller is better.)

There are two competing laws at work: Moore's Law and Wirth's Law. Moore's Law basically states, You can cheaply buy a doubly fast CPU in two years. In contrast, Wirth's Law states that each new release of a software program grows more complex and slower to cancel out the effect of the faster hardware. In other words, today's dual-core CPUs run Vista about as fast as an ancient 80286 (20 years ago) running MS-DOS 3.1. It's unrealistic for software to be faster over time: newer software does more work. When a newer version performs more quickly, it generally means the previous version was inefficient. The goal is to delicately balance performance, new features, and development resources.

In the 1.75 years since the fast version 2.0.3 release, startup time has crept up a modest 0.70 seconds to version 2.4—. That still leaves a 20% improvement over 2.6 years from version 1.1.5 to 2.4. So much for Wirth's Law: it's more of a guideline than a law anyway.

D300m3 is short for DEV300_m3, an early OpenOffice.org 3.0 developer's snapshot— even older than the OpenOffice.org 3.0 beta. Performance may vary in the final 3.0 release five months later. A newer 3.0 snapshot could not be tested because of bug 88221 in newer OpenOffice.org 3.0 snapshots available at the time of testing.

A warm application startup is defined as starting OpenOffice.org twice in a row, so the hard disk information is cached in RAM for quick access. On Windows, the Quickstarter feature accomplishes roughly this goal. Above the boxplot shows the upgrade from 1.1.5 to 2.0.0 yielded an impressive 50% gain, cutting warm startup time in half. Also impressive, changes in performance after 2.0.0 are relatively minor for either people or the benchmark to distinguish. The latest stable release version 2.4 weighs among the best scores.

The large gap between warm startup time and cold startup time ranges from 70% to 81% of the total. This implies three-quarters of the cost of starting OpenOffice.org is bottlenecked at the hard drive, but as an exception to Moore's Law, hard drive performance increases very slowly. If OpenOffice.org wants to cut cold startup, it must cut hard drive access and not wait for hard drive performance to increase. On the other hand, solid state drives (SSDs), such as those found in the ASUS Eee PC subnotebook, perform better than traditional hard drives and are slowly becoming mainstream.

The performance of opening a document follows a different trend than application startup. From version 1.1.5 to 2.0.0, the opposite happened: the newer version was much slower. The difference was 3.46 seconds (70%) in changes made over 2.5 years.

Cold and warm document open times resemble each other, but the cold times are twice as long. Again, the gap is caused by hard drive performance.

Scrolling is less important performance metric. Nevertheless, the chart's shape is interesting. To my surprise, OpenOffice.org 3.0 DEV300_m3 breaks the trend and improves performance over version 2.4. I would have expected new features such as notes-in-the-margin feature to slow 3.0 down.

The benchmark exports (or "saves") to the version's native text document format, PDF, and Microsoft Word 97/2000/XP (.doc). The native format for 1.1.5 is .sxw. The native format of OpenOffice.org versions 2.0 and later is the OpenDocument Format (ODF).

Versions 2.4 and 3.0 show noticeably longer times. In version 2.4, the increase may be the new PDF features. For version 3.0, the increase may be the new OpenDocument Format version 1.2.

Times for closing the document and application are all quick. It's hard to beat a fraction of a second.

The cumulative times (adding up all the individual tests) for each version show a gradual upward trend. For cold start times, version 2.4 is 38% slower than 1.1.5 over a span of 2.5 years.

Similarly for warm start times, version 2.4 is 40% slower than 1.1.5.

In conclusion, OpenOffice.org is generally getting slower with each release. However, startup performance has made great improvements, the performance losses are relatively small, advances in new computer hardware are more than making up the loses, and OpenOffice.org continues to mature with new features. OpenOffice.org doesn't compel users to upgrade, so you are welcome to continue using older versions.

"But my OpenOffice.org is still slow?

There is no perfect benchmark, and a number of factors influence performance. For example, your machine may benefit from cleanup or from more memory. You may suffering from the rare, specific cases such as a huge number of bookmarks.

Subscribe to the site news feed. In this series of articles, we'll test how to really improve OpenOffice.org performance.

Related articles