Project : Increasing the limit of number of columns from 1024 to 16384 in Calc

Motivation

Often we need the ability to load sheets from MS Excel/csv files that have more than 1024 columns in it. MS Excel can effortlessly open such large column files, but in Calc we are limited to loading files with 1024 columns.

Basics on the related code

The very basic classes involved are :

ScDocument - which represents a document. See its declaration here ScTable - which represents a sheet within a document, declared here ScColumn - represents a single column within a sheet, declared here.

Challenges

Before the work on 16k cols project started, the columns of a sheet were allocated statically as a plain array of ScColumn 's of size size MAXCOLCOUNT inside ScTable called aCol . We could have just altered the definition of MAXCOLCOUNT from 1024 to 16384, but it has the following two major issues :

Since we preallocate MAXCOLCOUNT number of columns at the beginning, the memory requirement for a single sheet will increase a lot, independent of how much columns the user is actually going to use. Most of the code in methods of ScTable and ScDocument has the following code-pattern in them : for ( SCCOL nCol = 0 ; nCol <= MAXCOL; ++nCol ) { } where MAXCOL is defined as MAXCOLCOUNT - 1 . If we increase the MAXCOLCOUNT statically, it would cause these loops to take longer even though the user may use only some of the first few columns. So the change affects even the users who just want to work with documents with few columns.

The obvious way to mitigate the above problems is to make the column storage in ScTable dynamic, such that a column is allocated if it is actually required by the user.

What have been done so far ?

What to do next ?