Re: tdf#50916 : Calc : Dynamic column container

On 14/10/15 02:08, Kohei Yoshida wrote:
> Also, this will be an on-going process.  This is not going to be like
> "if you do A, B and C it's okay to increase the column size and no
> problems will occur".  Rather, we'll probably encounter lots of
> performance issues that we'll have to spend some time fixing after the
> column size is increased.

Just throwing an idea into the mix...

Howsabout a function that, on being given the cell co-ordinates, returns
a pointer to the cell. Force everything through that.

Okay, that's an extra level of indirection, that on a large spreadsheet
could cost a lot. But is it worth it in terms of memory saved? The
important point here is that you can change the in-memory storage of
your spreadsheet to match its characteristics.

If you have a small spreadsheet, you simply have a two dimensional array
of cells.

If you have say 100 cells with stuff in, in a spreadsheet that might be
1000x1000, you can have a lookup table that lists all the columns with
data, and within that all the rows that have data ...

I can't think of any more examples particularly. But if you have that
abstraction layer, you can optimise for different characteristics of
different spreadsheets.

(I'm coming at it from my database background, where Pick stores
everything as strings, and it's amazing how fast it actually finds stuff
even despite having to search sequentially through what is possibly a
long string, and how convenient it is to be able to use "proper" arrays
and "dynamic" arrays (delimited strings) together to optimise speed.)

As I say, this is just an idea, it might be a no-no, but if it can
optimise large sheets while not costing small sheets much, it could well
be worth it.

Cheers,
Wol