Re: tests sc_array_functions_test fail due to floating point precision

Hi Milton,


> > > I've also opened the other test documents in
> > > sc/qa/unit/data/functions/array/fods/ using LibreOffice. The documents which
> > > fail are linest.fods and also logest.fods. The others succeed.
> > If your build completed ignoring these tests (i.e. produced a
> > usable office under ./instdir/), could you please open the
> > failing documents and tell us the cell addresses on the second
> > sheet where tests fail and the actual values they produced?
> Below are the numbers. Some remarks: Many errors are in the
> range of 1E-11 to 1E-12 but some are way larger >1E30. I assume
> this is due some division of a very small error.
>
> Has someone done a/some theoretical error analysis of the
> algorithms involved? (e.g. numeric (forward/backward) stability
> and error estimates similar to what is well-known for LU
> factorization and other algorithms.)
>
> Who/What provided the reference values -- or reformulated --
> what error do these numbers have (wrt to the mathematical
> solution).
Looking at the numbers it would be better to use ROUNDSIG instead of ROUND.
ROUND(1.2345E30,4) has no effect, ROUNDSIG(1.2345E30,4) rounds to 4 
significant digits, i.e. 1.234E30.
ROUNDSIG was only added to Calc last February, so most unit test 
documents don't use it (yet).

Winfried