Re: tests sc_array_functions_test fail due to floating point precision

Stephan Bergmann <sbergman -AT- redhat.com>
Thu, 7 Dec 2017 10:13:16 +0100

On 12/02/2017 08:47 AM, Milton Vandersloot wrote:

functions_test.cxx:43:Assertion
Test name: ArrayFunctionsTest::testArrayFormulasFODS
double equality assertion failed
- Expected: 1
- Actual  : 0
- Delta   : 1e-14


The issue seems to be (again) floating point precision. Indeed, I've compiled LibreOffice with FMA 
(fused multiply-add) instructions, if I disable them, i.e. passing -ffp-contract=off to GCC, then 
the tests succeed. Note that FMA has slightly different floating point precision than ordinary plus 
and mult [0, 1].

What is ultimately missing, I think, is a spec what the acceptableresults of calculations in Calc are, and tests written so that theycheck whether or not the calculated results are in the acceptable ranges.

Context

tests sc_array_functions_test fail due to floating point precision · Milton Vandersloot
- Re: tests sc_array_functions_test fail due to floating point precision · Eike Rathke
  - Re: tests sc_array_functions_test fail due to floating point precision · Milton Vandersloot
- Re: tests sc_array_functions_test fail due to floating point precision · Stephan Bergmann
- Re: tests sc_array_functions_test fail due to floating point precision · Eike Rathke
  - Re: tests sc_array_functions_test fail due to floating point precision · Milton Vandersloot
    - Re: tests sc_array_functions_test fail due to floating point precision · Winfried Donkers

Privacy Policy | Impressum (Legal Info) | Copyright information: Unless otherwise specified, all text and images on this website are licensed under the Creative Commons Attribution-Share Alike 3.0 License. This does not include the source code of LibreOffice, which is licensed under the Mozilla Public License (MPLv2). "LibreOffice" and "The Document Foundation" are registered trademarks of their corresponding registered owners or are in actual use as trademarks in one or more countries. Their respective logos and icons are also subject to international copyright laws. Use thereof is explained in our trademark policy.