Date: prev next · Thread: first prev next last
2016 Archives by date, by thread · List index


Hi Eike,


But when I try 3^31 (which is less than 2^52) and subtract 3^31-1, 3^31-2, 3^31-3 (resp. 1, 2 and 3 
less than 3^31) I don't get 1, 2 and 3 as result, but 0, 0 and 3.
This test is simply in Calc with e.g. =(3^31)-((3^31)-1).
That's due to approxSub() being used for operator '-', which lessens the
available precision. approxEqual() needs to be reworked to not scale in
such cases.

I'll investigate.

OK, that explains the remaining problem with my MOD() fix (https://gerrit.libreoffice.org/29365) too.


Winfried

Context


Privacy Policy | Impressum (Legal Info) | Copyright information: Unless otherwise specified, all text and images on this website are licensed under the Creative Commons Attribution-Share Alike 3.0 License. This does not include the source code of LibreOffice, which is licensed under the Mozilla Public License (MPLv2). "LibreOffice" and "The Document Foundation" are registered trademarks of their corresponding registered owners or are in actual use as trademarks in one or more countries. Their respective logos and icons are also subject to international copyright laws. Use thereof is explained in our trademark policy.