Date: prev next · Thread: first prev next last
2016 Archives by date, by thread · List index


https://bugs.documentfoundation.org/show_bug.cgi?id=90419

--- Comment #5 from Piet van Oostrum <piet@vanoostrum.org> ---
(In reply to Eike Rathke from comment #1)
While generally all floating point numbers that are not exactly
representable in an IEEE 754 double lose precision, this specific case of
filling a series of numbers could be improved by using multiplication
instead of incrementing (adding delta) values.

Actually, this is already how it is done.
In ScTable::FillSeries:

        case FILL_LINEAR:
            {
                //  use multiplication instead of repeated addition
                //  to avoid accumulating rounding errors
                nVal = nStartVal;
                double nAdd = nStepValue;
                if ( !SubTotal::SafeMult( nAdd, (double) ++nIndex ) ||
                        !SubTotal::SafePlus( nVal, nAdd ) )
                    bError = true;
            }

However, depending on the initial values, this may still give rise to
"unexpected" values (that is unexpected with regard to the "intuitive"
expectations). But that's life with floating point numbers.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

Context


Privacy Policy | Impressum (Legal Info) | Copyright information: Unless otherwise specified, all text and images on this website are licensed under the Creative Commons Attribution-Share Alike 3.0 License. This does not include the source code of LibreOffice, which is licensed under the Mozilla Public License (MPLv2). "LibreOffice" and "The Document Foundation" are registered trademarks of their corresponding registered owners or are in actual use as trademarks in one or more countries. Their respective logos and icons are also subject to international copyright laws. Use thereof is explained in our trademark policy.