[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [libreoffice-users] Calc error?


At 18:26 28/12/2018 +0100, Harvey Nimmo wrote:
I'm running LO on OpenSuse and I wonder if anyone else can confirm this error on calc. Multiplying 2048.28*.146 yields correctly 295.54488, ...

Er, that wouldn't be correct with my maths. Do you perhaps mean 2024.28 * 0.146 instead?

... but when rounding down to 2 decimal places, 295.55 is displayed (instead of 295.54) (either using ROUND(2048.28*.146,2) or with field format.

That would indeed be odd. It's what you would get if you repeatedly rounded the value to fewer digits one by one
295.54488 -> 295.5449 -> 295.545 -> 295.55
- instead of doing it in one go. So that's not the way to do it, of course.

There is one way that you can create this result, using options at Tools | Options... | LibreOffice Calc | Calculate. If you have "Limit decimals for general number format" ticked and set to "3 Decimal places", your original result will be displayed - correctly - as 295.545. If you then round *that* value to two fractional places, you will properly get 295.55. The situation can be complicated if you also have "Precision as shown" ticked, which causes subsequent calculations to be based on what you see in a cell instead of the underlying, unrounded value.

What happens if you ensure that both of these options are *not* ticked?

I trust this helps.

Brian Barker


--
To unsubscribe e-mail to: users+unsubscribe@global.libreoffice.org
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/users/
Privacy Policy: https://www.documentfoundation.org/privacy

Follow-Ups:
Re: [libreoffice-users] Calc error?Harvey Nimmo <harvey@nimmo.de>
References:
[libreoffice-users] Calc error?Harvey Nimmo <harvey@nimmo.de>
Privacy Policy | Impressum (Legal Info) | Copyright information: Unless otherwise specified, all text and images on this website are licensed under the Creative Commons Attribution-Share Alike 3.0 License. This does not include the source code of LibreOffice, which is licensed under the Mozilla Public License (MPLv2). "LibreOffice" and "The Document Foundation" are registered trademarks of their corresponding registered owners or are in actual use as trademarks in one or more countries. Their respective logos and icons are also subject to international copyright laws. Use thereof is explained in our trademark policy.