Re: [libreoffice-users] Calc error?

At 18:26 28/12/2018 +0100, Harvey Nimmo wrote:
> I'm running LO on OpenSuse and I wonder if anyone else can confirm 
> this error on calc. Multiplying 2048.28*.146 yields correctly 295.54488, ...

Er, that wouldn't be correct with my maths. Do you perhaps mean 
2024.28 * 0.146 instead?

> ... but when rounding down to 2 decimal places, 295.55 is displayed 
> (instead of 295.54) (either using ROUND(2048.28*.146,2) or with field format.

That would indeed be odd. It's what you would get if you repeatedly 
rounded the value to fewer digits one by one
295.54488 -> 295.5449 -> 295.545 -> 295.55
- instead of doing it in one go. So that's not the way to do it, of course.

There is one way that you can create this result, using options at 
Tools | Options... | LibreOffice Calc | Calculate. If you have "Limit 
decimals for general number format" ticked and set to "3 Decimal 
places", your original result will be displayed - correctly - as 
295.545. If you then round *that* value to two fractional places, you 
will properly get 295.55. The situation can be complicated if you 
also have "Precision as shown" ticked, which causes subsequent 
calculations to be based on what you see in a cell instead of the 
underlying, unrounded value.

What happens if you ensure that both of these options are *not* ticked?

I trust this helps.

Brian Barker


--
To unsubscribe e-mail to: users+unsubscribe@global.libreoffice.org
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/users/
Privacy Policy: https://www.documentfoundation.org/privacy