Re: [libreoffice-users] ROUND function consistency

At 14:44 05/06/2022 +1200, Steve Edmonds wrote:
> I have just put a basic macro together so it will calculate income 
> tax due from our local tax rates. I needed to round the tax to the 
> nearest cent so taking the easy route I added Option VBASupport 1 to 
> use the available VB round function. I then noticed that the VB 
> round function with say round(12.125,2) rounds down to 12.12 and the 
> LO inbuilt spread sheet function ROUND with ROUND(12.125,2) rounds 
> up to 12.13. Is rounding in this situation arbitrary or is there 
> some convention for consistency.

As is often the case, there are many conventions! See
https://en.wikipedia.org/wiki/Rounding#Rounding_to_the_nearest_integer .

The LibreOffice function is likely to use "round half away from zero" 
- the version we all learned in elementary school. But a quick web 
search reveals that the Visual Basic function does indeed use "round 
half to even" ("banker's rounding") - which explains the difference. 
You should find that both functions agree that round(12.135,2) is 12.14.

Note also two other matters:

o What is displayed in a spreadsheet as 12.125 may represent a 
slightly different underlying value - either more or less than this - 
with the display limited to three fractional places by cell 
formatting. The spreadsheet ROUND() function may act on the displayed 
value and give the expected answer, or it may act on the true value 
stored in the cell - which may round differently. Which of these 
happens is controlled by the option at Tools | Options | LibreOffice 
Calc | Calculate | General Calculations | Precision as shown.

o Some values which terminate as decimal fractions may not be capable 
of exact representation in the computer's binary form. This is likely 
to make a difference only when values are expressed to a significance 
near that of the underlying number storage. (And it happens anyway 
that your 12.125 *will* be capable of exact representation in binary.)

I trust this helps.

Brian Barker



--
To unsubscribe e-mail to: users+unsubscribe@global.libreoffice.org
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/users/
Privacy Policy: https://www.documentfoundation.org/privacy