Re: [libreoffice-users] ROUND function consistency

On 6/5/2022 6:54 AM, Michael H wrote:
> Are you sure it isn't somewhere else?  I.e. the input to the actual round
> step from the vb and the LO function?  LO automatically shortens numbers
> and rounds them as it does so... so a that shows 12.15 may actually contain
> 12.14999999 or 12.1500001

This whole discussion reminds me of a problem I had some 30+ years ago 
when I was the treasurer for my children's preschool. Every week, when I 
prepared the paychecks, one of them kept coming out one penny short. At 
the time, I used an old DOS spreadsheet to calculate the paycheck, and I 
had created a long complex formula to compute the paychecks.

At one point in the computation, the computer had to divide 28 by 7. Now 
every third grader knows that 28 divided by 7 equals 4, but for some 
reason, my spreadsheet formula gave a result of 3.9999999999, and that 
answer, when combined with other steps in the computation, resulted in a 
paycheck one cent short. A computer friend of mine said that my program 
was being more precise, but in my simplistic brain, I couldn't see how a 
wrong answer was more precise than a correct answer.

Oddly enough, when I switched to a Basic program on my antique Tandy 
Model 100 laptop, I got the correct answer and all was right with the 
paychecks.

Virgil



--
To unsubscribe e-mail to: users+unsubscribe@global.libreoffice.org
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/users/
Privacy Policy: https://www.documentfoundation.org/privacy