Re: [libreoffice-users] Re: Avery 8167 label printing

Virgil Arrington wrote:
> Ken Springer wrote
> > I remember years ago when Intel turned out a chip that had an error in
> > it's math calculations.  It was a rare happening, but when they
> > finally admitted it publicly, trying to say it wasn't important do to
> > the rare occurrence, it did not go over well at all!  <G>
>
> About 25 years ago, I was the treasurer of my children's preschool. I
> created a spreadsheet to calculate paychecks, and I found that the
> paycheck was consistently off by .01 (a penny). It drove me nuts. As it
> turned out, one part of the calculation required the division of 28 by
> 7, which every third grader knows is 4. Well, my spreadsheet gave an
> answer of 3.9999999999_. By itself, it wasn't a big problem, but later
> in the chain of operations, the 3.99999_ produced a result that rounded
> *down* to the nearest penny instead of *up*, which it would have done if
> the 28/7 had resulted in 4 instead of 3.9999. I complained to a computer
> friend of mine who tried to explain that the computer's answer was more
> "precise" than my mental math of 28/7=4. I didn't buy it.

I wouldn't say 3.9999999... is more precise, but it sounds like your 
problem is related to the precision of floating-point numbers. 1/7, when 
represented in binary, is a recurring fraction 0.001001001... (like how 
1/3 is 0.3333... in decimal), so cannot be represented precisely in 
binary with a finite number of bits (just as you can add as many '3's as 
you like to 0.3333, but it still doesn't exactly represent 1/3).

I don't think there should be a problem with calculating 28/7 = 4 as a 
single floating point operation, but if the calculation was done (either 
in the way your formula was expressed or the way the application 
processed it) as (1/7)*28, that may well give a slightly inaccurate 
result due to rounding of the (1/7). A bit like calculating 1/3 to 
0.333, then multiplying that by 12 to get 3.996 instead of 4. If using a 
round-down function, and the result was slightly less than 4, it would 
round down to 3. I guess the solution was to round to the nearest integer.

> I learned a valuable lesson in blindly accepting a computer's
> calculation simply because it was made by a computer.

Indeed. They do exactly what they're told by a combination of hardware, 
software and user input - but for various reasons that might not amount 
to what you thought you were telling it to do.

Mark.


--
To unsubscribe e-mail to: users+unsubscribe@global.libreoffice.org
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://listarchives.libreoffice.org/global/users/
All messages sent to this list will be publicly archived and cannot be deleted