Date: prev next · Thread: first prev next last
2014 Archives by date, by thread · List index

At 22:46 12/08/2014 +0200, Skiper Skippers wrote:
Check =2^53 it shows 9007199254740990

This is not an error!

You are using a finite computing machine - one in which each number in a cell is stored in a memory location of finite length. So there is always a limit to how large an integer can be stored and to how great a precision a floating-point value can be stored. Your computer is probably using "double-precision binary floating-point format", in which numbers are stored in eight-byte (sixty-four-bit) words. This allows around fifty-three significant binary digits, the rest being taken up by the sign and exponent. This level of precision corresponds to between fifteen and sixteen significant decimal digits, so integers can be stored precisely only up to this length of number, above which values can be stored, but only to around fifteen significant digits.

2^53 is one followed by fifty-three zeroes, of course - so is just too long to be stored accurately. You are very unlikely to need numbers of this magnitude as final results, and if you come across this problem, you can usually find a simple way to reorganise your calculation to avoid the problem.

Note that the (biased) exponent is stored in an eleven-bit field, so a separate problem occurs if you try to calculate with numbers above about 2^1023.

For a fuller explanation, see .

I trust this helps.

Brian Barker

To unsubscribe e-mail to:
Posting guidelines + more:
List archive:
All messages sent to this list will be publicly archived and cannot be deleted


Privacy Policy | Impressum (Legal Info) | Copyright information: Unless otherwise specified, all text and images on this website are licensed under the Creative Commons Attribution-Share Alike 3.0 License. This does not include the source code of LibreOffice, which is licensed under the Mozilla Public License (MPLv2). "LibreOffice" and "The Document Foundation" are registered trademarks of their corresponding registered owners or are in actual use as trademarks in one or more countries. Their respective logos and icons are also subject to international copyright laws. Use thereof is explained in our trademark policy.