alex sato schrieb:
I have a problem with functions I made.
Let's say I create the function:
Function Test (n)
Test = r
Then, in a empty spreadsheet take the following steps:
-Insert the formula =TEST(E1)
-insert 11,18 in E1 and see the result in cell D1
I think the result should be 18 but I'm getting 17
Where is the error?
It is the normal behavior of data type double. 11,18 has indefinite
digits in dual system. So the stored value is cut somewhere. Because
there are enough binary digits to round the dual number to 15 decimal
digits, you do not notice it. But when you subtract the integral part,
the others digits will shift left and new digits will be generated at
the end, which does not represent the original value. The calculated
difference becomes a little bit to small. After multiply with 100 you do
not have 18,00 but 17,99... And that results in 17. The problem becomes
more visible, when the integral part is larger.
You should not use Basic functions, but use Calc functions instead. They
are often tweaked to avoid such problems. You can call Calc functions
inside your Macro.
To get the first two decimal digits as integer value, it seems to be
more accurate to use
r(3)=(n * 100) mod 100
in your macro.
For unsubscribe instructions e-mail to: firstname.lastname@example.org
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://listarchives.libreoffice.org/global/users/
All messages sent to this list will be publicly archived and cannot be deleted
Impressum (Legal Info)
: Unless otherwise specified, all text and images
on this website are licensed under the
Creative Commons Attribution-Share Alike 3.0 License
This does not include the source code of LibreOffice, which is
licensed under the Mozilla Public License (MPLv2
"LibreOffice" and "The Document Foundation" are
registered trademarks of their corresponding registered owners or are
in actual use as trademarks in one or more countries. Their respective
logos and icons are also subject to international copyright laws. Use
thereof is explained in our trademark policy