Date: prev next · Thread: first prev next last
2015 Archives by date, by thread · List index


Sounds to me like normal floating number behavior. I know someone is trying
to document this and explain why it happens (it's a technical reason that
goes beyond my skills) but this seems entirely normal behavior given the
constraints of programming languages generally and hardware in particular.


Best,
Joel

P.S. You can see similar issues in other spreadsheet software. This seems
to give some indication as to the reasons why:
https://support.microsoft.com/en-us/kb/78113 (yes I know it's MSO
link...just explaining the reason why floating number isn't entirely
accurate all the time)

On Wed, Jul 8, 2015 at 11:33 AM, Rui Pedro Caldeira <rpcaldeira@outlook.com>
wrote:

Hey and thanks for your fast answer. This is the data set:

  0.000001  0.000001  0.000001  0.000001  0.000001  0.000001  0.000001
0.000001  0.000001  0.000001  0.000001  0.000001  0.000001  0.000001
0.000002  0.000003  0.999997  0.999997  0.999997  0.999997  0.999998
1.000001  1.999997

Using the formula in the spreadsheet it gives the value: 0.328061419 (VAR)
and 0.5727664611 (STDEV)


And yes, i do mean CHISQ.INV :)

Thanks, Rui.

Cumprimentos,
Rui Pedro Caldeira

On Wed, Jul 8, 2015 at 7:24 PM, Brian Barker <b.m.barker@btinternet.com>
wrote:

At 18:44 08/07/2015 +0100, Rui Pedro Caldeira wrote:

I'm having some problems with the VAR and STDEV function, when executing
these lines of code. The average is well calculated but the remaining
two
return very low, unrealistic and equal values (6.93018471335974E-310).


That value is 2^-1027 - which is probably the smallest floating-point
value your system can represent without underflow.

Have you compared the results produced from your data by equivalent
normal
formulae in a spreadsheet cell?

Do you mean CHISQ.INV and not CHISQINV?

Brian Barker

--
To unsubscribe e-mail to: users+unsubscribe@global.libreoffice.org
Problems?
http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://listarchives.libreoffice.org/global/users/
All messages sent to this list will be publicly archived and cannot be
deleted



--
To unsubscribe e-mail to: users+unsubscribe@global.libreoffice.org
Problems?
http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://listarchives.libreoffice.org/global/users/
All messages sent to this list will be publicly archived and cannot be
deleted




-- 
*Joel Madero*
LibreOffice QA Volunteer
jmadero.dev@gmail.com

-- 
To unsubscribe e-mail to: users+unsubscribe@global.libreoffice.org
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://listarchives.libreoffice.org/global/users/
All messages sent to this list will be publicly archived and cannot be deleted

Context


Privacy Policy | Impressum (Legal Info) | Copyright information: Unless otherwise specified, all text and images on this website are licensed under the Creative Commons Attribution-Share Alike 3.0 License. This does not include the source code of LibreOffice, which is licensed under the Mozilla Public License (MPLv2). "LibreOffice" and "The Document Foundation" are registered trademarks of their corresponding registered owners or are in actual use as trademarks in one or more countries. Their respective logos and icons are also subject to international copyright laws. Use thereof is explained in our trademark policy.