Re: [libreoffice-users] VAR, STDEV, CHISQ.INV

Sounds to me like normal floating number behavior. I know someone is trying
to document this and explain why it happens (it's a technical reason that
goes beyond my skills) but this seems entirely normal behavior given the
constraints of programming languages generally and hardware in particular.


Best,
Joel

P.S. You can see similar issues in other spreadsheet software. This seems
to give some indication as to the reasons why:
https://support.microsoft.com/en-us/kb/78113 (yes I know it's MSO
link...just explaining the reason why floating number isn't entirely
accurate all the time)

On Wed, Jul 8, 2015 at 11:33 AM, Rui Pedro Caldeira <rpcaldeira@outlook.com>
wrote:

> Hey and thanks for your fast answer. This is the data set:
>
>   0.000001  0.000001  0.000001  0.000001  0.000001  0.000001  0.000001
> 0.000001  0.000001  0.000001  0.000001  0.000001  0.000001  0.000001
> 0.000002  0.000003  0.999997  0.999997  0.999997  0.999997  0.999998
> 1.000001  1.999997
>
> Using the formula in the spreadsheet it gives the value: 0.328061419 (VAR)
> and 0.5727664611 (STDEV)
>
>
> And yes, i do mean CHISQ.INV :)
>
> Thanks, Rui.
>
> Cumprimentos,
> Rui Pedro Caldeira
>
> On Wed, Jul 8, 2015 at 7:24 PM, Brian Barker <b.m.barker@btinternet.com>
> wrote:
>
> > At 18:44 08/07/2015 +0100, Rui Pedro Caldeira wrote:
> >
> > > I'm having some problems with the VAR and STDEV function, when executing
> > > these lines of code. The average is well calculated but the remaining
> two
> > > return very low, unrealistic and equal values (6.93018471335974E-310).
> >
> > That value is 2^-1027 - which is probably the smallest floating-point
> > value your system can represent without underflow.
> >
> > Have you compared the results produced from your data by equivalent
> normal
> > formulae in a spreadsheet cell?
> >
> > Do you mean CHISQ.INV and not CHISQINV?
> >
> > Brian Barker
> >
> > --
> > To unsubscribe e-mail to: users+unsubscribe@global.libreoffice.org
> > Problems?
> > http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
> > Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
> > List archive: http://listarchives.libreoffice.org/global/users/
> > All messages sent to this list will be publicly archived and cannot be
> > deleted
>
> --
> To unsubscribe e-mail to: users+unsubscribe@global.libreoffice.org
> Problems?
> http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
> Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
> List archive: http://listarchives.libreoffice.org/global/users/
> All messages sent to this list will be publicly archived and cannot be
> deleted



-- 
*Joel Madero*
LibreOffice QA Volunteer
jmadero.dev@gmail.com

-- 
To unsubscribe e-mail to: users+unsubscribe@global.libreoffice.org
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://listarchives.libreoffice.org/global/users/
All messages sent to this list will be publicly archived and cannot be deleted