[libreoffice-users] Assessing ODF Conformance (Re: OASIS Standard ODF 1.2 Approved)

Tom Davis remarks that he's noticed a couple of ODF-related bugs for LibreOffice and it is presumed 
that they have been fixed.  That's good to know.

I want to discourage the notion that the absence of format-related bugs says anything about 
conformance.  Most bugs users and developers encounter are related to OpenOffice.org in operation 
and not the way it supports the ODF format in particular.  That is natural.  Most users have no 
sense of what is ODF-related and what is not.  Developers are usually dealing with bugs that deal 
with visible operational behavior, not handling of the ODF format in terms of what the ODF 
specifications say.

I don't mean the level of conformance is not high, but that bug reports don't determine that.  
(This is just like the absence of bug reports not being evidence that there are no bugs.)

 - Dennis

AN EXAMPLE

I am fairly heavily involved in how ODF packaging works.  This involves some 
specifically-structured XML files included in a Zip archive.  That's what .odt, .odp, and .ods 
files are.

As an example, here is a set of test documents that examine one aspect of resilience of packages.  
They exercise one very tiny case of legal variations in the rules for XML.  The files help confirm 
that those variations are being handled properly: < 
http://tools.oasis-open.org/version-control/svn/oic/TestSuite/trunk/odf12/NameSpaceResilience/>.

This file describes what the variations in the documents are:
<http://tools.oasis-open.org/version-control/svn/oic/TestSuite/trunk/odf12/NameSpaceResilience/NameSpaceResilience.txt>

This web page describes the result of testing the 9 files against 10 different releases of 
ODF-supporting software:
< 
http://tools.oasis-open.org/version-control/svn/oic/TestSuite/trunk/odf12/NameSpaceResilience/NameSpaceResilience-Results.htm>.

Now, almost all ODF implementations produce the variation in test-case 01.  The question is, do 
they consume the other two valid ones different than the ones they produce.  All but two accomplish 
that.

The six remaining cases involve invalid manifest documents according to the XML schema and the ODF 
specification.  In terms of resilience in the face of defects, the implementations are all over the 
map, as you can see.  These cases are not about conformance of course, because the inputs have 
defects.  But they do show something about resilience in the face of defects.  That can matter in 
how users are able to manage anyhow and especially if documents that are produced incorrectly (with 
regard to this simple case) turn up in the wild.

SUMMARY

This test goes beyond what a product produces, which may be valid, to look at what is valid that it 
does not produce.  Users of the same product will not see these cases unless there
is some odd change from one version of a release to another or there is interchange between 
different ODF-supporting products.

The score is pretty good for this one tiny aspect of conformance.  Resilience in the face of 
defects is a bonus category unrelated to conformance.

It takes (tens of) thousands of exercises like this to ferret out bugs in how valid ODF documents 
are handled by software that supports the format.  Or to confirm that provisions for correct 
documents are handled properly for a feature that is supported and resiliently for a feature that 
is not supported.  Note that resilience can apply even when the ODF document is conformant but uses 
features that a given product release does not support.

-----Original Message-----
From: Tom Davies [mailto:tomdavies04@yahoo.co.uk] 
< http://listarchives.libreoffice.org/global/users/msg11817.html>
Sent: Sunday, October 02, 2011 18:00
To: users@global.libreoffice.org
Subject: RE: [libreoffice-users] Re: OASIS Standard ODF 1.2 Approved

Hi :)
I think there are a couple of bug-reports for deviations from the 1.2 format but probably some of 
that has been fixed by now.  
Regards from
Tom :)


--- On Mon, 3/10/11, Dennis E. Hamilton <dennis.hamilton@acm.org> wrote:
From: Dennis E. Hamilton <dennis.hamilton@acm.org>
< http://listarchives.libreoffice.org/global/users/msg11816.html>
Subject: RE: [libreoffice-users] Re: OASIS Standard ODF 1.2 Approved
To: users@global.libreoffice.org
Cc: discuss@documentfoundation.org
Date: Monday, 3 October, 2011, 1:52

Marc asks: How conformant is LibreOffice as a producer and consumer
of the OASIS ODF 1.2 Standard?

I don't know how to assess that.  I assume it is reasonably
conformant and that most documents that happen to be produced
are satisfactory to consumers that accept ODF 1.0/1.1 also.

[ ... ]


-- 
For unsubscribe instructions e-mail to: users+help@global.libreoffice.org
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://listarchives.libreoffice.org/global/users/
All messages sent to this list will be publicly archived and cannot be deleted