Re: Testing/Working on PyUNO?

[putting the dev list on CC, quoting the original mail in full below for 
reference]

Hi Kevin,

First of all: great that you want to improve the PyUNO situation!  This 
is a part of the code that could indeed benefit from lots of love.

Regarding the Python tests, I see both sides' points:

On the one hand, a test in Python indeed does not only serve as a test 
for the immediate subject of the test, but also---somewhat 
indirectly---as a test of the PyUNO infrastructure and as part of a 
corpus of PyUNO example code.  This is not much different from how our 
well-hated so-called "unoapi" and "complex" tests test URP (and Java) 
interaction as a side effect (though they're so ugly that nobody in 
their right mind would claim that they are valuable additions to any 
corpus of example code).

On the other hand, a test is intended to fail (as soon as you introduce 
an error), and should make it easy for the developer to find the 
underlying error.  For that, any extra layers between the test code and 
the code under test are detrimental, in that they make it harder to 
debug the problem and introduce the likelihood that a reported error is 
not actually in the code under test but rather somewhere else in the 
infrastructure (which remains an error that needs to be fixed after all, 
but less likely by the person who changed the code under test).

So, what can we do here?  How about an additional code module dedicated 
to subsequentchecks written in Python, which, while serving the useful 
purpose of testing specific application functionality (and in which role 
they may deliberately duplicate existing "native" tests), are meant more 
as a means to improve the PyUNO situation.

The idea would be that Kevin (and others) would fill this with PyUNO 
coding scenarios that cross their mind, discover errors in the PyUNO 
infrastructure, ideally distill tests for those specific errors out of 
the more general tests (that could then even go into more specific code 
modules like pyuno or testtools), and eventually prune test snippets 
again that are no longer useful.

What I would like to avoid though is a sort of code dump, filled with 
random one-off stuff of little value to anybody.  We have 
<https://gerrit.libreoffice.org/#/admin/projects/sdk-examples> already, 
and it lies virtually dormant.

How does that sound?

Stephan


On 02/17/2014 08:53 PM, Kevin Hunter Kesling wrote:
> Hi Stephan,
>
> Markus (moggi) suggested over IRC that I talk to you about how to test
> PyUNO interactions with Calc.  The long-winded email below is just
> background for this question: I claim the LibO/Python bindings are weak
> and poorly tested.  As my work often presents opportunities that could
> be brilliantly addressed by easier Python scripting of LibO, I'd like to
> improve the Python bindings situation.  How best can I accomplish this?
>
> I recently submitted a patch to gerrit that adds a test nominally for a
> recently closed bug:
>
>      https://gerrit.libreoffice.org/#/c/8078/
>
> This test is a functional test of a specific Calc interaction (inserting
> cells).  As noted in the commit message, the bug was already quashed,
> and Kohei said in the IRC conversation[1] that the offending library
> (mdds) already has a test for it's role in the related fdo bug. However,
> that still says to me that the actual part that an end-user cares about
> -- in this case the inserting of cells -- is not directly tested.
>
> Through interaction with Kohei and Markus, it's clear that writing the
> tests for the internal workings of a C++ project in Python is not
> acceptable (not the right "tool for the job"), and I can see the
> perspective that not one, but two core maintainers have.
>
> However, this leads me to the larger observation (the one for which I
> originally decided to write the test in Python over C++) is that I see
> hardly any tests of LibO's Python bindings of Calc (sc/).  Ideally, I'd
> like to be able to do more Python scripting of LibO, and the quality of
> the bindings is what holds me back.  This snippet from the attached
> larger IRC conversation may help elucidate where I'm coming from:
>
>      (01:57:35 PM) kohei: and I'm generally not in favor of python
>         unit tests. it's not very useful for the maintainers to
>         debug failures.
>      (01:58:48 PM) frothnicator: kohei: okay, I hear that I don't
>         get it.  I'm trying to.  The only way I know to test and
>         learn something is to use it.  My main interaction would be
>         through UNO, unless I learn of a better way.  Thus, when I
>         try to introspect an item through Python's REPL, and get
>         back an exception for merely calling repr(some_object), or
>         my remote connection is broken because of a recoverable
>         exception, I don't know what else to call that unstable.
>
> The point being that though I wrote it in Python, the test is actually
> testing multiple things, both of which are important to me: the
> interaction between Python and LibO, and the core functionality that
> motivated the now-abandoned patch.
>
> The more important observation, however, is that while writing this
> test, I encountered many niggles: poor REPL-accessible documentation,
> broken remote connections despite recoverable exceptions, useless
> attempts to introspect object details, undiscoverable imports, other
> crasher bugs, a lack of easily discoverable (read: Googlable)
> Python-based examples from which to learn, and so on.  I would not mind
> addressing these as I have time as well.  But I need a place to put
> these Python based tests.
>
> The question that was still on my mind after the above interaction was
> "Should I infer that LibO/Python bindings are not a high priority for
> LibO devs?"  or "Is using Python with LibO a foolhardy wish on my part?"
>   Markus assured me that no one said that, and directed me to ask you.
>
> So, upon articulating my thoughts in this email to you, what I'm really
> looking for is an "in" such that I can help the situation that most
> directly affects me.  (The logic being that with luck, I'm not the only
> one who would like to use LibO in this fashion!)  I don't expect that
> I'll write perfect code or tests, but that by at least getting the ball
> rolling, iteration -- by me or others -- will fix any issues.  (After
> all, that's why we developed revision control systems, yes?!)  I, for
> one, would appreciate a more robust Python scripting experience with LibO.
>
> Do you have any suggestions for how I can help improve the PyUNO
> scripter's experience?
>
> Thanks,
>
> Kevin
>
> P.S.  I was told to ask you, so I opted to email you directly.  If you
> think the -dev list is a more appropriate place for this thread, please
> CC: there as appropriate.
>
> [1] Attached is an IRC conversation from earlier today.