Re: Python UnicodeEncodeError in the unittest runner during uicheck on Windows

Hi Stephan,

On Thu, Apr 09, 2020 at 05:02:10PM +0200, Stephan Bergmann <sbergman@redhat.com> wrote:
> Trying to make uicheck work on Windows, I encountered a test failing due to
>
> > FAIL: test_tdf125104_pageFormat_numbering (tdf125104.tdf125104)
> > ----------------------------------------------------------------------
> > Traceback (most recent call last):
> >   File "C:/lo/core/uitest/test_main.py", line 128, in <module>
> >     result = unittest.TextTestRunner(stream=sys.stdout, verbosity=2).run(test_suite)
> >   File "C:\lo\core\instdir\program\python-core-3.7.7\lib\unittest\runner.py", line 183, in run
> >     result.printErrors()
> >   File "C:\lo\core\instdir\program\python-core-3.7.7\lib\unittest\runner.py", line 110, in 
> > printErrors
> >     self.printErrorList('FAIL', self.failures)
> >   File "C:\lo\core\instdir\program\python-core-3.7.7\lib\unittest\runner.py", line 117, in 
> > printErrorList
> >     self.stream.writeln("%s" % err)
> >   File "C:\lo\core\instdir\program\python-core-3.7.7\lib\unittest\runner.py", line 25, in 
> > writeln
> >     self.write(arg)
> >   File "C:\lo\core\instdir\program\\python-core-3.7.7\lib\encodings\cp1252.py", line 19, in 
> > encode
> >     return codecs.charmap_encode(input,self.errors,encoding_table)[0]
> > UnicodeEncodeError: 'charmap' codec can't encode character '\u0916' in position 195: character 
> > maps to <undefined>
>
> I assume the test itself fails due to some other reason, and then printing
> out that reason causes this error.  But I have no idea on which level to fix
> this.  Do we need to change something in the UITest machinery when we call
> that Python unittest runner, or is it a bug that needs patching in that
> Python unittest runner?

One option would be to tweak test_main.py to default to UTF-8, like
Linux does. You could try something like this:

if sys.platform.startswith("win"):
    import _locale
    _locale._getdefaultlocale = (lambda *args: ['en_US', 'utf8'])

If this helps, we could do this early in test_main.py, before it calls
into unittest.

Regards,

Miklos

Attachment: signature.asc
Description: Digital signature