Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call

On Thu, Dec 08, 2011 at 06:45:30PM +0100, Lionel Elie Mamane wrote:
> On Thu, Dec 08, 2011 at 04:19:04PM +0000, Michael Meeks wrote:

> > * postgresql fun. non-internal

> >      + Windows - run-time DLL dependency a pain (Lionel)
> >              + ship DLL ourselves ?

> This is not a question mark, we have to do that, or link statically,
> be it only because of Fridrich-mentioned problems.

> Note that currently postgresql-sdbc always links statically when
> using internal PostgreSQL.

I did that because it seemed the easiest thing to do at first
sight. However, we are losing the advantage of that, and thus I
propose we switch to dynamic linking, and bundling the lib on
platforms / situations that need it.

I started actually testing the "internal PostgreSQL C client library"
build in use, and I realised it is rather crippled:

 - no SSL/TLS support, for encrypted communication with the DB server
   and authentication by X.509 certificate.

 - no Kerberos support: authentication; apparently can be used for
   windows domain "single signon" from a non-windows machine.

 - no GSSAPI support: yet another authentication mechanism

 - no LDAP support: yes, in the client library: this allows to store
   the connection info in LDAP and thus manage it centrally. Can be
   really convenient in enterprise environment.

Leaving only hardcoded-in-LibreOffice's .odb file connection string
with user/password authentication. Of these features I'd expect SSL
(encrypted connection) is the most widely used one.

I started hacking our internal postgresql to activate all this, but
this introduces dependencies on yet other libraries: OpenSSL, MIT
Kerberos, com_err, ldap, ... This means that linking statically
against libpq is not the easiest anymore, since static linking does
not automatically recurse, in the way that dynamic linking does :)
More deeply, we don't gain anything/much by linking statically against
libpq, but dynamically against these other libs. But linking
statically against some of these other libs is
significantly harder than dynamic link:

 - At least on my Debian, libssl is not compiled with -fPIC, so a
   dynamic library (postgresql-sdbc-impl.so) cannot link statically
   against it. This could be solved by recompiling OpenSSL ourselves.

 - MIT kerberos does not support static linking; vaguely discussed how
   to allow it in 2006, but no progress since then:
   http://krbdev.mit.edu/rt/Ticket/Display.html?id=3838
   That's not something we'll solve simply by recompiling MIT krb5...


So, to me it looks like we'll switch to dynamic linking against libpq
and shipping the DLL/SO/DYLIB on the platforms/cases that need it. Any
platform where this will be harder than what I presume?

As an added bonus, this makes it easier for our users to upgrade the
libraries we link against (for bugfixes or new features in new
versions or ...): just overwrite the DLL/SO/DYLIB file with an
upstream-provided one (downloaded from http://www.postgresql.org).


What I'm not totally clear about is how we will generate the list of
libraries to bundle. I could imagine recursing through the DT_NEEDED
fields of the .so files (and the equivalent of that on other
platforms), and stopping when hitting a known "system library" (such
as libc, libresolv, etc); I imagine this exact same problem must have
been solved already elsewhere in LibreOffice and we have a dmake/make
recipe we can apply / reuse? Or is it not so?


This also means that by our "we have to build everything we ship"
rule, we have to internalise, according to my first quick survey:

 - OpenSSL: I expect all platforms. It is a system lib on most
            GNU/Linux distributions, but the soname is not stable.

 - MIT Kerberos:

   * MacOS X bundles Kerberos since 10.2, which AFAIK is behind our
     baseline of 10.4; is it binary-compatible across releases? Then
     we can consider it a system library on MacOS X and not
     internalise / ship it.

   * GNU/Linux: What's our baseline there? Has the soname changed
     since our baseline? Is the soname consistent across
     distributions? If no&yes, then system lib and don't ship it.
     Debian seems to have a stable soname since woody (released 19
     July 2002), so I expect this will be OK.

     If we have to internalise it, it depends on:

     ° libcom_err: built from e2fsprogs sources on my Debian, soname
                   libcom_err.so.2. Google tells me (some?) RPM
                   distributions have a libcom_err.so.3 that is
                   shipped by the MIT Kerberos RPM itself.

 - Kerberos for Windows: that's not part of the OS -> internalise
   (http://web.mit.edu/kerberos/dist/index.html#kfw-3.2)

 - LDAP: We already have OpenLDAP-related stuff in configure.in... So
         can postgresql-sdbc just use that, or is the existing stuff
         "too optional"? I mean it seems to be replacing some Mozilla
         parts, not sure we enable it on our builds or if it just a
         service to distributions. If cannot just use:

   * MacOS X: is it bundled? Some googling finds LDAP-related commands
              in MacOS X 10.4, our baseline. So we are good to
              consider it a system lib?

   * GNU/Linux: Debian has stable soname since 2008. Assuming other
                distros are similar, is that sufficient to consider it
                a system library?

   * Windows: The Microsoft implementation is used, Google tells me it
              is in the Windows platform SDK. Our current way to build
              libpq does not have an option for enabling LDAP, but
              peeking at the official Perl-driven process, it seems
              easy enough to hack in:

              ° #define USE_LDAP 1

              ° link against wldap32.lib


Actually, I'm still not sure I understand 100% our internalisation
policy. For example, we internalise pretty "basic / system" stuff like
cairo, zlib, mesa, gdk-pixbuf, berkeleydb, some X11 headers, ... But
not GDK, GTK, QT, gstreamer, cups, openldap (for the use we make of
it), gconf, fontconfig, freetype, ...

> >      + building vs. an internal libpostgresql (Lionel)
> >              + client library code is:
> >                      + around 25 files + a few headers
> >              + can knock up a gnumake makefile in a module
> >                as a static library.

> PostgreSQL on Windows can be built in more ways than I knew:

>  - Borland C++ / old MSVC: not useful

This turned out to be useful after all: we ended up using the
win32.mak file intended for old MSVC, slightly patched.

-- 
Lionel