[libreoffice-users] RE: [Libreoffice] Word doesn't see symbols - More Analysis

Although the symptoms of font problems show up in the context of Word <-> LibreOffice Writer, the 
situation is trickier than that.  It is exacerbated by failure to use standard Unicode code points, 
eliminating the possibility of successful font substitutions, especially for symbols that don't 
vary that much from one font to another and there is little danger of confusion.
 
 1. AVAILABLE FONTS

Most office-productivity software products rely on the fonts installed on the operating system, 
sharing those fonts with all software on the platform.  It is also the case that the installation 
of software products will install fonts, optionally or automatically, for users on that computer.   
(Some fonts are only licensed to be used with the product or platform on which the product is 
installed.)

Users can also install an enormous variety of fonts of their own choosing, whether licensed from a 
font foundry or obtained from one of the sources of unrestricted fonts.

Example: Beside the fonts that are available on the platform directly, my main Windows 7 desktop 
system has fonts installed by Microsoft Office 2010, Visual Studio 2010, Photoshop Elements, Word 
Perfect X5 Standard, and LibreOffice 3.3.2, among others.  All of these fonts are available to 
LibreOffice 3.3.2 Writer.

 2. INTERCHANGING DOCUMENTS WITH FONT DEPENDENCIES

When fonts use essentially the same code points for the same characters, but with differences in 
font-face design, there are techniques to substitute a close kindred font when the specific font is 
not available to the consumer of a document.  This might create problems with metrics, but there 
are many fonts that substitute well enough.   Systems may provide automatic substitutions for fonts 
that are not installed.  Products also have ways to let users direct the substitutions.  

Example:  In LibreOffice, the Tools | Options | LibreOffice | Fonts dialog provides for 
substitutions.  The Help topic indicates the range of capabilties.

Font substitutions don't work so well for decorative fonts and it is generally not possible for 
symbol fonts that use special code pages (older systems) or non-standard Unicode code points for 
their characters.

Some fonts don't provide substitutions very well at all.  In particular, there is no assured 
substitution for Unicode-based fonts that rely on code points in the Unicode Private Use Area, 
U+E000 to U+F8FF.  

Example: The Microsoft Symbol, Webdings, Wingdings, Wingdings2, and Wingdings 3 each use 
private-use code points in the same range: U+F020 to F0FF.  U+F020 is always a space character, but 
the other code points are not substitutable characters among those fonts.  Likewise, OpenSymbol 
uses some standard Unicode code points but it also includes an extensive number of code points in 
the private-use range U+E001 to U+E6A3.   These do not correspond to the use of private-use code 
points by Linux Libertine G (a Serif Unicode-based font) and Linux Biolinum G (a Sans Serif 
Unicode-based font).  None of these characters that have private-use code points are substitutable 
among different symbol-only or Unicode-based fonts.  

A significant number of the characters defined in the Private Use Area also have standard Unicode 
code points.  The standard Unicode code points should always be used instead if fonts that include 
them are available, such as Lucida Sans Unicode and Cambria Math.  Linux Biolinium G can also be 
used this way so long as its Private Use Area characters are avoided.

Generally, the greatest fidelity is obtained if it can be arranged to have the same font and font 
metrics installed on the computer system of the document consumers as were used by the document 
producer and on which the document depends.  

For documents that use characters from the private use area of fonts available to the document 
producer, the only prospect for successful interchange is if the document consumer has a font that 
defines those very same characters at the same code points of the private use area.  This has to be 
accomplished by private convention.

 3. PROVIDING THE FONTS THAT A DOCUMENT DEPENDS ON

If the same software is used by the producer and consumer, there is no difficulty when fonts 
consistently-supplied with the software, are used even if the platforms are different.

If the font has a different source, or different software is being used, there needs to be another 
way to provide the necessary font for use by the consumer.

There are two ways to do this:

    - Using a document format that embeds the fonts

    - Transferring font files to the other computer system

4. USING EMBEDDED FONTS IN AN INTERCHANGE DOCUMENT

A common method for ensuring that the consumer will view the document with fidelity to the document 
that was produced is to export Adobe PDF documents.  These documents can carry embedded fonts and 
it is possible to arrange production of bitmap versions of characters when the font is not 
embeddable.

Microsoft Word documents can also carry embedded fonts.  This allows for correct viewing by a 
recipient as well.

In general, having an embedded font in a document is not the same as having it available for use in 
other documents.  There are also barriers to using the font in further collaborative editing of the 
same document.  There are manual workarounds that experts might employ, but those are probably too 
tedious for anything but rare occasions.

Also, if the collaborative interchange is between different products, embedded fonts won't be 
enough.  The odds of passing embedded fonts through a converter are rather low at this point.  The 
possibility of round-trip preservation of embedded fonts by conversion in both directions is even 
more remote.

Example: ODF lacks an agreed means for embedding of fonts.  Word saving of documents with embedded 
fonts as ODF text documents loses the embedded fonts.  (If the receiving ODF consumer has the 
fonts, it will still work though.)  LibreOffice could provide font embedding as part of Save As 
Word format, but the current converters do not provide that capability.  Also, even with an 
embedded font, it may be cumbersome to use that font in further editing of the received document 
unless the embedded font can be installed on the receiving system by some means.

5. TRANSFERRING FONTS TO ANOTHER COMPUTER SYSTEM

One way to transfer fonts is to install a software application that provides them.

This is not so far-fetched.  LibreOffice can be installed for free, with minimum configuration, 
simply to obtain its unique fonts.   It is not necessary to run it.  We just want to have a common 
set of fonts.  

Another way is to package and transfer fonts for mutual installation and agreed use.   Fonts are 
not difficult to install.  It is likely that the LibreOffice fonts are more amenable to 
unrestricted transfer onto other computers.  This may be the only way of adding fonts into some 
controlled computer configurations.

Round trip collaboration also requires agreement to use only fonts that are arranged to be 
available on any of the computers and software products used among the collaborators.   Users will 
need to understand how to limit themselves to use of mutually-available fonts and to avoid pitfalls 
such as relying on characters in the Private Use Area instead of their standard Unicode code-point 
positions when those are mutually supported.

The limitation to mutual agreed use is the tricky part, since it requires users to be aware of and 
careful of the conditions that assure mutual success.

6. FOR LIBREOFFICE DEVELOPERS

It is unfortunate that there are so many uses of the Private Use Area at large in the world.  It 
would be valuable to avoid distributing more files that rely on them.  (The existing conflict 
between Open Symbol and Linux Biolinum/Libertine G is amusing enough.)

  a. It would be useful to update OpenSymbol to use the now-standard code Unicode code points for 
those characters that were previously unavailable as standard characters.

 b. It would also be useful to document the usage of the Private Use Area by fonts distributed with 
LibreOffice.  A PDF that shows the code points and character appearance would be an excellent way 
to make these known, so people would be informed on the impact their usage has for interchange with 
other systems and conversion to other formats.

 c. It would be especially valuable to adjust the LibreOffice repertoire of bullet symbols, 
especially those used by default, to avoid using code points in the Private Use Area for those 
characters.

 - Dennis


-----Original Message-----
From: Dennis E. Hamilton [mailto:dennis.hamilton@acm.org] 
Sent: Friday, May 27, 2011 13:53
To: 'Deve'; 'libreoffice@lists.freedesktop.org'
Cc: 'LOffice Users List'
Subject: RE: [Libreoffice] Word doesn't see symbols - NEW PROBLEM

There's a new problem.  I checked around for middle dots, centered dots, etc.

The SAFE one is U+22C5, the Middle Dot in the extensive Unicode block on Mathematical Operators, 
from U+2200 to U+22FF.

The dots that I saw in OpenSymbol are these: U+E146, U+E468, U+E466, U+E58D, and U+E584.  My copy 
of the Unicode Standard 4.0 says that these are all PRIVATE USE SYMBOLS.   This Private Use Area 
has existed since at least Unicode 3.2 and is unchanged in Unicode 6.0.  See 
<http://www.unicode.org/charts/>.

"These areas will never be defined by the Unicode Standard.  These code points can be freely used 
for characters of any purpose, but successful interchange requires an agreement between sender and 
receiver on their interpretation."  

Furthermore, the codes extending upward from U+E000 are intended for END-USER assignment and the 
codes extending downward from U+F8FF are intended for CORPORATE USE (including vendors, platform 
providers, etc.), the idea that the chance of collision is reduced thereby.   This is only a 
suggested convention, however.

These depend on private agreements for having matching fonts or even being used for visible 
characters.  Something tells me you'll be hard-pressed to find these in fonts on Windows unless you 
can send the OpenSymbol font to the recipient.  

[ ... ]

My recommendation is to use a defined Unicode character (and a font that supports it) in preference 
to the same OpenSymbol character whenever possible.

 - Dennis

[ ... ]


-- 
Unsubscribe instructions: E-mail to users+help@libreoffice.org
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://listarchives.libreoffice.org/www/users/
All messages sent to this list will be publicly archived and cannot be deleted