[internal API] New OOXML password hashing and helper functions

Hi,

With a series of commits for
https://bugs.documentfoundation.org/show_bug.cgi?id=104250
and some follow-ups we now have functions to generate proper password
hash values as specified in OOXML, see also
https://msdn.microsoft.com/en-us/library/dd925430 (Standard Encrytpion)
https://msdn.microsoft.com/en-us/library/dd924776 (Agile Encryption)
https://msdn.microsoft.com/en-us/library/dd920692 (ISO Write Protection Method)

The actual "raw" implementation is in comphelper::Hash with

    static std::vector<unsigned char> calculateHash(
            const unsigned char* pInput, size_t nLength,
            const unsigned char* pSalt, size_t nSaltLen,
            sal_uInt32 nSpinCount,
            IterCount eIterCount,
            HashType eType);

and a convenience function that takes an UTF-16 encoded password string
and a vector with salt value

    static std::vector<unsigned char> calculateHash(
            const rtl::OUString& rPassword,
            const std::vector<unsigned char>& rSaltValue,
            sal_uInt32 nSpinCount,
            IterCount eIterCount,
            HashType eType);

See include/comphelper/hash.hxx and descriptions there.

Along with that there are some convenience and helper functions at
comphelper::DocPasswordHelper, see
include/comphelper/docpasswordhelper.hxx
and descriptions there.

    static css::uno::Sequence<sal_Int8> GetOoxHashAsSequence(
            const rtl::OUString& rPassword,
            const rtl::OUString& rSaltValue,
            sal_uInt32 nSpinCount,
            comphelper::Hash::IterCount eIterCount,
            const rtl::OUString& rAlgorithmName);

    static rtl::OUString GetOoxHashAsBase64(
            const rtl::OUString& rPassword,
            const rtl::OUString& rSaltValue,
            sal_uInt32 nSpinCount,
            comphelper::Hash::IterCount eIterCount,
            const rtl::OUString& rAlgorithmName);

    static std::vector<unsigned char> GetOoxHashAsVector(
            const rtl::OUString& rPassword,
            const std::vector<unsigned char>& rSaltValue,
            sal_uInt32 nSpinCount,
            comphelper::Hash::IterCount eIterCount,
            const rtl::OUString& rAlgorithmName);

Specifically GetOoxHashAsBase64() can be used to pass the strings
obtained from the OOXML file as they are and compare the result against
the document's hashValue string.

oox/source/crypto/AgileEngine.cxx
oox::core::AgileEngine::calculateHashFinal() now uses
comphelper::DocPasswordHelper::GetOoxHashAsVector()

Newer OOXML documents written by MS use "SHA512" (for encryption) or
"SHA-512" (for sheetProtection and fileSharing elelements and maybe
others) algorithm names. The helper functions accept both.

Note that to calculate hashes for new passwords a new salt value MUST be
generated and never existing salts reused, otherwise the whole purpose
of salts is defeated.. for this
comphelper::DocPasswordHelper::GenerateRandomByteSequence() can be used,
a length of 16 currently seems to be a common value.

Happy Hashing

  Eike

-- 
LibreOffice Calc developer. Number formatter stricken i18n transpositionizer.
GPG key 0x6A6CD5B765632D3A - 2265 D7F3 A7B0 95CC 3918  630B 6A6C D5B7 6563 2D3A
Care about Free Software, support the FSFE https://fsfe.org/support/?erack

Attachment: signature.asc
Description: PGP signature