LibreOffice run modes (headlessness)

Hi everybody

I'll start of a link to an old thread: "How many degrees of headlessness do we need?"
https://lists.freedesktop.org/archives/libreoffice/2015-March/067334.html

After wondering why my LO master would just start broken on MacOS in the debugger, I thought about
fixing the "headless handling". Unaware of the old thread I opened this can of worms / Pandoras box,
initially breaking stuff left and right. Originally I just wanted to get rid of VCL_HIDE_WINDOWS,
and while looking at that code I also found GetPseudoHeadless().

FWIW: the current state is here: https://gerrit.libreoffice.org/#/c/61598/

So this tries to condense the various requirements to get a plan of implementation (TODO list) and -
unexpected - it didn't result in a total nightmare (yet).

My current plan is to have just three commandline switches, which directly map to enums / bool

--run-mode=visible|hidden|headless(|console)

1. visible: normal GUI (default)
2. hidden: like normal, but start hidden / no window
3. headless: can not become visible; still uses OS GUI functions, so needs GUI
(4. console: can not become visible, as it doesn't use GUI functions at all)

* AKA level of visibility / headlessness
* Level 4 is done by SAL_USE_VCLPLUGIN=svp as a virtual GUI, which just uses images for output.
* Or use console only internally for stuff like gengal?

--dialog-cancel-mode=off|silent(|fatal)

off: show dialogs (default)
silent: automatically cancel dialogs
(fatal: aborts LO - currently internally only for debug)

* Possible new feature: allow to enable "silent" via GUI for "trained touch-typists"
  => needs to be clearly visible in UI (maybe via this info bar used for R/O open etc.)
* "silent" conflicts with --run-mode < 3 as commandline?
* Not sure we want to export fatal to "users".
  If not use --dialog-cancel-automatically.

--no-shutdown / --server

* Don't shutdown LO if the last document closes.
* Just allow terminate via UNO or signal
* Continue to run as background job

Mappings:

--quickstart => --run-mode=hidden --no-shutdown
--invisible => --run-mode=hidden
--headless => --run-mode=headless --dialog-cancel-mode=silent
VCL_HIDE_WINDOWS => drop
comphelper::LibreOfficeKit::isActive() => bool

Application::EnableHeadlessMode => SetDialogCancelMode(dialogsAreFatal ? DialogCancelMode::Fatal :
DialogCancelMode::Silent);
Application::IsHeadlessModeEnabled() return IsDialogCancelEnabled() { GetDialogCancelMode() !=
DialogCancelMode::Off; }

Not sure why IsHeadlessModeEnabled maps to the DialogCancelMode.

comphelper::LibreOfficeKit::isActive() { return g_bActive; }
MiscSettings::GetPseudoHeadless() { return getenv("VCL_HIDE_WINDOWS") ||
comphelper::LibreOfficeKit::isActive(); } => drop
Application::EnableConsoleOnly() => { EnableHeadlessMode(true); bConsoleOnly = true; }

LOOL: uses "SAL_USE_VCLPLUGIN=svp --headless"
LOK: a lot of sprinkled comphelper::LibreOfficeKit::isActive() calls. I'm not sure how many could be
dropped for default headless mode
UI tests: run as SAL_USE_VCLPLUGIN=svp *without* --headless. That's why they just work on Linux.
 => --run-mode=headless should be also sufficient, just not via SSH.

Then there is stuff not directly related:
* create new hidden documents => can be turned visible. Visibility depends on run-mode.

Comments?

Jan-Marek