[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[libreoffice-website] Minutes from the Thu Apr 18 infra call


[Rescheduled from Tue Apr 16.]

Participants
============

1. guilhem
2. cloph
3. Brett

Agenda
======

* Post mortem March 26 incident
+ Distributed FS
. cloph: not opposing other solution, cloud solution probably not
feasible monetary wise (Brett and guilhem agree, AI guilhem to double
check)
. would require rebalancing (hence changing isolation level) as ordering
50-60 VPS would be very costly
. cloph: a single big issue with Gluster in ~6 (?) years, convenient
scapegoat but not all issues should be attributed to that setup
. ceph was considered at the time, abandoned since ≥6 nodes are (were?)
needed
+ guilhem: would like some SSD-based cache for faster reads; cloph: Gluster
suport tier-based setups, but we're not using that currently, not sure if
there are any slot left in the hypervisors (guilhem to check)
. Another problems is random IO from the crashtest VM (vm138), the disk
image of which resides on charly's /data/fast like gluster's delta
arbiter. The two race and trigger heals
. If there is room for one or more SSD on each hyper we should move the
arbiter (and swap) there
+ cloph: could also compare our current machines with the newer generation
from manitu (125€/month, Intel Xeon E3-1240 v5)
+ rebalance gluster volumes: too many VMs on delta, heals drains IO on too
many services
. delta 2 x (2+1), alpha, gamma, kappa 1 x 3
. cloph: straightforward to change the layout on an existing volume
. can rebalance, can also tune the healing settings so it doesn't starve
all guests
+ Continuous Archiving and Point-in-Time Recovery (PITR)
. replace dump-based database backups with continuous logs
. https://www.postgresql.org/docs/9.6/continuous-archiving.html
. can be dry-run without disruption to the current setup
. Brett: can try out on the [Matrix] host (vm222), push the WAL logs to
itself for now
* OCS-Webserver (new extension site) deployment
+ somewhat delayed due to infra issues
+ succesfully bridged to SSO, some hard-coded third-party requests to chase
and remove still, before opening the site for contributions
* tb31
+ connects to the outside from 212.234.77.252 (not 212.234.77.246) but
seems to block incoming TCP SYN, ICMP
+ is it sitting behind a firewall/NAT box?
+ cloph: we can move lcov to another host (part of LODE)
+ AI guilhem: check with Sophie/thb for contact info, fix incoming
connections, perform full backup and ask for an installation of CentOS7 (or
just ask for KVM access if possible)
* Build bots:
+ tb66 will be brought offline permanently in early May
+ large build logs
. 4GiB logs are not sustainable (a build issue caused a serie of build
logs to grow by a factor of >20 last months) and DoS CI
. gerrit_linux_clang_dbgutil (normally up to ~250MiB then they grew to
4GiB) cf. gerrit_linux_clang_dbgutil/builds/28522/log.gz (288MiB
compressed, 4GiB uncompressed)
. there is a jenkins plugin to limit log size but that only works with
pipeline-based setups
* Preserve old download links (redirect to downloadarchive), cf. wget's mail
+ cloph: not keen to manually maintain a map for a single distro
+ switching the links to use downloadarchive instead of download might
take time but that's just a one-off overhead
* Chance to speed up latam conference site deployment? → done
* Next call: Tuesday May 21 2019 at 18:30 Berlin time (16:30 UTC).

--
Guilhem.

--
To unsubscribe e-mail to: website+unsubscribe@global.libreoffice.org
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/website/
Privacy Policy: https://www.documentfoundation.org/privacy

Privacy Policy | Impressum (Legal Info) | Copyright information: Unless otherwise specified, all text and images on this website are licensed under the Creative Commons Attribution-Share Alike 3.0 License. This does not include the source code of LibreOffice, which is licensed under the Mozilla Public License (MPLv2). "LibreOffice" and "The Document Foundation" are registered trademarks of their corresponding registered owners or are in actual use as trademarks in one or more countries. Their respective logos and icons are also subject to international copyright laws. Use thereof is explained in our trademark policy.