Re: [libreoffice-l10n] Pootle migration

Hi *,

On Mon, Apr 4, 2011 at 2:58 PM, Rimas Kudelis <rq@akl.lt> wrote:
> Stuff to consider: Pootle may run slower on this machine, and we may
> experience other problems, at least for now. I wasn't able to convince the
> admin that Pootle needs more resources, so we have what we have.

Again, as you apparently still don't understand what I already wrote many times:
* Adding more resources will /NOT/ make pootle run faster than it does now.
The VM already has way more resources assigned than necessary. It is
/idle/ almost all the time.
* The only thing that is slow (when executed the first time) is
generation of the zips. So when you as translator request a zip: Don't
click the link multiple times because you don't immediately get the
zip. It can take 10 seconds for the files to be generated. Again:
* Adding more resources will /not/ make that time shorter. It is a
single-threaded process that can only uses one single CPU, thus
assigning more CPUs won't help at all (the VM has 4CPUs assigned
already)
Requesting that same zip another time (or different zips of the
project belonging to the same language is fast/instant, but requesting
the zip for another language again may take some seconds for the first
request (or again after the files did change in between).
* Pootle has a memory leak when creating the zips. It won't release
memory after processing the files.
This would be the only time where the assigned resources may run out
(the VM has 1GB or RAM assigned): Multiple different languages request
the zip at the same time. Then memory usage increases, memory runs out
and either it is crawling along or the process gets killed.
* I will NOT assign RAM to a VM (and thus block that ram for other
use) to satisfy a memory leak, when that RAM is unused 99% of the
time.
* The effects of the memory leak can be nullified by just restarting
the worker processes more frequently. Thus again:
* Adding more resources will /NOT/ make the VM run faster, it will
/NOT/ allow it to handle more requests

Pootle is idling almost all of the time. There is less than one apache
request per second on average (and for regular requests (i.e.
non-"generate-a-zip" actions) it can easily serve >>50 simultaneous
requests per second.)

> Maybe if we
> manage to give enough load to the server, he'll change his mind (or we'll
> find other ways to deal with the problem).

No, I won't change my mind, but depending on the load/effects of the
memory leak I'll reduce lifetime of the server-processes further.

Again:
The only time where pootle is "slow" is:
* Creation of zips for the first time / after files have changed.
This is CPU intensive process, and the CPU cannot be made faster by
assigning more resources. Live with it. Redesign pootle to use another
method (i.e. a multithreaded one that can use multiple CPUs at once)
or whatever, but this one problem is not solvable by assigning more
resources.

* This also is only noteworthy when the files are big/numerous.
* Requesting the other zips in the same project & language is fast.

So there is no point in clicking all zip-URLs on a page at once (on
the contrary, than the request will all cause CPU to be burnt, while
all that is only needed one single time). If you want to download more
than one zip of a page, click the first one, wait until it is
generated and handed over to your browser, then click as many others
on the same page and get all of them quickly.

* Any other time, doing in-place-translation, just browsing along is
supposed to be fast.
If it is not, it needs to be investigated - i.e. please report it with
a description of what you did, when the problem occurred (and of
course what you definition of "slow" is)

The server is under close monitoring using munin, so bottlenecks
should be easily identified.

But with an (daily) average load of 0.09 and an average of 0.08 apache
requests/second (to which munin also contributes) are no way reason to
assign more resources.

Again:
* The VM has plenty of resources, with much reserves for more traffic/accesses.
* Creation of ZIPs is CPU-intense and thus take some seconds (for the
large packages up to 10 seconds) Be patient when requesting a zip for
the first time.
* When you see "premature end of script headers" or similar error
message - report it. This means the memory-leak did exceed the limits
and the lifetime parameters need to be tweaked.
* If you experience any "slowness" on other operations than requesting
zips: Report them.

(for users in Europe, it  might now even be faster, as the data
doesn't need to cross the Atlantic anymore - well, not that humans
will notice that difference, but still :-))

ciao
Christian

-- 
Unsubscribe instructions: E-mail to l10n+help@libreoffice.org
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://listarchives.libreoffice.org/www/l10n/
All messages sent to this list will be publicly archived and cannot be deleted