Re: 3.5.3rc1 win32 / debug package ...

On Fri, 2012-06-01 at 13:10 -0700, bfo wrote:
> Michael Meeks-2 wrote
> >
> I think it won't do any good in the long term. Enabling reporting tool would
> give you a chance to
> get more data without the need to know the bugzilla magic by the users (just
> opt in to send the reports). 
> It could bring more testers to the project, as now you have to do an own
> debug build to test LO 
> with proper backtraces (especially on Windows).
> Then (or I should write "Before that") you can think about what to do with
> the data (and how). 
> Luckily you can use experience of other projects (nice reading at
> http://blog.mozilla.org/metrics/) 
> or code even and work with dashboards like https://crash-stats.mozilla.com/ 
> or http://brasstacks.mozilla.com/orangefactor for instance. 
> I can see you have quite a few  http://tinderbox.libreoffice.org tinderboxes  
> but the question is - how do you use them for QA? 
> Browsing through QA articles I can feel it is more like - download, run,
> test, report scheme atm.

I hope that we could end here. I also like Launchpad from Ubuntu. It
merges reports with the same backtrace. So, it automatically handles
duplicates. You could easily see how many people are affected and find
the related comments on a single location.

Note that we need to do this step by step. First, we need to have the
builds with debuginfo. Then we need to have infrastructure to filter the
results, handle duplicates, ... Finally, we could enable the reporting
tool.

Note that we already have troubles to filter all the manually created
bugs. We are looking for more people helping with the bug triage, see
http://wiki.documentfoundation.org/BugTriage

If you could help, please step in.

> >     Anyone is welcome to put / link them into whatever wiki page they
> > like :-) if you want it - go for it ! I'm happy to add a link to it at
> > the bottom of my template so it's easy to find in future.
> Yea... As wiki.d.o can have 
> http://wiki.documentfoundation.org/TDF/BoD_Meetings TDF BoD meetings  or 
> http://wiki.documentfoundation.org/TDF/Membership_Committee_Meetings
> Membership Committee Meetings  
> I simply do not understand why Engineering Steering Committee Meeting are
> not there. 
> It is very valuable information about what is going on in the project. 
> http://wiki.documentfoundation.org/RBd/TSC_Call_Minutes RBd 's notes are a
> good read, 
> but you know... pasting minutes into wiki template do not need another
> volunteer...

Heh, Michael already does so many things. In fact, he is looking for a
volunteer who could join the call and take the minutes.

Anyway, you might volunteer and take the minutes from the mailing list
and paste them into the wiki. If you prepare the pages and template and
if it is easy to add new week, Michael might do it himself in the
future.


> > > Also I am really concerned about your QA priorities. IMHO you should
> > > care little more about Windows builds and MS Office filters.
> >     Who is the 'you' here ? you are one of us if you're helping out :-) 
> > [...]
> >     However, getting the best and fastest possible list of and
> > prioritisation of bugs is a crucial task of QA 
> By "you" I mean QA Team.

We are still building the QA team. Feel free to join and move forward
some activities.


>  Reading ESC minutes I do not see general rule of
> prioritisation of bugs.

It is pity but we do not have defined rules for prioritization of bugs.
It currently depends on the feeling of the reporters and bug triagers.

Would you be interested into creating such a proposal? You might take
inspiration in the attached document that describes rules for Novell
bugzilla. IMHO, it is reasonable and might fit even LibreOffice. We just
need to use LibreOffice-specific examples.

> Is it 1. crashers 2. regressions 3. enhancements or any other scheme? I
> can't feel it atm. 

We use the keyword "regression" to mark regressions. We motivate
developers to work on these bugs with higher priority.

AFAIK, we do not have any keyword for crashers. IMHO, we use a
workaround because these bugs usually have the work crash in subject.

In each case, Rainer is leading the activities in this area. I am not
sure if he has it described in the wiki. I can't find it easily.


> You have MAB meta bugs, but it's more "my bug is more buggy than your bug"
> attitude there.

This is only a workaround. We do not have enough people to go trough all
reported bugs and standardize the priority and severity flags. Users
tend to set higher severity for their bug, so the original setting is
not useful. MAB is the place for high priority bugs that have been
confirmed by a developer or bug triager.

MAB wont be needed once we have good rules for setting severities and
priorities and enough people who could confirm the original reports.


> > > Sadly I'm not QA specialist. I just decided to help LO project by
> > > confirming as many bugs as I can find in Bugzilla. Hope this will help
> > > increase the quality of LO for Windows in any way...
> >     Certainly - it's a really good way to help out. 
> Yes, I started doing that. It seems it moved things a little bit in few bugs
> already.
> Should I do something more?

Yes, any help is really appreciated. If you feel like, please propose
some prioritization. Create the wiki pages about getting the windows
backtrace. Try to review, prioritize, and confirm new bugs. See
http://wiki.documentfoundation.org/BugTriage


>  Maybe, but changing to NEW is quite complicated 
> in your workflow, so mostly I leave them as UNCONFIRMED.

Heh, what is complicated about it? Is the documentation unclear? IMHO,
if you make sure that e bug is reproducible with the given information,
you could move it to the state NEW.


Best Regards,
Petr

=== Bug Severities ===
====Blocker====
* Prevents developers or testers from performing their jobs. Impacts the development process.
* (Documentation) Key documentation is missing for critical testing and review.
* (Security) An issue that blocks the completion of an SRB architecture and/or export review.

Examples:
* Unable to login
* Unable to performance certification tests
* Unable to update system

==== Critical ====
* Crash, loss of data, corruption of data, severe memory leak.
* (Documentation) prescribes or doesn't warn against actions that cause data loss or corruption.
* (Security) A CVSS base score of 5.0 - 10.0 is a critical defect.

Examples:
* Crash that is repeatable and evident to multiple users
* Memory leaks that lead to OOM errors during average use in one week or less

====Major====
* Major loss of function, as specified in the product requirements for this release, or existing in 
the current product.
* (Documentation) missing, misleading, inaccurate, or contradictory information to the degree that 
by following the documentation successful completion of fundamental tasks is unlikely.
* (Security) A CVSS base score of 2.5 - 4.99 is a major defect..

Examples:
* Prevents mandatory feature from working properly
* Feature regression from previous release

====Normal====
* Non-major loss of function.
* (Documentation) missing, misleading, inaccurate, or contradictory information in the 
documentation, but successful task completion is probable.
* (Security) A CVSS base score of 1.0 – 2.49 is a normal defect.

Examples:
* Prevents important or desirable feature from working properly

====Minor====
* Issue that can be viewed as trivial (e.g. cosmetic, UI, easily documented).
* (Documentation) contains stylistic or formatting issues, but functionality is not hindered.
* (Security) A CVSS base score of 0 – 0.99 is a minor defect.

Examples
* String typo

=== Bug Priorities ===

====P0 - CritSit====
This priority is reserved for NTS.  It is not used for defects associated with products in 
development.

====P1 - Urgent====
Use this priority for urgent issues

Examples:
* Blocker: Generally is a P1
* Critical: Nautilus crashing while opening a file for all x86_64 installations
* Major: Fingerprint support authenticates regardless of the fingerprint swipes
* Normal: Package management log does not get rotated (will get large fast)
* Minor: SLED is misspelled in bootsplash

====P2 - High====
Use this priority for mandatory defects, enhancements, and work items. That is, for items that  
must be resolved in this release.

Examples:
* Critical: Nautilus crashing while opening a file for all x86_64 installations over ssh
* Major: Fingerprint support (mandatory feature) does not work with gnome-screensaver
* Normal:  Package management system is not able to lock packages with regular expressions (but rug 
parity is needed)
* Minor: Notification about potential security issue is obscured on screen

====P3 - Medium====
Use this priority for desirable defects, enhancements, and work items. That is, for items we would 
like to fix, but we won't hold shipment for them.

Examples:
* Critical: Nautilus crashing while opening a file ssh for certain non-default configurations
* Major: Fingerprint support (mandatory feature) does not work with sudo
* Normal: Package management system is does not display correct progress
* Minor: Notifications do not wrap text properly and can be cut off sometimes

====P4 - Low====
Use this priority for optional defects, enhancements, and work items. This priority is not as 
strong as desirable.

Examples:
* Critical: Nautilus crashing while opening a file ssh for particular user with a provided backtrace
* Major: Fingerprint support (mandatory feature) does not work with sudo for users with complex 
configurations
* Normal: Package management system does not show correct icon for enhancement updates
* Minor: Notifications do not have the correct icon sometimes

====P5 - None====
Indicates that a priority has not been assigned.