Re: [libreoffice-design] Re: Calc Survey

Hello,

My thoughts on the survey:

1) GENDER

> If it is really-really necessary to ask…

I think it makes sense to ask for gender and age – if your survey only 
reaches male users aged 20-30 you want to know, as results will apply to 
this group then, but possibly not generalize well to other parts of the 
user community.

> discouraging for non-binary people

Indeed – I would suggest the following question:

My gender is…

- Female
- Male
- Non-binary
- Prefer to self-define: _________

"other" is not considered good practice – needing to label oneself as 
belonging to what is marked as residual category.

An interesting and and hands-on overview of gender questions in survey 
can be found in Spiel at.al, 2019, acm interactions [1] though their 
suggested method might be a bit hard to analyze.

2) AGE

Stopping at "over 55"seems pretty young for an upper end. Quite some 
people who are in their mid-60s today will have used computers when they 
were 20 or so in case they went to university.
I recently did research in a relatively young community but "over 55" 
would have been the largest group, as the whole tail of the distribution 
would get in there, so just extend the scale.

4) FREQUENCY

I agree with Csongor here: I would suggest to ask for "once a month", 
"weekly" etc. here

An overview of these question types is at 
https://www.qualtrics.com/blog/measuring-behavioral-frequency/

13) SUS

In general, I would not change the wording or scale except we know that 
it is OK to do. Many changes one can and can not do to the SUS are 
researched.

The positive/negative reversal was researched and it turned out that you 
can, or rather should, use an all-positive version: "Researchers  who  
do  not  have  a  current  investment  in  the standard   SUS   can   
use   the   all   positive version   with confidence  because  
respondents  are  less  likely  to  make mistakes  when  responding,  
researchers  are  less  likely  to make errors in coding, and the scores 
will be similar to the standard SUS" Sauro, Lewis "When Designing 
Usability Questionnaires, Does It Hurt to Be Positive?"

Another change that is supported by data is replacing "cumbersome" with 
"awkward": James R. Lewis, Jeff Sauro:  The Factor Structure of the 
System Usability Scale,  2009

– Jan

[1] "How to do better with gender on surveys: a guide for HCI 
researchers" 
https://www.researchgate.net/publication/334073420_How_to_do_better_with_gender_on_surveys_a_guide_for_HCI_researchers


Am 06.07.2021 um 16:24 schrieb Csongor Halmai:
> Hi All,
>
> I wrote my response before reading the response of others and there is 
> a little bit of overlapping. Sorry for that.
>
> Here are my thoughts.
>
> 1.
> I don't think the first question helps too much about the future of 
> the LO. It is very discouraging for non-binary people. If it is 
> really-really necessary to ask, let's put it to the end of the list. 
> But the best would be to drop this question completely.
>
> Similarly, the age also should be asked only at the end of the survey.
>
> 2.
> A much more relevant question would be how long the person has been 
> using computers. Are you 30 and started at the age of 10, or are you 
> 25 who started at 24? 30-10=20 is much more than 25-24=1.
>
> 3.
> Instead of asking the age with non-continuous intervals, a much 
> easier-to-grasp question would be "which year you were born in?".
> - 1966 or before
> - 1967-1976
> - 1977-...
>
> This is easier than expecting them to decide if 34.5 belongs to the 
> 25-34 or 35-44 group.
>
>
> Here above I just converted the age limits to years but round year 
> numbers are probably even better:
> - before 1960
> - 1960-1969
> - 1970-1979
> - 1980-1989
> - 1990-1999
> - 2000-2009
> - 2010 or later
>
> 4.
> The optional answer for the usage frequency are very hard to 
> understand. Do I use LO Calc "All the time" if I use it every time 
> when I need it, two times a year? Instead of these soft categories, I 
> would be more exact:
> - less than once in a year
> - a couple of times in a year
> - a couple of times in a month
> - cca once a week
> - almost every day
> - several hours each workday (at least 4 days a week)
>
> 5.
> The answers for Q7 (What is the size of your typical LibreOffice Calc 
> dataset?) will be very hard to process. Instead of this opened 
> question, I wold ask something like this:
> - How many sheets you normally use in a Calc spreadsheet? Categories: 
> always 1, 1-3, 2-5, more
> - How many rows do you use in a Spreadsheet? Categories: <50, 50-500, 
> 500-5000, 5k-50k, more
> - How many columns do you use in a Spreadsheet? Categories: <10, 
> 10-20, 20-50, more
>
> 6.
> The row labels "Select 6 as youranswer choice" and "Select "No Answer" 
> asyour answer choice" for Q9 seem to be a mistake.
>
> 7.
> Typo: "Liner and Non-Linear Solvers" => "Linear and Non-Linear Solvers"
>
> 8.
> "Please select "No Answer" if you are not familiar with a particular 
> feature."
> There is no "No Answer" column.
>
> 9.
> For Q13, the row labels could be simplified if the introduction would 
> be rephrased.
>
> Instead of
> "Please choose the appropriate response for each item"
> I would write
> "To what extent do you agree with the following statements"
>
> After this, the row labels can be simpler:
>
> "I think that I would like to use LibreOfficeCalc frequently.
> =>
> "I would like to use LibreOffice Calc more frequently."
>
>
> 10.
> It's hard to switch between positive and negative statements. Say, if 
> I love LO Calc then sometimes I need to tick the leftmost circle, 
> sometimes the rightmost one. This is a bit hard.
>
> It would be better to rephrase the negative statements to positive 
> ones. For example:
>
> "I think that I would need the support of atechnical person to be able 
> to useLibreOffice Calc."
> =>
> "I can use LO Calc without the support of a technical person."
>
> In this case it is more visible. If all the ticks are on the right 
> side then everything is good.
>
>
> 11.
> After rephrasing the second statement "I find LibreOffice Calc 
> unnecessarily complex", we get the third one: "LibreOffice Calc is 
> easy to use". I would remove the second one and keep just the third 
> one. Why would we ask the same thing twice?
>
>
> 12.
> Probably this is just a draft-draft version but the final version 
> should be typographically perfect. It would be a big shame for a 
> Document Company to share a PDF in which the cell content doesn't fit 
> into the cells. Example: "1 -Not Important at All" in Q9.
>
>
>
>
>
> These were just a rough list I could put together quickly but I am 
> sure there would be a lot more things to change after trying to 
> actually answer the survey and then evaluating it too. Therefore, I 
> suggest making these changes, ask 10 people to answer it and then try 
> to evaluate it. Probably we will see that they answer the questions in 
> a way we cannot interpret, something it not clear, something is hard 
> to process.
>
> Otherwise, this is a very promising survey.
>
> Cheers,
>
> Csongor
>
>
> On 1/07/2021 02:35, Heiko Tietze wrote:
> > First reply from Stuart (per direct mail):
> >
> > > "Two confusing entries on page 5 of the draft survey:
> > > Select 6 as your answer choice Select "No Answer" as your answer choice
> > > I think they need to go to have valid results (and force respondents 
> > > to answer each question)."
> >
> > Yes, the 'Select "No Answer" as your answer' choice slipped 
> > accidentally in.
> > Also the item 'Select 6 as your answer choice'.
> >
> > Second from Kompilainnen (on Telegram):
> >
> > > * Q1: Drop question about gender (answer has no value for the results)
> > > * Q3: Add description like 'Basic - typing data, simple functions 
> > > like SUM only', 'Intermediate - Basic + use Conditional formatting, 
> > > Charts, more functions, like VLOOKUP, AVERAGE, SUMPRODUCT, etc', and 
> > > 'Advanced - Intermediate + use many different functions, Pivot 
> > > Table, write some macro, create work templates, etc'
> > > * Q4: same here, 'Never - 1 time per month - 1 time per week - 1 
> > > time per day - It's my main work tool'
> > > * Q7: doubt this result in data that can be evaluated; would rather 
> > > ask both number of cells and size in two questions
> > > * Q8: could benefit from a link to
> > > https://wiki.documentfoundation.org/Documentation/Calc_Functions/List_of_Functions/en 
> >
> > I agree with removing Q1, don't see why a gender should have an 
> > impact on Calc usage. But would keep the question a bit vague on Q3 
> > and 4. Also looking rather forward to analyze Q7, the results from 
> > the Draw survey on "what size does your document have in average" 
> > [1,2] shows the different view on this topic. Surely, it's still open 
> > to challenge.
> >
> > [1] 
> > https://design.blog.documentfoundation.org/2016/04/01/the-many-faced-god-part-1-how-libreoffice-draw-is-being-utilized/
> > [2] 
> > https://design.blog.documentfoundation.org/2016/04/01/the-many-faced-god-part-2-how-libreoffice-draw-is-expected-to-evolve/

--
To unsubscribe e-mail to: design+unsubscribe@global.libreoffice.org
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/design/
Privacy Policy: https://www.documentfoundation.org/privacy