From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jakub Narebski Subject: Re: Advertising the Git User's Survey 2011 Date: Sun, 3 Oct 2010 11:20:59 +0200 Message-ID: <201010031121.03795.jnareb@gmail.com> References: <20101002062056.GB16626@kytes> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: "=?iso-8859-1?q?=C6var_Arnfj=F6r=F0?= Bjarmason" , Git Mailing List To: Ramkumar Ramachandra X-From: git-owner@vger.kernel.org Sun Oct 03 11:21:09 2010 Return-path: Envelope-to: gcvg-git-2@lo.gmane.org Received: from vger.kernel.org ([209.132.180.67]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1P2Kkm-0006mK-67 for gcvg-git-2@lo.gmane.org; Sun, 03 Oct 2010 11:21:08 +0200 Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752850Ab0JCJUw convert rfc822-to-quoted-printable (ORCPT ); Sun, 3 Oct 2010 05:20:52 -0400 Received: from mail-bw0-f46.google.com ([209.85.214.46]:45243 "EHLO mail-bw0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752840Ab0JCJUu (ORCPT ); Sun, 3 Oct 2010 05:20:50 -0400 Received: by bwz11 with SMTP id 11so3009937bwz.19 for ; Sun, 03 Oct 2010 02:20:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:subject:date :user-agent:cc:references:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:message-id; bh=D/fP23R+NUy4L5UUMUrNuEtUArwIjGwvGENzdDG86U0=; b=MIiyGtE1OKakehdKGZOhnNWsaI4iUv01/6mCM4sGYsFD3oURTLnspBWYfXyH5fRKeJ hrKNqCRyrajTVv0DmUgVxyuCQQ+/wVlszzYKvAb71JxLTEULyicEkHbFIuWHyl0zVm0f AFhTzhowbLBd8Sali3MRz/HCoFXuRiASUVqRY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:cc:references:in-reply-to :mime-version:content-type:content-transfer-encoding :content-disposition:message-id; b=TnS8vtPNqC0LvDBrzXssQpnbiTe6a8Mk0rBeiyWoh2T0XOmYybBfwZExlMYzqMpvhS cJHS6RLHfugv8aZN0tppA1DOdPJn7IrVryf2g9F/+yhb/teYTJUxQHlQoRHsv5IJG+5I EOgQ+6Q4YL3eZ6JM1dvImvC4yLpQcgcz3OfZY= Received: by 10.204.120.79 with SMTP id c15mr5652943bkr.188.1286097649113; Sun, 03 Oct 2010 02:20:49 -0700 (PDT) Received: from [192.168.1.13] (abwc112.neoplus.adsl.tpnet.pl [83.8.226.112]) by mx.google.com with ESMTPS id g12sm2582745bkb.2.2010.10.03.02.20.45 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sun, 03 Oct 2010 02:20:46 -0700 (PDT) User-Agent: KMail/1.9.3 In-Reply-To: <20101002062056.GB16626@kytes> Content-Disposition: inline Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Archived-At: On Sat, 2 Oct 2010, Ramkumar Ramachandra wrote: > =C6var Arnfj=F6r=F0 Bjarmason writes: > > On Mon, Sep 20, 2010 at 13:38, Jakub Narebski wr= ote: > >=20 > > > Currenly the survey has more than 5000 responses (in a not whole = month) > >=20 > > That seems somewhat low, but maybe people just aren't that interest= ed > > in taking surveys. >=20 > Nice work on the survey! This is our best year by far. Some general > observations: >=20 > Interesting statistic: 24508 people viewed it, 7821 people completed > it, but 0 people started filling out information and later decided no= t > to submit it. It could mean that many people clicked through and foun= d > the survey, but probably left because it looked too long at a glance? Without the knowledge how those two numbers are calculated we can only speculate what do they mean. I think that '0' in 'Incomplete' statistics is here because this survey doesn't have compulsory questions: answering all questions are optional= , so it might mean that even if one question in the survey is answered, then the survey is considered complete by Survs.com statistics. I personally do not like the "wizard" formatting of surveys, i.e.=20 dividing survey into page so you are not presented with very long page, but are presented withc chunks of survey at glance. Even if you see how much survey did you fill in (how many pages there are in total), and even if you can go back to previous page. I'd prefer to create a better information about survey upfront. We say that all questions are optional ("Note that you may skip questions as you like"). It is also stated that you can fill only a part of survey, and later go back to finish it... hmmm, I wonder if those cases where one edited his/her survey responses multiple times are counted as one finished survey, but multiple views. We could also write how much time it takes on average to fil the survey= =2E > The average time spent on the survery is 34 minutes It would be interesting to have more detailed statistics of time spent on the survey that only average time, a single number. When one is filling open-form essay-length question, it would obviously take much more time than for one who doesn't. But Survs.com currently doesn't provide it. I can try to ask for it, though (via feedback). > - I think we can bring that down to 10~15 minutes if we design > questions to extract more information. Also, there's little incentive > for taking the survey: while many companies actually give out > discounts/ coupons for taking surveys, the least we can do is present > real-time results in the most interesting manner possible ie. survey > takers should see the "results so far" immediately after taking the > survey; some visualizations such as pie charts? What we can do is after finishing the survey to redirect to the survey analysis page: https://www.survs.com/results/33Q0OZZE/MV653KSPI2 http://tinyurl.com/GitSurvey2010Analysis instead of IIRC currently used redirect to https://git.wiki.kernel.org/index.php/GitSurvey2010 As far as I know Survs.com doesn't provide any API for extracting data or survey statistics required for creating such visualization. Neither we have a place where such app could be created, I think. >=20 > In questions 5, 10, 12, 13, 16, cut down out the options that have > very few respondents and let them all go into "other". It probably > doesn't actually save the survey taker any time, but I think seeing a > long page with many options can be scary. The "5. Which Git version(s) are you using?" is not that long. We coul= d create a cut off a bit earlier, perhaps on 1.4 (i.e. have "pre 1.4") or even earlier, we could remove alternate implementations answers=20 (git-bigfiles, JGit, other implementations), or even concatenate 'maste= r', 'next', 'pu' into single response... but would it buy us much? The other side of removing choices, relying instead on "other, please specify" response is that it makes it harder to analyze results of surv= ey: different people use different words for the same thing (and there are also spelling mistakes), and make results less reliable: people do not fill "other" if there is at least partial match, or do not know how to specify their version. In "10. What Git interfaces, implementations and frontends do you use?" we can remove tools marked as deprecated... if not for the fact that it is actively interesting to know how many people use such deprecated tools. Besides, the list of answers to this question is not overly lon= g, I don't think. We can remove those choices in "12. What Git GUIs (graphical user interfaces) do you use?" that got less than 1% rounded, or less than 10 responses. On the other hand some people stated earlier that the list of possible choices in the survey (not necessarily about this question in specific) serve as reminder / information about possible choices. The other side of removing options from "13. Which git hosting site(s) do you use for your project(s)?" is that when sending requests to=20 announce the survey to those git hosting sites that are not on this list, some of them requested to be added (which is impossible after starting the survey; and before survey begins it is little sense to send announcements). Besides all of those below 1% rounded (Codesion, GitFarm, The Chaw, CipherHive) are also those that I didn't get response to request for announcing Git User's Survey 2010... =20 We could make it more organized though, e.g. by sorting list of options alphabetically, or something like that. Removing options from "16. Which of the following features do you use?" would make it harder to analyze and less reliable. Especially in this question different people consider different features important enough to mention, and describe feature in many different ways. Besides, each option except of "git cvsserver" got more than 1% rounded= , and having "git cvsserver" is interesting on its own (perhaps in other question?). > 1. Country of residence: we can probably make this a nice click-on-ma= p > interface as opposed to freeform text. It'll be more useful to us, > and more interesting to users when we advertise the results. It would be nice to have click-on-map (Google Maps or Bing Maps based), something like Ohloh provides, resulting in map of survey responders similar to the map of git users and git contributors on Ohloh http://www.ohloh.net/p/git/map it isn't something that Survs.com offers currently. I can only ask for it to be provided... Another solution would be to have pre-filled combo box (