There was a request to share some research from Bell Labs, the research division of Alcatel-Lucent. The paper from the Bell Labs Technical Journal is called Questions about Questions, and it is an in-depth study of how adult knowledge workers go through the process of asking a question and getting an answer. I wrote a cliff notes version of the paper for our employees because some of the research is based on Engage - Alcatel-Lucent's employee community which is Jive straight out of the box - no customizations. For some background, our community is for employees only. We have roughly 72k employees globally and 99.x% adoption. A typical month see 25k members actively logging in and almost everyone who logins (there's usually between 5 to 20 people in a month who are the exception) does some other interaction besides reading. There are over 210k pieces of content in the system. As you will see in the paper or notes a question is the true dictionary definition of the word and not just the construct of a discussion marked as such within Jive. If you want to read the whole paper it is worth a read, but if not here's the executive summary.
"To date, there have been no studies that characterize the types of questions knowledge workers ask in the course of their daily work. Further, there has been no systematic investigation of the means that they use to answer those questions."
The paper looks first at results of 8 focus groups where "participants were asked to:
1) recall and write down recent job-related questions for which they had sought answers;
2) talk aloud through the details of the steps they had followed in attempting to find answers to selected questions that had been recalled in the first step; and
3) brainstorm categories or properties that could be used to distinguish some questions from others."
The focus groups produced a sample of 90 recent questions 60 of which were technical in nature.
The focus groups described 82 properties of questions when asked, but their overall responses to the focus group questions indicated that questions are far more complex and really have a set of dimensions where different questions can be placed.
These dimensions are shown in Figure 1 on page 46, but are:
-Knowledge needed for answer
-Depth of answer needed
-Question asker's expertise
-Expectation of success
The process of just asking a question was surprisingly complex as well which is often iterative.
A diagram of the overall Q&A process with all the dimensions on p 47.
There is a table of design implications for social Q&A applications on p48 I recommend looking at.
The second half uses the open/members-only content on Engage (jive) as a real living lab. Most people who ask a question in a discussion don't actually mark it as such. Basically, Bell Labs built an algorithm that gets 80% accuracy deciding if discussions are indeed questions.
They looked at SNA analysis with particular interest in people who were "connected" because the person had replied to the other (no not necessarily friends or following each other). The structure in this enterprise social network is similar to public social networks. They also looked at the graph that is just for questions. "Perhaps the most interesting thing to note across these different metrics is how similar the two networks are, despite the question graph having almost twice as many nodes as the non-question graph and roughly two and a half times as many edges."
"Non-questions have more communities and more weakly-connected components than questions. Both of these measures could be taken to indicate that there are more clusters or cliques of users for non-questions than for questions. By this reasoning, when someone asks a question through Engage, they are likely to get replies from a wider spectrum of other users than if they start a non-question discussion. As we will see below, questions tend to get more replies and more people replying than non-questions, so these factors are all inter-related.
Stats on the SNA for questions & non-questions and number of repliues are on page 53 & 54.
"...non-questions are almost twice as likely as questions to receive no replies at all. In fact, over half of non-question discussions received 0 replies." So if you have a question it pays to mark it as such. Around page 55 there is also stats on time to first reply and last reply if you are interested.
"It appears that questions have a shorter, but somewhat more intense (in terms of number of replies and the time to the first reply "lifespan" than non-questions. Once answered, question discussion "decays" more rapidly than non-question discussion."
The study looks at tagging from questions to non-questions which I didn't find of particular interest.
"From Figure 7, we see that the main difference between questions and non-questions occurs in the ranges of 50 and 100 words, where more nonquestions than questions have 50 or fewer words in the initial post, while more questions than nonquestions have initial posts containing between 50 and 100 words. One possible explanation would be
that users write slightly longer posts for questions as a way to try to convey the question clearly, provide context, or break up the question into parts."
More people partake in question discussions. "The mean number of people contributing to question discussions is higher than for non- questions, with a mean of 2.1 compared to 1.3 (t(6862) =10.151, p < 0.001). As we saw earlier, questions receive more replies than non-questions, so it is not surprising that there are more people replying to questions than non-questions."
At one point they note that people are either askers or responders in general.
"an interesting bifurcation of user-posting activity, where a sizable proportion of users posted only questions or only replies to questions, but not both questions and replies."
Generally, nothing was conclusive when looking at geography specific data.
If you want to read just the summary & conclusions it starts on page 59.
-Q&A is a central part of daily knowledge-work
-knowledge-worker questions are complex & can be describe by scaled dimensions
-they created a process model for the interative strategy used for questioning
-They came up with design implications based on all of these.
(All of this is covered above)
-"The reply graph for questions contained a core of densely connected high-degree nodes, with other strongly clustered, low-degree nodes at the boundaries of the graph, indicating a "small world" effect for this graph."
-questions get more replies, faster first replies, have more people participating
-At least for Engage the mean time to first reply is 3.5 hours
-questions get less tags & the conversation ends faster than non-questions
-people are either discussion starters or replies but tend not to be both
-"While there were differences across countries in terms of how likely people from that country were to start questions, it was hard to discern any real pattern across geographic regions."