Metrics and goals are good, but don't underestimate the qualitative value of users who are so addicted to your pilot site that they will take up torches and pitchforks and demand that the organization fully adopt the tool after the pilot is over. That's essentially what happened to us...
Hi Damon – I’m in somewhat the same boat as you. We’re not rolling out one “master” internal community, but rather a collection of small spaces that are targeted towards individual groups within the company (HR, engineering, etc.). We have a brief checklist that we go through before setting up a new space – things like who the business owner is, what their success criteria is, etc.
There are some baseline metrics that are good for all communities – things like number of discussions/blogs/documents, number of registered and active users, etc. However, I think that other metrics are very highly situational and will change based on what a specific business owner needs.
If you’re interested in our checklist/template, let me know and I can send it.
Jim, We too are rolling out a master community as well as some spaces targeted to HR, Labor Relations, Engineering, etc. We just started the pilot phase in January. That is what I am trying to accomplish within the next two weeks:]
What are some success criteria for the pilot?
What are some baseline metrics that other communities use to measure success within a pilot and production?
What criteria would be good to use when deciding if there is a need for a Space or a Group?
I realize that metrics will change we we move forward and mature. But I would like to have a handful of metrics to at least help me to measure the health of our community. I am very interested in your checklist. I am also interested in keeping in touch....sounds like we are at very similar stages.
I copied/pasted the contents of this doc below (I'll also send you a copy out of band, so you have it in Word format). Keep in mind this is just a rough list, and it may not be 100% applicable to your environment - but you get the general idea. Our potential users (and even some sponsors) internally had this mentality of "If you build it, they will come." This list really forced everyone to think about why we'd do a specific community, what the end goals are, etc. We even had a few people say "Oh...nevermind" after we gave them the list - because they thought it through, and decided a community wasn't going to solve their problems.
Anyway - please (anyone!) let me know if you have questions on this.
- Goals of the community
- Metrics for measuring success (short term vs. long term)
- Community Manager (driver of forum)/Czar (in charge of administration)
o Existing groups (email aliases)
o Support, Engineering, Marketing, etc
o Identified Wind River champions (experts, engineers, product managers, etc.)
- Types of content – white papers, minutes, discussions, projects (short term vs. long term)
Review of working communities:
Networking and Telecom Solutions, Dev Tools Product Management, Compiler, VxBus, Networking, Components, and Middleware
1. Create a community name
2. Create HTML email to audience announcing the site
3. WebEx about the new community (functionality and forum specific)
4. Create fun ways to use the community and simple instructions for how to sign up/follow
o Use video (screen shots/music/audio)
5. Schedule of planned topics for discussion that may be a video, podcasts, stories, live discussions, new announcements, or hot topic . A featured expert bi-weekly or monthly.
6. Member participation – Motivate with recognition and reward
o A complete user profile (photo, bio, etc.) earns a reward
o Content or topic sharing (frequent flier)
o At registration, ask users to provide topics of interest
7. Promote becoming a specialist
o Become a specialist in different areas by passing a test/ earn credentials in a specific area. Monthly drawing to get winner and announce new specialists. (see Cadence)
8. Community 2 Community promotion – When reaching out to staff, promote on other internal sites like CSO Twiki
1. Recognize first time contributors when they make a post– have a monthly drawing and give away rewards
2. Spotlight high contributors/ show ranking and earn points for each posting- create monthly drawing for top contributors
3. Create value - be consistent with daily and weekly Wind River contributors early on- ask questions to the group and generate discussion.
4. Design goals and yearly achievements around contributions
1 person found this helpful
I think that metrics change over the lifecycle of an internal community. Metrics are always difficult, but I don't often see references to the used of graduated metrics that rely on the lifecycle of a space.
In the beginning, you might want to measure how many unique users you have been able to attract to the space (registered users). The primary goal, in the beginning, is to get people to participate (IMO). No pilot can be successful without people, and you'll need some well defined strategies to engage people to even come to your space (if you build it, they won't necessarily come :-)
As you move out of the pilot phase and into 'production,' metrics can become more refined to include the quantity and quality of discussions, document sharing etc.
1 person found this helpful
Michelle - thank you for your reply and thoughts. I do agree that metrics will change over the lifecycle of the internal community. I do believe that metrics will be difficult to begin. You have given some good suggestions for a initial metric set. That is what I am looking for ....anyone else have some ideas??
There are really a lot of different metrics you could use. Does your pilot have specific goals defined or challenges it is meant to try and resolve? I agree with Michelle that the starting point is just whether people are in and using it. You can count new members, trend the amount of activity in the community week to week or month to month, measure the number of viewers to creaters, trend the average number of contributions per user, or if a business unit wants to move all of its communication to a group/space then you can measure what percentage of the BU has joined or is receiving email notifications, etc.
Business value metrics are really tied to your pilot goals. Some of that can be gauged by survey questions. Highly recommend you send the survey out now so that you can baseline it before your community is put in play. There are a quite a few example for survey questions you can glean from the survey results Jive just announced. You can also ask other questions like
- asking users how they rank the various sites in your company as their preferred source of information or trust or responsiveness
- how much they agree to statements like "I use this community to solve a problem", "I use the community to find experts in the company", "I use this community to stay connected to others in the company", etc.
- Or ask for free text responses on how they use the community and quantify those into categories when doing your analysis
Other metrics could come from specific business units looking to use a group in a specific way, such as:
- decrease use of email DLs
- Get answers faster - % of discussions marked as questions have correct answers indicated within X days
- decrease in issues logged to your formal help desk
- Innovation - count of the number of ideas generated, count of the number of people submitting ideas
- connecting a global workforce - number of geographies/work locations with membership in a group (or the entire community)
- Making information more accessible - comparison of the number of views an item got in the community versus where it previously lived (e.g. intranet)