7 Replies Latest reply on Jan 5, 2012 3:13 PM by Andrew Kratz

    How Do you Measure Success?

    chconkl

      I've seen many people asking this question, but I haven't seen a really tangible example.  We're launching at the end of February and beginning testing in about a week.  Is there any kind of common curve I can look at/show as an example?

       

      Thanks!

        • Re: How Do you Measure Success?

          Hi,

          I ask each of new community owners that question before we create their community space.  For some communities, such as internal com, page views work.  For communities that focus on a workspace for the team, activity (number of new discussions, number of replies, active members, etc) are more important.

           

          You need to connect the community with a business goal.  That should provide an indication of what to report on.  If the community is replacing some other tool, the transition and adoption numbers will be important.

           

          It is one of those, "It Depends" things.

          Mike

          • Re: How Do you Measure Success?
            Andrew Kratz

            I have been giving a lot of thought to this question recently.   So the answer is quantifying business value.  You can demonstrating this with the analytics tools and surveys of your users to capture anecdotal evidence of value. 

             

            Some of the specific reporting we have created or are working on:

            • Engagement - By the entire company/business unit show the % that have done at least some minimal activity.  Number of active groups(activity within a month), executive usage (isolate execs as a group to protect privacy) to demonstrate their involvement which is a key to success.
            • Hard Core Business - Show the % of business content versus soft content (water cooler issue) for us it is 85%...many of the non-believers are not in private groups so they only see water cooler and think the platform is a toy…at least for us we work hard to combat the whisper campaign.
            • Key tool to do business - Categorize the ski slope graph of all users and their total usage.  Today its less than 1% of users leverage Jive for all (or nearly all) day to day activities.  As a baseline how does that number move up over time.  If we break our users up into quartiles how do each of these buckets improve.  So Q1 might be "heavy users", Q2 "use often" down to a Q5 "infrequent users". 

             

            For these items I would like to have us set goals or targets to reach.  I am not sure how impactful that would be (who would know what a good target is if we set it or made it?  Would it mean enough to management?)  We certainly can show progress and growth against these values, that I know..  

             

            I posted one report example we use that demonstrates one or two of the above.  When I get more time I'll try to share more report examples. 

              • Re: How Do you Measure Success?
                nbussard

                Although some groups will have very specific measures for success based on their goals, we used the Net Promoter Score survey that I learned about from Brice Jewell and Ted Hopton in their session at Jive World as a general measure for the overall community. You can find more information on this in the Analytics community and also through an internet search, but the gist of it is that you ask a single question, "How likely are you to recommend to a colleague?" as indicated on a scale of 0-10. The NPS is calculated by subtracting the number of detractors, defined as scores from 0-6, from promoters, defined as 9 and 10.

                 

                We also added questions to segment by tenure, business unit, and career stage, as well as an open comments box. We will be looking at whether that number increases over time, but what is more interesting is seeing how the numbers break down across various groups. When we have those numbers, we can dig deeper into why some groups prefer it over others, and where we have successes that can be replicated elsewhere. We also categorized all the open-ended responses into big buckets to see the most common themes. We'll use this to drive priorities for the next several months.

                 

                Although we do track participation and adoption, generally speaking our goal is continuous improvement, rather than hitting a specific metric.

                 

                Nikki

                  • Re: How Do you Measure Success?
                    Andrew Kratz

                    Nikki,

                     

                    I like it for groups where people came to your social network by choice.  In our case we are using jive as a social intranet so we have a rather large population that feel they were forced into this new world.  Academically, it would be an interesting result to get that baseline of net promoters and even though it would be highly negative we could watch it get better.  Practically, that type of score even if it was improving would seem very negative for quite some time and be a bit of a black eye.

                     

                    But again, I know most have their Jive site as an optional item and you would think that those that participate are willing and excited so the score there would make sense.

                     

                    I thought I recall that you too have an Intranet situation with your Jive site, no?  Is that a concern for you at your company?  Or maybe you just have a more socially savvy population?

                     

                    Thanks for the thoughts!

                     

                    Andrew

                      • Re: How Do you Measure Success?
                        chconkl

                        We are also using jive as a social intranet and I anticipate the "forced" reaction as well as our population may not be particularly tech savvy.  Every time I talk about the Intranet I get this worried "how will you train everyone?" response.

                        • Re: How Do you Measure Success?
                          nbussard

                          Andrew, we actually have a separate intranet. We use it for policy & procedure and internal comms, and we use Jive (we call our community Vox) for what I recently summarized as the place to "get and give solutions, inspiration, examples, information, advice and opinions about projects, technology, design, development, trends, clients, processes, sales and our company from other Sapient people." In other words, it's focused on our client work, is peer-to-peer, and is social, versus the internally-focused top-down intranet. So, people don't have to use it, but I think we are not different from many companies in that users complain a lot about the tools provided for them, regardless of whether they are required.

                           

                          It's interesting that you think it wouldn't be useful because it would be negative. We have a lot of hurdles to overcome yet--most notably we don't have single sign-on, which makes people really irritated—so we weren't looking for, nor did we receive, all rosy feedback. What was interesting was seeing how the scores broke down across different groups. For example, Senior Managers and Directors had the highest NPS scores, whereas Managers had the lowest. Based on that, we are looking to leverage the senior leaders to influence the manager group and help them see the value. Similarly, we have one business unit which has fully embraced Vox and has been actively promoting it through their leadership and various campaigns, whereas another hasn't done anything formal at a BU-wide level. The BU promoting it has better satisfaction scores.

                           

                          So all in all, while it's scary to think about asking people to do what feels like rating how much they dislike you, it's very useful to see who is more in your camp and who is less so. In fact, the most eye-opening thing for me was seeing all the "negative" feedback coming from people who had 9 and 10 NPS. It seems that people who are passionate about using the tool are also passionate about making it be the very best it can be—they are just as vocal as the grumblers. This leads me to believe that you guys might be surprised by the actual results, if you're estimating NPS based only on anecdotal feedback.

                           

                          Cheers,

                          Nikki