10 Replies Latest reply on Jul 12, 2012 1:37 PM by bchamberlain

    Calculating Reach

    Ted Hopton

      I'm building a leaders leaderboard -- called the Leaders Board -- for our CEO to display how senior leaders in our company, and their teams, rank in terms of activity and engagement in our internal Jive community (the Hub). Although the number of these leaders is relatively large (18), it's small enough that I am willing to undertake some manual data collection processes so I can make this report meaningful.

       

      I came up with a new metric I'd like to share here, and I'd like others' thoughts about it, including suggestions for improving it or criticisms of it. I call it Hub Reach, because our community is the Hub, but we can refer to it simply as reach. It is a highly imperfect metric. I am well aware it has many flaws. But it's the best I can do right now with the limited data I have available to me.

       

      Reach as a Metric

      Intent: reflect the reach that an individual has within the community. This is a measure of influence. Reach very roughly approximates the number of people that a person reached within the community.

       

      Components:

      • count of content authored (from Content section of Community Manager Reports, filtered by username)
      • total views for content authored (manually tallied by scrolling the Content tab, filtered for Authored, of each person's profile)
      • count of content participated (manually tallied by scrolling the Content tab, filtered for Participated, of each person's profile. This is usually comments on content, but also includes voting in polls -- perhaps more is in this?)
      • number of followers (manually recorded by looking on the person's profile)

       

      Calculation: Reach = Followers * (content authored + content participated in) + views of content authored

       

      Rationale for Calculation

      Views of content are the number of people who consumed it directly in the site (this omits people who got email notifications and read it without ever clicking through)

      Followers * (content authored + participated in) is only a rough estimate of "reach" in terms of how many people may have been reached. It is inaccurate and problematic because

      • we don't know how many of the followers
        • get the content in email notifications, nor how many of those only read it in email or how many click through to also view it
        • track in communications, and so may get it in email (with same issues cited for email, above)
        • get Followed activity in email (same issues as above)
        • read the content in their Followed Activity stream without expanding it (which is required to register as a view) -- for short items this may not be necessary to view it
      • many Followers may not see the content at all, as it goes past in their activity stream and they don't notice it -- so multiplying Followers times content is likely hugely overstating the number who actually SAW the content
      • we don't know how many people are following the container where the content was posted. This reach calculation omits those numbers, thus under-representing the actual number of people the content "reached." And then we have the same issues for those container Followers as listed above for Followers of the author.
      • Simply using the count of content participated in gives no information about how many views the comment may have gotten (beyond the commenter's own followers). An early comment will get fewer views than a late comment on a very active thread, because all the earlier posters will be notified about it.

       

      My Conclusion

      As long as we acknowledge this is an imperfect metric and don't try to represent it as the number of people who actually read what the author posted, then I consider it a decent way to compare the relative reach and therefore influence of individuals in the community.

       

      My Wishlist

      If others agreed that this were a worthwhile metric, I'd love to see Jive include it in the Community Manager Reports so I don't have to do all the manual work to calculate it!

        • Re: Calculating Reach
          nbussard

          Hi Ted,

          I love this idea. One component I see as potentially missing is amplification and acclaim. Amplification of your content or profile could be resharing, linking, @mentioning, bookmarking, etc. I think bookmarks are especially important because if people bookmark your content, that is a clear indicator that they find it valuable, even if they don't come back to it. I'm pretty sure you can get bookmarks from the analytics database, so this may be an easy one to add in.

           

          I think it would also be nice to add in the number of likes. I would not worry about ratings, because they are so subjective, but likes are also a form of acknowledging your reach and amplifying your content as the Like goes into the activity stream.

           

          I'd love to see Jive make this kind of thing available. I also wonder if it can be achieved through one of the gamification apps. I haven't explored those, but there may be a way to set status points as a combination of all of these things and calculate a leaderboard.

           

          Nikki

            • Re: Calculating Reach
              Ted Hopton

              Hi Nikki,

               

              Glad you like! I don't think any gamification app is going to approach this level of detail, but perhaps I am biased.

               

              Amplification is an excellent thing to consider adding. I say consider because with my current set of data tools it would mean more manual tallying of data and perhaps diverging from my simple "reach" concept. But, in terms of dreaming up what Jive could build as a way to measure influence, I think you have added some valuable components for that calculation.

               

              In the here and now, looking at Content on each profile, the same place I find the Views data, I can see Likes, Bookmarks and Comments. So, I could manually collect those counts for each piece of authored content (oh, joy!). I have no idea how to get any of that, including Views, from SAP BO. I can pull View activity counts for a person easily enough, but I don't know how to create a report that counts all the views that content created by a specific person got. Maybe it can be done, but not by me -- not in my skillset. Same would go for the other elements you suggested, even if they were all in the analytics database.

               

              But what would the calculation be? And what would the result really be?

               

              For my simplistic Reach metric I can say it's an attempt to reflect how many people the author reached. I am not sure what to do with Liking data -- it seems a stretch too far to say that all the followers of the person who clicked Like have been reached. Yet, potentially, that is exactly what happened, so we could make a case for including it in the Reach metric. Not that I have time to manually chase down those numbers, anyway!

               

              And for Bookmarks, you could say the same thing, as it should appear in the bookmarker's followers Followed Activity stream, AND if we knew how many people viewed the bookmark... again, I don't have time to chase these numbers manually.

               

              On the other hand, the really powerful multiplier in the Reach calculation is the number of Followers. Adding a few bookmark and Like counts sounds initially trivial, but when you multiply them by the Followers there could be a sizable effect. The catch is, I don't know the number of followers each person who takes those actions has (liking, bookmarking, commenting).

               

              So in my current world, with limits on the data I can collect, I don't see a way to add those amplification activities into my Reach formula. I can't multiply them by the author's followers -- when someone else likes my post, my followers don't get notified. And I don't know how many followers each person who takes an amplification action has.

               

              But if Jive were to build this, and do it right, yes, it should account for the number of followers the person liking, bookmarking, and commenting has. And you could do the same kind of math to include the number of people shared with when that happens, and the number of people exposed to an @mention. (The mentions would be really hairy to calculate -- if I also mention a group or space or several of those, all of those people are now part of the reach of that message.)

               

              OK, Karl Rumelhart, we have come up with enough to keep your team busy for a while

                • Re: Calculating Reach
                  nbussard

                  All valid points, Ted. I guess I was thinking this could be a calculation for reach as well as something like influence or klout. But the reach metric has a ton of value in itself. I've recently been talking to various folks about helping leaders use Jive effectively, and this is a measure that I think would help stimulate them to action. I think some people feel as though they create content and it just goes into the void, so they stop doing it. If you could show people this metric and give them the goal of simply growing it over time, it would feel much less overwhelming. In other words, it would take the burden off of the goal to "get everyone to read my posts" or similar, and more towards seeing how they could get a higher number than they got last month or last week.

                   

                  The fact that people's comments on other posts are considered in the metric is also really helpful when trying to talk to leaders, because they tend to only initiate posts, not engage in conversation, and then they wonder why people don't leave comments. I think people are less willing to leave comments if there is no existing dialog going on anywhere else (in person or online), so I am always trying to get our leaders to humanize and more fully define their personas by dipping in to other conversations. Having those simple interactions roll up in this number, which they can then compare among themselves, can help.

                    • Re: Calculating Reach
                      Ted Hopton

                      I agree this could grow into something more nuanced and that would be exciting. I have not rolled it out to our leaders, yet, so I don't know how they are going to view it. Probably with suspicion if they have a low score and delight if they have a high score!

                       

                      Completely agree that I want them to learn how to grow this over time, and hope to teach them the value of growing their network of followers -- not in the stupid way some people collect LinkedIn connections just to have a big number, but to instead understand that when they earn followers it expands their reach, and the best way to earn followers is to post things that people find valuable.

                       

                      Comments are huge, you are right. I see more value in comments by leaders, honestly, than in any other kind of content, if I had to pick just one thing for them to do. People love it when their leaders take a moment to acknowledge what they have posted with a comment. Leaders can influence far more people with 30 minutes of reading and commenting concisely than they can by spending 30 minutes composing a blog post (but they should do both!).

                      • Re: Calculating Reach
                        Ted Hopton

                        Just had another thought on terminology around this, Nikki. My original metric could continue to be called Reach and the metric you have proposed could be called Amplified Reach. So, it would build upon the Reach metric to account for amplification effects.

                  • Re: Calculating Reach
                    bchamberlain

                    Ted -- love the idea you are putting out there! I think the discussion points you and Nikki have made are good and I think it is something that could be tweaked over time.

                     

                    In theory, this is something we could automate into the framework we have built. A couple thoughts:

                    • It would be simple to load the identfied leaders into a group that to display on a dashboard. As those leaders change, it could be as simple as updating an Excel spreadsheet.
                    • I believe everything identified in your metric is an activity in the Jive analytics database. So it is just a matter of identiying those activities and applying the appropriate calculation(s). (which is far less tedious than manual calculations!)