3 Replies Latest reply on Aug 1, 2012 9:43 AM by daniel.marotta

    Best Practices for Weighted Engagement Score?


      We're interested in developing a score to measure engagement levels in our external community. This would only factor views, replies, and creates (blog posts and discussions posts only).


      We're currently using a 1-10-100 scale based on Nielsen's Theory of Participation Inequality. I firmly believe the hierarchy should have create at the top, then reply, then view - but I'm uncertain how to best assign weight.


      Do you currently have a way to score engagement? I'm really interested in best practices and what's working with other Jive customers.

        • Re: Best Practices for Weighted Engagement Score?

          Brad Popiolek - I came across this question as I was searching for some engagement metrics and best practices. I see no one has replied here, but have you ever heard from anyone on this? Have you come to your own conclusion?

            • Re: Best Practices for Weighted Engagement Score?
              RyanSe Advanced

              Hi Katrina,


              To me, engagement and adoption are rather subjective topics dependent on each distinct use case. That and goals evolve over time


              I've taken several different approaches but the one I find the most effective aligns with the multiple levels approach Brad mentioned above for measuring a generic high level goal. The system I used shortly after we launched our community involved the activities of login,view,search then moved up to rating/liking/creating then finally up to helpful/correct answers. Here's the basic math I used to make it work:


              • For any given time period (I focused on monthly rollup) did the user perform the specified action? If so:
                • Login: 10 points (we didn't force login to consume info so this was to differentiate pure lurking vs. taking that initial step of actually logging in)
                • View: 10 points (Just because you log in, doesn't mean you do anything...this is the next step in the process)
                • Search: 10 points (Just because you view content doesn't mean you are actively looking for it, searching means you are trying to locate something)
                • Rating/liking/replying: 100 points (You're starting to engage more and attempting to add value)
                • Helpful/Correct Answer: 1000 points (Your contributions are starting to be acknowledged as useful)


              Since our primary use case is a tool for finding answers/resolutions this process worked great for us. These are the different levels of engagement:

              1. 0 points - Not engaged at all
              2. 10-30 points - Level 1 - You are starting to engage, taking those first steps. We need to find ways to move you up in the engagement model.
              3. 110-130 points - Level 2 - You have some of the fundamentals down but also are interacting in the community.
              4. 1110-1130 points (ideally 130) - Level 3 - You are an active and contributing member of the community. We need to keep you here.


              As you can see, I used the positions to move up the ladder ever so slightly by just having each level add another zero to the previous. I also chose to treat each activity as a TRUE/FALSE since I felt it was more important to track when folks hit different levels vs. the volume of activity.


              We then focused on adoption and engagement against the model. Peer-peer answering of questions is important to us since staff only answers do not scale well when you have thousands of questions being asked and only a limited number of shared hours to answer them.


              Hope this helps.



            • Re: Best Practices for Weighted Engagement Score?
              daniel.marotta Advanced

              Brad and others,


              I blogged about a similar weighted scoring system earlier this year, Community Health Index; A Shift To A Weighted Scoring System


              Feel free to join the group; External Community Success Metrics.