5 Replies Latest reply on Aug 24, 2016 2:28 PM by Ryan Rutan

    Can We Hack a Way to Measure Inactive Users in Jive DES?

    Ted Hopton

      I am trying to solve a long-standing problem: how can I use Jive DES to measure the number of inactive members in my community? Not only is the inactive percentage a key metric we want to track, it’s an essential component of many other common measures. You need it for a Participation Ladder, for example, and any time you ask the question, “what percentage of users did _____?” you need to be able to calculate the denominator, the number of users, which includes inactive users.

       

      It’s easy to count the active users in any time period with Jive DES: count the unique actors among all the records in Jive DES for that period. By definition, if they did something in Jive, they are active users, and if they did something there is a record of it in Jive DES.

       

      The problem is identifying inactive users, which are defined as enabled (their Jive account has not been de-activated) and registered users, who are enabled users who have logged into the system at least once. Let’s take these issues in turn.

       

      Enabled Users

      I can query a user’s enabled status in the present: 0 means deactivated and 1 means enabled. But Jive DES has no record of when that status was changed. It has no way to tell whether a user was enabled or not at a specific time in the past. It only shows the present state at the time of the query.

       

      Registered Users

      I can also look at the user’s last login date, and if it’s a non-valid date then infer that the user has never logged in. So if I know the user is enabled and there is a valid last login date, then the user is registered. But this, too, exists only for the present state.

       

      Jive DES Current Capability

      Is there something in Jive DES currently that can provide the enabled and registered status at any given point in time in the past? Not to my knowledge, and when I asked (former) Jive engineer, Dirk McNealy, he acknowledge that it was not available to me (although he had an idea of how it could be calculated, but he never created that solution).

       

      Jive Needs to Solve This

      So, in thinking about this critical reporting need, I first want to call out Jive and ask them to create a solution. I believe this falls under Udit Shah's domain. This is a significant gap in Jive DES reporting capability that should be addressed.

       

      Can We Hack This?

      But I recognize that whatever solution Jive might develop will take time and I don’t want to wait. That’s why I am posting this in the Developer space instead of where I usually post about reporting questions (the Jive Advanced Customer Measurement space). I want to hack together a solution quickly.

       

      Is there a way to automatically:

      • Execute some action that Jive DES already tracks (to create a Jive DES record)
      • That will not impact the user in any disruptive way (e.g., it can’t trigger sending the user a message)
      • Once every day
      • By (so we can query on Actor) or for (so we can query on Object ID) every user account that is
      • Enabled on that day (a record for this scenario = Enabled)
      • And has a valid date in the Last Logged In field (a separate record for this scenario = Registered)?

       

      Create an Add-on 

      What I’m thinking about would ideally be a simple add-on that runs a process daily at a set time and thereby takes actions that are captured in Jive DES for every enabled and registered user for that day. Then we can query those specific records and know the enabled and registered status of each user on each day.

       

      I’m not sure what that action should be, as it can’t be intrusive. I’d be willing to accept something slightly intrusive, such as awarding each user one Rewards point if that would accomplish the objective of creating a record. Ideally it would be something completely invisible to the user, such as granting permission to something that is not actually used for anything, or removing permission for that same thing. Maybe enabled users are granted permission to something and deactivated users get that permission removed. So long as you remain enabled, you get the permission assigned again every day. And if you’re not enabled, then the process skips you. Perhaps we could leverage StreamOnce and generate activity in a space/group that only sys admins see, so as not to disrupt any users?

       

      Anyone brilliant ( Ryan Rutan? Andrey Mikhalchuk? Doug MacKay? Frank Pathyil? jlevi? Dane Slutzky ) have any ideas about this? I’d love to have a solution.

        • Re: Can We Hack a Way to Measure Inactive Users in Jive DES?
          Ryan Rutan

          I'll be the first one to take a stab at answering....and you are right this is a tough nut to crack.

           

          The fundamental flaw in using DES/Webhooks for this style of reporting is that they are based on the opposite (activity), not inactivity.  As such, using their current model...the only way to quantify it from my perspective is the same that is used in the quantification of Dark Matter, which is knowing the whole and known and capturing the difference to quantify the unknown.

           

          With the super high-level posit above, using tools today, the only ways I think I could approach it with today's tools (some of which stated above in your post):

          1. Create a temporary dump of User ID's and last-logged (userId, last logged in, last modified) in dates and bind that to a Webhook listener that will keep those three fields in-sync with production.
            • Based on this dump, you can make a global extrapolation as to the number of Inactive people per community.
          2. To get per place statistics, you'd have to get an initial load from the API as to who is Following/Member of the place and then you can use DES or Webhook activity to keep that in sync.
            • Subtract out any users you know to be globally inactive (#1) and then subtract distinct activity actor's from the place total.

           

          All this being said, I agree there is no clean way to do this with just a single report...and it will definitely take a series of queries and systems to keep the reports accurate.

           

          Some ideas perhaps for the product to embrace:

          • Explicit Activity for Inactivity - Inactivity beacons emitted per user per {threshold} per {place}
            • Inactivity would need to be able to be defined per Jive instance using an agreed set of toggles.
            • This would give existence in the DES stream to inactivity and would allow for people to aggregate and count the inactivity in a single data-stream. 
              • Getting the data to roll-up could be a challenge but it's possibly a start.

           

          Looking forward to seeing other ideas this topic.

           

          Andrey Mikhalchuk Benjamin Taub - Perhaps you may have some ideas on this given your Insite solution dabbles in segmentation from DES, etc...?

           

          Udit Shah - This could turn into a great discussion =)

          1 person found this helpful
            • Re: Can We Hack a Way to Measure Inactive Users in Jive DES?
              Ted Hopton

              Thanks for jumping in, Ryan Rutan! You've identified something I overlooked: the ability to measure inactivity at a specific place level. I can't say this is a current priority of mine, or even think of a reason it would become important, but for Jive to develop a global solution I agree it should be evaluated.

               

              For my simpler needs, a hack that will work in the very near future, your 1 and 2 above don't seem suitable to me because I am not interested in high-level counts. I need to be able to look at the specific user level, so I need a record created by or for each inactive user every day. That's the only way to address this consistently with how reporting is created with DES data. I do NOT want to have to export DES data to manipulate it in Excel (that's how I calculate inactivity % now, running individual reports in CMR, and it sucks in so many ways, including that I can't drill down).

               

              You've built many add-ons, Ryan Rutan. Can you envision a simple one that pings an action every day such as I described above? I would not know where to start...

                • Re: Can We Hack a Way to Measure Inactive Users in Jive DES?
                  Ryan Rutan

                  The more I think about it...the more it gets a tangled.  Turning inactivity into activity for a report (may, but not saying it will) have adverse affects on Jive's internal understanding of "inactive" users.  There's a 80% chance in my head that it will be fine, but for some reason I can see Jive having some internal optimizations around dormant users and cache optimization...and breathing life into all the dormant people may mess with that or other reports.  Any possible side-effects discounted ... I can see a way to do some of this...but in the end, you are still going to need some sort of daily purgeable user index to store active.

                   

                  Idea #1 - Inactive Status Determined via Middleware

                  • Jive Signed Add-On using the Core V3 API - Run-As Feature & Signed Add-Ons capability.
                  • You'll need a middleware service to pull a list of every new user...(there are ways to do this to make this efficient using 1x load with API and then DES)
                  • Every day, everyone starts out as inactive....and polling DES queries through the day remove people that are detected.
                  • End of the day (or 24hr period), what is left standing is a list of in-active people ...at which point you can export/save, or
                    • execute some minimal activity (such as liking/disliking a piece of content) to give those people activity.
                  • Logically, this process has holes for people who may perform actions during the time calculations are being made...but it should be a small window.

                   

                  Idea #2 - Inactive Status Determined via Data

                  • Jive Signed Add-On using the Core V3 API - Run-As Feature & Signed Add-Ons capability.
                  • Every morning a process runs to grab a list of all the Jive Users. (let's say 6am)
                  • Use Run-As, you have everyone in the list Like and/then Dislike a piece of specific piece of content.
                    • Will have to be public content, but could be stored in a personal public container to keep it out of the boulevard
                    • You'll want to mark the content as Outdated to keep all the activity from turning it into a trending document.
                    • Note:  This will give you 2 activities per person per day (but you may be able to filter-out disliking) or perhaps there is another action that you can do with only one action...but nothing comes to mind.
                  • At this point, it's about using the detection of people with only 1-2 activities in the data-set as "inactive" people and accounting for them accordingly.

                   

                  Unfortunately, I don't see a "super easy" way to do this without some middleware service running logic on a daily basis.  You could put #2 into an App that is initiated with a button...but it would become someone's job to press the button every day...and I don't think that is what you are looking to do.

                   

                  Hope that helps. =)

                    • Re: Can We Hack a Way to Measure Inactive Users in Jive DES?
                      Ted Hopton

                      Really appreciate your advice on this, Ryan Rutan -- thank you! Your Ideas #1 and #2 are on the path I'm looking for.

                       

                      I don't want a solution that requires pushing a button. Seems surprising that an app could not be created that runs at a set time every day. Need a solution, for sure, that we can set and then forget.

                        • Re: Can We Hack a Way to Measure Inactive Users in Jive DES?
                          Ryan Rutan

                          The piece that's missing for Apps is the persistent background state to execute the timer.  For even non-Jive platforms, when running an arbitrary set of code (rather than one that is structured and parameterized), it is common to offload the execution to a remote system (i.e. middleware).  Places where I could see improvements, would be a built-in DES report scheduler/exporter, but in this case ...you would still need middleware to receive the export and execute the Like/Dislike logic.

                           

                          Hope that helps clarify why the Apps framework doesn't support this capability.

                          1 person found this helpful