12 Replies Latest reply on Dec 1, 2017 4:12 PM by b.taub

    Community Manager Reports Showing Incorrect Data

    sfield

      Has anyone noticed if their community manager reports are showing incorrect data? We pulled the monthly report for November and one of our measurements is number of correct answers. This shows an aggregated, all-time view of correct answers and somehow we are -8 this month, which is incorrect. We tested to see if the report was pulling the same data as Oct. 2017 and Sept. 2017 and received different data. We are Jive-x cloud. Anyone else having a similar issue? Anyone know if deactivating an account would have an effect? Jive Internal Communities Jive Advanced Customer Measurement

       

      Alternatively, anyone know the call for pulling the number of correct answers in Jive DES?

      Adding some data guys who may have an idea: Ted Hopton Benjamin Taub

       

      CC: Ed Sullivan, Helen Chen

        • Re: Community Manager Reports Showing Incorrect Data
          bengarves

          We have deployed adobe analytics to envelop the THWACK community and track everything. It was an internal initiative with its own momentum and gave us some granularity we wouldn't otherwise get from the Community Manager Reports. It did show us that there's virtually no agreement between the two systems. I've always chalked it up to different analytical methods being deployed, but it's nice to know someone else is in the same boat.

          1 person found this helpful
          • Re: Community Manager Reports Showing Incorrect Data
            oonagh.m

            Yes, we've been having the same problem.

             

            I started a case last month and it's been escalated but I'm still waiting on a resolution. I'm concerned that some of my reports are incorrect too, like the average response times report which is harder to spot check.

              • Re: Community Manager Reports Showing Incorrect Data
                Helen Chen

                Hi Oonagh - I think your community launched fairly recently.  Do you happen to know if this is changed behavior, or has this always been the case? 

                 

                fwiw -  I pulled this report on jive-n and jive-x.  I looked at the default report view, and then selected a date range for April thru May of this year.  The displayed charts changed and the CSVs had data specific to the selected time ranges.  Is this was you were asking in Answered questions report: does it show TOTAL questions answered/asked? 

                  • Re: Community Manager Reports Showing Incorrect Data
                    oonagh.m

                    I'm not sure if this is new behavior or not. I only realized a month or so ago that my answered questions report was showing the wrong number of questions.

                    And yes, that's what I was wondering about in the other question. I wasn't sure if that report showed the total number of questions in the community or if it showed the total number of questions posted to the community inside the date range.

                    My data does change if I move move the date range back (from April to July vs October to November, for example) but when I move the first date range forward I can still see the same number of questions.

                    1 person found this helpful
                • Re: Community Manager Reports Showing Incorrect Data
                  Helen Chen

                  I would not expect deactivating a user to change the numbers, but deleting is likely to mess things up royally as some content could become orphaned. 

                   

                  That said, Ben, when you were up here visiting for the Boston User Group, I think you mentioned an issue with some data missing for deactivated users. Your app goes back and does a second pass on data to make sure that you get some missing subset.  Could this be related to Stephanie's problem?

                    • Re: Community Manager Reports Showing Incorrect Data
                      b.taub

                      Helen,

                      For Profiles, there is an Updated timestamp that is reset anytime there is any change to the record, but it does not get reset when you deactivate a user - for some odd reason.  However there is a DES record written when a user is deactivated. So for our nightly data pulls - after we have pulled any new or changed profiles, InSite will look through the last 24 hours of DES activity and looks for the USER_DEACTIVATED activity there, pull the Jive ID of that user - and then issue an API call for Just That One Record.

                       

                      I really just use that as an example of some of the idiosyncrasies of the APIs.  In this case, I think Stephanie Field is experiencing something a bit different, which is that DES data doesn't always line up with API data - sometimes correctly.  For instance, Content Views in the APIs is over the life of the object.  Whereas the DES can be time-bound - e.g. "Views in April".  Some objects can also predate the DES, which started in 2014, so any views or activity prior to 2014 would be missing from the DES data.

                       

                      The third issue is that CMRs appear to pull data from the APIs, the DES and data sources that are not accessible anywhere else.  As an example the CMRs the Profile Completion Report will tell you if a user has uploaded a Profile photo, but that is not available at all in the APIs, although their Avatar image and the Avatar Thumbnail ID (the numerical name of the image used for the Avatar) are both available via the APIs. And in the DES documentation here: Using the V2 Jive Data Export Service (Analytics) it states explicitly that ACTIVITY_ADDATTACHMENT_PROFILEIMAGE is a Non-Supported Event.  So either the documentation is incorrect or Jive is getting that from a non-published source.

                      So it is possible that what Stephanie is experiencing is something similar and this lack of data agreement is one of the reasons why they are moving the CMRs into the Community Analytics framework - precisely because numbers were not in agreement across multiple Jive data sources.  Registered Users is another one.  We often found that the Admin Console and the CMRs were close, but never matched.  Drove us batty.  Sorry I don't have a better solution for Stephanie - other than to assure her that she is not crazy and she is not wrong.  But I suspect that is cold comfort when trying to pull a report.

                        • Re: Community Manager Reports Showing Incorrect Data
                          Helen Chen

                          Thanks Ben.  We're tending towards thinking that something is really wrong with the CMR reports in our Jive-x community.

                           

                          Three of us pulled the same Answered questions report, using the same filter, looking at the same date. Two of us get one value (which is lower than last month) and one gets the expected result.  I would expect 3 people, in the same community, pulling the same report should get the same result...  

                           

                          Total Freaky Friday reports! Thanks for taking the time to respond.  I hope everyone has a good weekend!

                            • Re: Community Manager Reports Showing Incorrect Data
                              b.taub

                              I have this very low bar for this stuff: "Computers should at least be able to do math right."  But apparently, sometimes not.  So irritating.  And Support won't be able to reproduce it - so you end up feeling crazy or stupid or both.

                               

                              Sorry this is happening to you guys.  It irritates us too, but we try to develop a permanent workaround, so that even if we stepped in the pot hole, you won't have to.  Have a great weekend.

                        • Re: Community Manager Reports Showing Incorrect Data
                          sfield

                          Unfortunately me and Ed Sullivan are now seeing completely different numbers in our Community Manager Reports. His shows that there are 18 more questions marked as correct than me . Very odd, inconsistent and frustrating behavior. Anyone else have that happen before?

                            • Re: Community Manager Reports Showing Incorrect Data
                              b.taub

                              Yes.  You run a report and the person next to you - with the same permissions - runs the same report and they are not the same results.  Yup.

                               

                              This may be a very granular permissions issue (where Ed has access to some content that you can't see or vice-versa) or we saw timing issues sometimes.  There are a lot of indexes in Jive that seem to be rebuilt on some kind of a schedule - once an hour, once a day, etc.  So occasionally, as near as we could figure - sometimes in the 10 minutes between Ed running the report and you running the report - an index would get rebuilt - and you would pull different results.  Best thing to do is to rerun them simultaneously and see if that does the trick.

                               

                              But you aren't crazy.

                              1 person found this helpful