Version 40

    Let's crowd source the gaps we are finding between old (CMR) and new (Community Analytics) reports. I'm sure we all are using them in slightly different ways, so things that are problems for one might not be problems for others. However, we are also likely to run into some things that are common shortcomings. Please help to fill in the table below. And consider adding comments to indicate use cases - WHY these shortcomings or gaps will be problematic for you.

     

     

    Gap Table

    Reporting featureOverall or PlaceGapCustomer CommentsCustomers to whom this is importantProduct team notes

    Include Subspaces checkbox

    Space reports

    There is now no way to collect data for a whole "collection" of spaces.

    Example Use Case: We move archived content to limited access subspaces so that we can maintain analytics and history. It will be much harder to include these analytics moving forward.

     

    Carbon Black Use Case:  Departments have a main space with subspaces for subdepartments or teams.  Not being able to pull a consolidated list is a show stopper for me.

    • Commvault
    • Carbon Black
    • Novartis Biomedical
    • Logicalis
    • Medidata
    • Accolade
    • Thornton Tomasetti
    • Savills
    • Novartis
    • Prudential
    • Western Digital
    • Proximus

    Add filter options - to allow filtering on custom profile fields

    Top level and Place levelCan't see differences in how different parts of the company interact with the overall site or specific places within it.

    This was a hard won option added into CMR to assist customers in providing the kinds of reporting that executives demand of community managers. The loss of this filtering capability negates much of the value of the existing reports because a significant level of detail is missing in terms of how individual companies define themselves and thus want to report based on.

     

    Savills Use Case:  The ability to multi-select from filter fields would help, and save time in producing reports

    • Commvault
    • Medidata
    • Accolade
    • Thornton Tomasetti
    • Savills
    • Prudential
    • Western Digital
    • Proximus

    User-to-user interactions - custom profile filtering (CMR)

    (See Department Interactions in Community Analytics)

    Top LevelCan only see at Department level in Community Analytics. CMR allowed filtering using custom profile fields as well

    This was a hard won option added into CMR to assist customers in providing the kinds of reporting that executives demand of community managers. The loss of this filtering capability negates much of the value of the existing reports because a significant level of detail is missing in terms of how individual companies define themselves and thus want to report based on.

     

    Logicalis Use Case: As a global company, we would like to improve the interactions between specialists that belong to different geographies, and for such we must be able to see the graph showing interactions between countries.

     

    Savills: Note that Department names differ as per geography, so again, being able to multi-select would help.

    • Commvault
    • Logicalis
    • Medidata
    • Accolade
    • Thornton Tomasetti
    • Savills
    • Prudential
    • Western Digital
    • Proximus
    Content LeaderboardPlace LevelThere is currently no replacement for this report.This report has been used to identity the pieces of content that generated relevant increases on the number of views in a certain place.
    • Logicalis
    • Medidata
    • Commvault
    • Massage Envy
    • Accolade
    • Thornton Tomasetti
    • Filtration Group
    • Prudential
    • PwC
    • Western Digital
    • Proximus
    Top Content/Places (Under Community Usage Dashboard)Top level

    "Hidden Content" and "Hidden Place" for many of the results, even for Full Access admins. In CMR we could see the full list regardless of whether they were private/hidden.

     

    Most recent update from L2 support on my support case:

    "In the new 'Community Analytics', the "Hidden Content" and "Hidden Place" shows up for many of the results. This happens even for Full Access admins. This is unfortunately an intended behavior for Community Analytics as of now."

     

    Shai Sagi - this is totally unacceptable, as this provides no mechanism to report on all content and all places.,

    We can't report properly if we can't see half the items on the Top Content and Places leaderboards.

     

    • Medidata
    • Commvault
    • Accolade
    • Thornton Tomasetti
    • Massage Envy
    • Filtration Group
    • Prudential
    • Western Digital
    • Proximus
    Top Content/Places (Under Community Usage Dashboard)The CSV only shows 100 Places, regardless of how many communities exist.

    With over 450 communities, there's no way for us to generate metrics for the overall community here.  Use case: How many have <25 members, followers, Which communities were created prior to XX/XX/XXXX, etc.  DES will not support this type of a query.

     

    Update: Being addressed by Aurea Engineering on JVANLT-3981.  The defect will be fixed and available in version 2019.2

    • Novartis
    • Prudential
    • Western Digital
    • Proximus
    Top ContentTop levelCSV is seriously lacking in information compared to the community manager reports. CSV only shows top 20, compared to top 1000 and no information on author, place, content type or place type. (CSV examples provided in next set of tables, for comparison purposes)Because only top 20 by like, view, share etc are available, no easy way to compare the most viewed piece of content in a space with the most liked, since they don't always show up in the same top 20, and there's no way to make sure the content is in the same space for comparison. We use this report to determine a huge amount of comms strategy, making sure we meet benchmarks, checking for successful implementation of programs
    • Massage Envy
    • Accolade - VERY important
    • Commvault
    • Thornton Tomasetti
    • Filtration Group
    • Savills
    • Novartis
    • Prudential
    • Proximus
    Questions - Average Response TimesTop level and place levelNo report to determine the average length of time from when a question is posted to its eventual marking as answered

    We use this report heavily to measure how our departments are doing in terms of answering questions in a timely manner. This info is regularly reported to department heads and execs, both as a site-wide number and each department space individually

     

    In the comparison report, this was listed as an external feature, but we used it frequently in our internal instance.

     

    Commvault uses this report as a way to report personal metrics around answer platform questions to enable employees using the community.

    • Massage Envy
    • Commvault
    • Accolade
    • Proximus
    Top Successful search QueriesTop levelNo replacement reportThis report was used on a monthly basis to help teams determine tags and promoted search results to focus on for the month.
    • Massage Envy
    • Accolade
    • Thornton Tomasetti
    • Novartis
    • Proximus

     

    CSV Report Comparison

    Place Reports

    Images pasted in are the CSV downloads, not the actual charts shown inside the interface

    Report Name

    CMR/Community Analytics (2016.3.x)

    Report name

    Community Analytics (2018.x)

    Potential Impact

    User Adoption

    This chart shows the number of active, participating and contributing users for the place.

     

    It can be filtered by date range, all profile fields (including custom), and skills/expertise.

    Subspaces can be included in the data displayed for any space.

     

    For this report, there are actually TWO CSV files that download, which allow us to get at both the actual underlying data as well as the manipulated information used to create the chart.

    • Chart results
    • User results
    Participation Ladder

    This report shows the number of active users for the place, and breaks this number out into consumers, participators and contributors for each month.

     

    Clicking into one of the month bars will display a daily chart for that month.

     

    The chart can be filtered by date range, department and title.

    Note: at this time, it is not possible to include subspace data.

     

    Appears to only download the chart results, and not the user results. For most CMs, the user results are more important.

    Not having access to either individual user activity level OR profile date filtering makes it impossible to measure user engagement by any meaningful factor other than total platform participation. We have many target groups of users who we focus on driving engagement with, and tracking changes in their activity levels is critical.

     

    The new reports appear to only download the chart results, and not the user results. For most CMs, the user results are more important.

    User Leaderboard

    This chart shows the top 10 users in the place, along with the points earned by the user by the activities they have done in the place.

    There is currently no replacement for this report. It is under consideration to be added to the roadmap.Knowing who is most active in a place is hugely important for people who are actively managing groups and spaces in Jive. This helps identify key audience members, influencers, champions.
    Content Creation

    The content creation chart shows the number of pieces of content (status updates - if allowed in groups, documents, discussions/questions, blog posts, polls, ideas and videos) that have been created in the place for the given date range. It can also be filtered by all profile fields (including custom) and skill.

    Content Creation

    This report is the same as the CMR report, except that it no longer counts status updates; and discussions and questions have been broken out into two separate content types.

     

    Total content created is displayed in the large number.

     

    It can be filtered by date range, as well as department and title profile fields.

    Note: Is only listed by month, not by day of the month. Place location is also not listed. Selected date range and place information are also not included in the download.

    Really missing that the old reports had plain English column headers instead of database lingo.

     

    Like that new versions created is listed, as that is valuable information.

     

    Miss the day breakdown, which can be useful to help prove value in an effort to drive participation.

    Content Leaderboard

    The Content Leaderboard displays the top 20 pieces of content with engagement activities in the time period. It also shows the number of views the piece of content has had.

     

    It can be filtered by profile field, skill and date range.

    The Top Content chart replaces the Content Leaderboard chart is able to be sorted by Likes, Views, Shares and Comments and shows the top 100 pieces of content per filter type for the date range.

     

    The large number shows the total number of content items.

     

    In 2018.3, the ability to download a csv file has been added.

     

    No data on the space, the author, container type or the content type removes any value this report had. We use this report extensively to track the impact of communication strategy, promotion campaign and to get a sense for what people are doing in the community, in their groups and in specific spaces.  Because it's just the top 20, rather than the top 1,000 pieces of content, if the most liked content and most viewed content aren't within 20 places of each other, there's no easy way to compare most liked to most commented on to most viewed, etc
    Daily Activity

    This report shows the daily activity for the community for these activity types:

    • Views
    • Likes
    • Responses
    • Revisions
    • Creates

    For the given time period. Averages for the time period are posted below the chart. It can also be filtered by all profile fields (including custom) and skill.

    ActivityThis report shows the daily activity for the place for these activity types:
    • Views
    • Likes
    • Comments
    • Revisions

    For the given time period.

     

    Clicking into the data point for a month will display a chart with the values for each day of the month.

     

    Total number of activities for the time period is shown in the large number

    This chart can be filtered on date range, department and title.

    Note: Is only listed by month, not by day of the month. Place location is also not listed. Selected date range and place information are also not included in the download.

    Miss the day breakdown in the CSV, which can be useful to help prove value in an effort to drive participation. And unsure why Creates was removed - that seems to be very relevant and valuable data.
    Answered Questions

    Displays the total number of questions for the date range, broken out by those with responses, replies marked helpful and replies marked correct.

     

    It can also be filtered by all profile fields (including custom) and skill.

    Questions and Answers

    The Questions and Answers chart displays the total number of questions, and the number of replies marked helpful and replies marked correct within the date range.

     

    Can also be filtered on department and title profile fields

    Note: Is only listed by month, not by day of the month. Place location is also not listed. Selected date range and place information are also not included in the download.

    Not sure why With Responses was removed - there is value in knowing that someone has responded, whether or not the response was marked helpful or correct.
    Questions - Average Response Times

    Displays the average response time in days for questions, broken out by time to first response, time to first helpful answer, and time to correct answer.

     

    It can also be filtered by all profile fields (including custom) and skill.

    Worth noting that the comment about response times being in hours is CRITICAL context to include when replicating this report.

    There is currently no replacement for this report - it was an external community feature

    While this was intended for an external community audience, many internal audiences use community for Q&A, and so this is still VERY RELEVANT.

     

    For example, this is a perfect element to be used in Community Management KPIs to track how quickly answers about the community itself take to get answered.

    Profile Completion ReportProfile Completion
    • Missing key data, like user names
    • Dates in the report are not actually dates
    • Field names are not plain language, but actual database names
    • No custom profile fields are included
    As is, the report is virtually useless. Worse than not having it at all is thinking I'm getting it, only to find it no longer provides the data I used it for.

     

    Related Content

    Does anyone have any best practice ideas/suggestions for reporting on top content across an entire instance?