Version 23

    Let's crowd source the gaps we are finding between old (CMR) and new (Community Analytics) reports. I'm sure we all are using them in slightly different ways, so things that are problems for one might not be problems for others. However, we are also likely to run into some things that are common shortcomings. Please help to fill in the table below. And consider adding comments to indicate use cases - WHY these shortcomings or gaps will be problematic for you.

     

     

    Gap Table

    Reporting featureTop level or Place levelGapCustomer CommentsCustomers to whom this is importantProduct team notes

    Include Subspaces checkbox

    Space reports

    There is now no way to collect data for a whole "collection" of spaces.

    Example Use Case: We move archived content to limited access subspaces so that we can maintain analytics and history. It will be much harder to include these analytics moving forward.

     

    Carbon Black Use Case:  Departments have a main space with subspaces for subdepartments or teams.  Not being able to pull a consolidated list is a show stopper for me.

    • Commvault
    • Carbon Black
    • Novartis Biomedical
    • Logicalis
    • Medidata
    • Accolade

    Add filter options - to allow filtering on custom profile fields

    Top level and Place levelCan't see differences in how different parts of the company interact with the overall site or specific places within it.This was a hard won option added into CMR to assist customers in providing the kinds of reporting that executives demand of community managers. The loss of this filtering capability negates much of the value of the existing reports because a significant level of detail is missing in terms of how individual companies define themselves and thus want to report based on.
    • Commvault
    • Medidata
    • Accolade

    User-to-user interactions - custom profile filtering (CMR)

    (See Department Interactions in Community Analytics)

    Top LevelCan only see at Department level in Community Analytics. CMR allowed filtering using custom profile fields as well

    This was a hard won option added into CMR to assist customers in providing the kinds of reporting that executives demand of community managers. The loss of this filtering capability negates much of the value of the existing reports because a significant level of detail is missing in terms of how individual companies define themselves and thus want to report based on.

     

    Logicalis Use Case: As a global company, we woud like to improve the interactions between specialists that belong to different geographies, and for such we must be able to see the graph showing interactions between countries.

    • Commvault
    • Logicalis
    • Medidata
    • Accolade
    Content LeaderboardPlace LevelThere is currently no replacement for this report.This report has been used to identity the pieces of content that generated relevant increases on the number of views in a certain place.
    • Logicalis
    • Medidata
    • Commvault
    • Massage Envy
    • Accolade
    Top Content/Places (Under Community Usage Dashboard)Top level"Hidden Content" and "Hidden Place" for many of the results, even for Full Access admins. In CMR we could see the full list regardless of whether they were private/hidden.

    We can't report properly if we can't see half the items on the Top Content and Places leaderboards.

     

    • Medidata
    • Commvault
    • Accolade
    Top ContentTop levelCSV is seriously lacking in information compared to the community manager reports. CSV only shows top 20, compared to top 1000 and no information on author, place, content type or place type. (CSV examples provided in next set of tables, for comparison purposes)Because only top 20 by like, view, share etc are available, no easy way to compare the most viewed piece of content in a space with the most liked, since they don't always show up in the same top 20, and there's no way to make sure the content is in the same space for comparison. We use this report to determine a huge amount of comms strategy, making sure we meet benchmarks, checking for successful implementation of programs
    • Massage Envy
    • Accolade - VERY important
    • Commvault
    Questions - Average Response TimesTop level and place levelNo report to determine the average length of time from when a question is posted to its eventual marking as answered

    We use this report heavily to measure how our departments are doing in terms of answering questions in a timely manner. This info is regularly reported to department heads and execs, both as a site-wide number and each department space individually

     

    In the comparison report, this was listed as an external feature, but we used it frequently in our internal instance.

     

    Commvault uses this report as a way to report personal metrics around answer platform questions to enable employees using the community.

    • Massage Envy
    • Commvault
    • Accolade
    Top Successful search QueriesTop levelNo replacement reportThis report was used on a monthly basis to help teams determine tags and promoted search results to focus on for the month.
    • Massage Envy
    • Accolade

     

    CSV Report Comparison

    Place Reports

    Images pasted in are the CSV downloads, not the actual charts shown inside the interface

    Report Name

    CMR/Community Analytics (2016.3.x)

    Report name

    Community Analytics (2018.x)

    Potential Impact

    User Adoption

    This chart shows the number of active, participating and contributing users for the place.

     

    It can be filtered by date range, all profile fields (including custom), and skills/expertise.

    Subspaces can be included in the data displayed for any space.

    Participation Ladder

    This report shows the number of active users for the place, and breaks this number out into consumers, participators and contributors for each month.

     

    Clicking into one of the month bars will display a daily chart for that month.

     

    The chart can be filtered by date range, department and title.

    Note: at this time, it is not possible to include subspace data.

     

    Not having access to either individual user activity level OR profile date filtering makes it impossible to measure user engagement by any meaningful factor other than total platform participation. We have many target groups of users who we focus on driving engagement with, and tracking changes in their activity levels is critical
    User Leaderboard

    This chart shows the top 10 users in the place, along with the points earned by the user by the activities they have done in the place.

    There is currently no replacement for this report. It is under consideration to be added to the roadmap.
    Content Creation

    The content creation chart shows the number of pieces of content (status updates - if allowed in groups, documents, discussions/questions, blog posts, polls, ideas and videos) that have been created in the place for the given date range. It can also be filtered by all profile fields (including custom) and skill.

    Content Creation

    This report is the same as the CMR report, except that it no longer counts status updates; and discussions and questions have been broken out into two separate content types.

     

    Total content created is displayed in the large number.

     

    It can be filtered by date range, as well as department and title profile fields.

    Note: Is only listed by month, not by day of the month. Place location is also not listed. Selected date range and place information are also not included in the download.

    Content Leaderboard

    The Content Leaderboard displays the top 20 pieces of content with engagement activities in the time period. It also shows the number of views the piece of content has had.

     

    It can be filtered by profile field, skill and date range.

    The Top Content chart replaces the Content Leaderboard chart is able to be sorted by Likes, Views, Shares and Comments and shows the top 100 pieces of content per filter type for the date range.

     

    The large number shows the total number of content items.

     

    In 2018.3, the ability to download a csv file has been added.

     

    No data on the space, the author, container type or the content type removes any value this report had. We use this report extensively to track the impact of communication strategy, promotion campaign and to get a sense for what people are doing in the community, in their groups and in specific spaces.  Because it's just the top 20, rather than the top 1,000 pieces of content, if the most liked content and most viewed content aren't within 20 places of each other, there's no easy way to compare most liked to most commented on to most viewed, etc
    Daily Activity

    This report shows the daily activity for the community for these activity types:

    • Views
    • Likes
    • Responses
    • Revisions
    • Creates

    For the given time period. Averages for the time period are posted below the chart. It can also be filtered by all profile fields (including custom) and skill.

    ActivityThis report shows the daily activity for the place for these activity types:
    • Views
    • Likes
    • Comments
    • Revisions

    For the given time period.

     

    Clicking into the data point for a month will display a chart with the values for each day of the month.

     

    Total number of activities for the time period is shown in the large number

    This chart can be filtered on date range, department and title.

    Note: Is only listed by month, not by day of the month. Place location is also not listed. Selected date range and place information are also not included in the download.

    Answered Questions

    Displays the total number of questions for the date range, broken out by those with responses, replies marked helpful and replies marked correct.

     

    It can also be filtered by all profile fields (including custom) and skill.

    Questions and Answers

    The Questions and Answers chart displays the total number of questions, and the number of replies marked helpful and replies marked correct within the date range.

     

    Can also be filtered on department and title profile fields

    Note: Is only listed by month, not by day of the month. Place location is also not listed. Selected date range and place information are also not included in the download.

    Questions - Average Response Times

    Displays the average response time in days for questions, broken out by time to first response, time to first helpful answer, and time to correct answer.

     

    It can also be filtered by all profile fields (including custom) and skill.

    Worth noting that the comment about response times being in hours is CRITICAL context to include when replicating this report.

    There is currently no replacement for this report - it was an external community feature

     

    Related Content

    Does anyone have any best practice ideas/suggestions for reporting on top content across an entire instance?