72 Replies Latest reply on Dec 19, 2011 12:12 PM by mnevill

    Do reputation points help or hurt?


      I have customers who take a look at the reputation points available in the solution I sell, and say, "that will never work here." I know they work well in external communities, but do they work inside the firewall? If so, what attributes of your culture make it successful?

        • Re: Do reputation points help or hurt?

          I my years of community experience, the point systems do work and some uses will go above and beyond to get the points. AT few of my gaming communities I have worked at, the point system was a game in it self.




          As far as internal community I don't see why they will not work, people will always have Ego.  I always ask people what is the first thing you looked at when you went to the community? Most of the time the response is the top members.




          Is that a special rank you got there. or will we all get the medals. I have been thinking of creating a status level based on groups not points and assigning that to my community leaders to give them some distinguishing mark.

          • Re: Do reputation points help or hurt?

            Hi John,


            Yes, my "medal" is a way for this community to identify me as a Jive employee. Since this community is designed to be vendor-agnostic, it helps members to see when a Jive employee is creating comments or content. That way, they can weigh our content and comments accordingly. :)

            • Re: Do reputation points help or hurt?

              I definately think they help especially in finding the experts or long time contributors.  I think internally you'd want change some of the point giving variables like including view count and document ratings.  In both internal and external the notion of badges would be helpful in rewarding different types of people.  The single point system only rewards the top contributors because the rest usually fall off very quickly.  It would be good to show some achievement badges (or trophies) that would stay pinned to a person (like most questions answered in a week).   

              • Re: Do reputation points help or hurt?

                You know, we've been talking about badges here. I'd like to award a colleague a badge, which would appear on their profile page. It would be cool to have all kinds of badges, defined by each company.


                Reminds me of Girl Scouts!

                • Re: Do reputation points help or hurt?

                  We are just getting started, but we certainly have high hopes for Reputation Points. For our operations-centric use case, understanding who can contribute a quality solution to a customer problem is always a challenge. With Clearspace, we hope to expand the sphere of influence of our best employees by providing them with a global platform from which to practice.


                  Our initial challange was developing the taxonomy for RP. What are the best practices for enterprise communities on: How many points do you reward for each activity? What are the point ranges that define the levels? What is the name of each level? etc.


                  Didn't find a lot of useful guidance via web search. Gia do you have any?

                  • Re: Do reputation points help or hurt?

                    There are many ways to structure your points and names. You can relate your names to you products or user general terms like pro guru....We are currently renaming all of our rank systems as we went kinda of generic for the first launch. The point system I have set up many diffrent ways over the years. When I first launch the community I like to make the points a little bit higher to give my early adopters a little bonus for joining early. Hope this helps.

                    • Re: Do reputation points help or hurt?

                      Hi Carl! I'm going to engage my counterpart in this conversation, since I'm not the right person to talk about best practices re: point scales. Standby...

                      • Re: Do reputation points help or hurt?

                        Would appreciate the help. Here is our first take and what we launched with:


                        Principles we followed:

                        1. Tiers should grow logarithmically, with early tiers making easier to gain status (see progress) and later tiers representing a well-earned, quality achievement.

                        2. Use graphics that indicate progress and communicates that there are award levels yet to be earned. This Status Icon 1.png ; not this Status Icon 2.png

                        3. Choose names that related to community and would work internationally.

                        Note: the most difficult name to come up with was the first, 0 - 99. Something that didn't offend, indicated that user was just getting started, but also a name that a user would want to leave behind -- no honor or dishoner in it.


                        Point Range







                        Active Contributor



                        Valued Contributor



                        Senior Contributor



                        Premier Contributor






                        Grand Master



                        • Re: Do reputation points help or hurt?

                          Looks like you are on the right track for sure. I have my community set up in a similar way right now. We have been talking about possiblity of changing the names to something more on the funny side.

                          • Re: Do reputation points help or hurt?

                            I guess this depends - for external communities your a right in my opinion - it might be a kind of incentive to gather "stars", but from internal point of view e.g. for usage within intranets it might be highly deprecated by for example workers council as it might apply e.g. for us (Lufthansa)...

                            • Re: Do reputation points help or hurt?

                              Hi Carl,

                              You worked it out beautifully. My only change from an internal point of view is to allow the "Newbie" to become a "Contributor" more quickly. The almost instant "reward" pays off in that they notice their status change very quickly from the start.  Something along the lines of 0-5 is a newbie and 6 - 99 is a contributor.



                              • Re: Do reputation points help or hurt?

                                Well, we've been using the points thing here internall since Sept '07, and -- to be honest -- I can't see that it makes a significant difference one way or another.


                                Sure, at the outset, we had a few people who were trying to rack up points, but they didn't turn into long-term contributors.


                                I mean, sure, it's nice to see someone out there with a bazillion points, but you might wonder if they ever have time for their day job.


                                And newbies seeing that sort of ranking might be put off.


                                Bottom line?  I think we could turn it off tomorrow, and no one would notice.

                                • Re: Do reputation points help or hurt?

                                  Thanks for the recommendation -- we will take a look at lowering the threshold.

                                  • Re: Do reputation points help or hurt?

                                    Appreciate the long view -- Do you think there would be a difference if the vast majority of the users were 18-22 yr. old, entry-level associates?

                                    • Re: Do reputation points help or hurt?

                                      Perhaps the demographics might matter.


                                      My target audience is "high value knowledge workers in a corporate setting", so I can't really offer up any insight there.


                                      But another thought that I'm encouraging with my team is to step back, describe what you want to have happen, and look at different ways that you might get the same results, only better.


                                      I'm assuming that your goal is "vibrant, enthusiastic participation".  Other mechanisms might be management recognition, small prizes, a night on the town with your significant other, and so on.


                                      And, just maybe, the points thingie.

                                      • Re: Do reputation points help or hurt?

                                        I can pretty much guaruntee that demographics matter. Even here at Jive, I'm not sure how much points matter. On the various game-based communities I've run, points or int hose days, post count, was HUGELY important to many people.


                                        "Use graphics that indicate progress and communicates that there are award levels yet to be earned. This Status Icon 1.png ; not this Status Icon 2.png"


                                        I couldn't agree with you more Carl. Pretty much a requirement in my book.

                                        • Re: Do reputation points help or hurt?

                                          Im somewhat suprised this has yet to come up in this discussion, such nice people it seems

                                          theres a "flipside" to "points systems", a small unobtrusive points system for those organisations who think such would never work but have inhouse forum/none real-time communication methords can serve a good purpose for managers.


                                          knowing which employee is "just there" working or is "-working- and commited"


                                          post count and the "stars" system (Ala Sun's forums, also Jive based ) awarded within the teams to other members can tell project leaders/managers who are really working at the project and are commited to its growth as opposed to just "along for the ride", and give early indication of "problems", A regular contributor suddenly slopeing off -could- be having problems that they havent felt able to discuss just yet.


                                          it -can- be an excellent barometer.


                                          (though its important to be aware some of the introverted employee's who could be wonderful and commited may be too "nervous" to get involved on that level).

                                          • Re: Do reputation points help or hurt?

                                            If the purpose of a point system is to "reward" the behaviors you want then you have to consider: what are all the behaviors you want to reward?  You can easily reward contributors by giving them points -- but contribution is not the only behavior you want to encourage.  There are many other rewardable behaviors that a point system fails to identify.


                                            For example, I would want people in my company to at least read the topics in a forum that pertain to their job.  I'd want them to subscribe efficiently (e.g. an RSS feed to their approved reader).  Even more importantly, I'd want them to use the information they see in our collaborative spaces.  The act of getting others on board is also something I'd reward (viral evangelism points).  So if we can figure out how to rate behaviors other than contributing content, then I think we can begin to measure the workplace -- how many contributors, how many spectators etc.  And then the reward points might be fair.  Right now, I think that most "Karma" systems (I'm thinking of Plurk.com here) are skewed -- since they measure what they can measure, not what they should reward.


                                            Let me fully disclose that I'm a Forrester analyst and am reading the Groundswell book, published by colleagues of mine, and they present a model of multiple types of participants in collaborative site (including those who are contributors, spectators, as well as others in the mix).  So my comment is partially influenced by the model presented in that book.  But I present it for you to consider and comment upon.

                                            • Re: Do reputation points help or hurt?

                                              Reputation management emerged as a way of compensating for the rich context of clues that we use in real-world, person-to-person interactions, in online environments where anonymity is unavoidable. If I don't know Tom from Adam, and I need to know whether to trust his comments or disqualify them, the reputation management system comes to the rescue. The alternative (reading every post from everybody so as to develop my own trust evaluations, is not feasible).


                                              Because its main benefit is minimizing the impact of anonymity, the value of RM is much lesser in communities where (a) people know each other in larger chunks –e.g., five offices connected globally, where most people in each office will know about one fifth of all people in the community – and (b) people are not anonymous – i.e., nobody will post porn because tracing the origin is only as far as checking the AD record. I have seen that intranet RM tends to be perceived as overhead (even when the very valuable objective of qualifying content could still justify the extra work) and slowly but almost certainly die under its own weight. The only place where I have seen it work internally is a company where the reputation "medals" of employees where publicly recognized and even made part of the business card (if the employee so chose).


                                              So, talking only about external communities now, let's continue by saying that the job for reputation management is not easy: it has to compensate for the extremely rich and complex nuances of human trust (the alternative is to only collaborate to the point where trust doesn't matter, which is an impossibly low threshold) . So, how hard is it to encode trust? Just think how many books have you read about the codified and immutable values, scales and colors of trust, and you will quickly realize the task is all but impossible.


                                              So, in their attempt to describe something so complex, points are just the tip of the iceberg. For a reputation management system to be effective it must satisfy many other requirements, some of which are:

                                              • Rating filters, a la slash-dot (i.e., I chose to automatically eliminate, as noise, anything that does not satisfy a certain criteria in points) – Valuable because it is assistive, as opposed to still exposing the reader to junk: once the lower limit is set, posts are automatically eliminated
                                              • Bozo filters (so that flamers keep investing more time that it takes readers to ignore him/her) – One of the key resilience aspects of Web 2.0 co-creation is that it takes more effort to vandalize than it takes to fix the effects
                                              • Distinct point systems for content than for people – The nuances of trust are different from the attributes of valuable content
                                              • Un-game-ability – This is a tough one, but somebody has already written about it: if RM systems can be "gamed", they will. That is the nature of the beast, and bozos and flamers will invest whatever is needed to play the game. There are no easy solutions to this aspect except vigilance and tuning.


                                              Sad to say, but points have not yet reached their potential as trust replacements. I believe that the reason is (a) incompleteness –only used for people, or only for content, (b) that the usability of point systems is still too  chunky in granularity (I may love something you say in the beginning of your post, but hate your conclusions) and (c) too disruptive an operation (most collaboration systems make you believe that after you assigned those points you are pretty much done... Next?  All of these issues are addressable; for example, I've participated in tests where different simple gestures could be used to indicate agreement/liking/value or its opposites in any part of a page, and thus referring to the 'object' in that part of the page. Others have experimented with sliders that qualify different attributes (did it help? is it complete? do you follow this author? etc.), and so on.


                                              My only advise to practitioners is to always remember, and highlight, that points systems are substitutes for trust, and therefore:

                                              • Avoid manipulation and increase transparency – I have also given early "seeded" staff "starter advantages", but I have always come to regret them for a multitude of reasons
                                              • Follow suit and empower those with more points, whether you like them or not
                                              • Avoid commercial translation of points, and replace with true reputation and respect indicators (offering iPods will get you easy eyes, but asking for advice will get you true advocates)
                                              • Remember that some people are just shy, and thus points mean trustability, but lack of points doesn't imply the reverse – That's why advanced RM systems tend to evolve to assign both positive AND negative points
                                              • Tell users HOW to measure their trust. Whether it helped or not is usually a poor measure. Create an easy way to read about the importance of references, the avoidance of loaded language, and the adherence to the charter of the community (Some very valuable posts are, unfortunately, posted in the wrongest of places)

                                              • Re: Do reputation points help or hurt?

                                                A few ideas off the top of my head for rewarding points. I try to focus on activities that have more benefits than points alone. Some of these are more internally focused while others are external.


                                                1. Completing profiles
                                                2. Succesfully inviting new users into the community
                                                3. Answering questions (like the reward system for that in clearspace)
                                                4. Having your content "featured". Imagine a button for admins where they could push your content to the homepage or blog.
                                                5. Donating to the community
                                                6. Having your content marked as "official"
                                                7. Completing tasks and projects
                                                8. Allowing users a pool of points to allot to various users they find helpful or interesting. This would reset over a period of time.


                                                It's super easy to abuse point systems. I remember some of our early Clearspace tests in which a snarky engineer began publishing an aggregation of news feeds to his internal blog. Fortunately that's what mods are for.


                                                This brings up another thought...


                                                What happens when you use points as punishment? Say as a deterrant to trolls, useless post, link bate, etc.


                                                I personally doubt its all that useful, as long term community members are more likely to desire points and less likely to need "punishment."

                                                • Re: Do reputation points help or hurt?

                                                  I loathe reputation points / karma / ranking systems of every sort, frankly.  My experience is that a) the negatives far outweigh any positives (especially long term), b) there are other, better ways to achieve the underlying goal (identify the "good" people), and c) they encourage social misbehaviour far more than the opposite (hence a).  That's a provocative, and no doubt controversial stance, I understand.  So before I try to explain my position in any depth, let me add some context to "my experience".


                                                  First, I am not speaking for my company here.  My company is evaluating various social tools (including CS), and is enthusiastic and optimistic about the value of reputation points. I disagree, but that's my personal opinion - no more, no less.  Second, I am a veteran both of the online forum (particularly gaming) scene and (in a prior life) a committer in some open source projects.  Thus, I have a lot of personal experience with (various) rating systems (as well as some alternatives), and am also acquainted with a reasonably sized population of people who also have broad experience with them (including a fair number of Gen Y gamers who routinely whup my ***).  So I believe my opinion is not wholly uninformed.


                                                  Having established that, why do I dislike them?  What negative experiences formed my opinion?  I'll work backwards: "they encourage social misbehaviour far more than the opposite".  In my experience, reputation systems are always an end in and of themselves.  Others in this thread have remarked on the potential for "gaming the system" -- I think that rather dramatically understates the problem.  They are always gamed.  Always.  The more advanced systems develop counterbalancing systems (as also remarked on by Carlos, among others), and this is great, but it's like being a Windoze virus protection vendor -- you're always fighting a retreating, defensive battle against the endless inventiveness of users, throwing up one Maginot Line after another, to defend against yesterday's exploit.  More importantly, they distort social behaviour.  Endless sociological and psychological studies on altruism indicate that it works most effectively in the absence of direct compensation.  Add any form of "money" into the mix, and compensate actions with it, and the overall degree of selflessness in a given social system almost always drops.


                                                  The most advanced ones (with mechanisms to vote down, as well as up), by providing a mechanism for "punishment", are also the most awful, in my opinion, although the effect is often subtle, and goes unnoticed by the participants in many cases I've seen.  Such communities exhibit mob behaviour -- they converge on unwelcome and divergent content, and "punish" it into silence.  That's what the mechanism is there for, after all! It's meant to allow communities a way to silence trolls (like, say, an "Amanda Chapel").  You'd think that would be good, but it has a huge downside -- it also stifles creativity, emergent ideas, innovation.  It produces echo chambers.


                                                  All that may be irrelevant to a public community on the Web.  In fact, there are obvious use cases where none of that is a problem, but rather an advantage.  The gaming site example has been used here before, but generally speaking, anywhere that "page views" are the goal, these effects probably have no significance.  If all you want to achieve is that a group of people frequent your site (and see your ads) on a regular basis, then perhaps none of this applies.  I would question even that, personally, from my own personal experience, with a snarky remark about the quality of the people that choose to participate in such communities, but let that pass.  What is worth mentioning, perhaps, is that my personal experience with savvy younger people (I'm Gen X), is that they tend to have either a) nothing but scorn for reputation systems, or b) a shoulder-shrugging lack of interest.  What they always have is an extremely well developed bullshit detector, which allows them to quickly assess the people with nominally "high" scores, and determine if they are really "experts", or not.  In other words, they bypass the rating system -- it's often nothing more than a distraction to them.


                                                  For an internally facing, enterprisey use case, these effects are disastrous.  And the really frustrating thing is, they become part of the overall problem space, despite the fact that the metric (the reputation point score) doesn't measure what it purports to. I'm reminded about the old sawhorse of the drunk guy looking for his watch under a streetlight.  A couple passes by, sees him and stops to help.  After a few minutes, it becomes apparent that the watch is nowhere to be found under the streetlight, and the woman turns to the drunk guy and says, "Excuse me, but we've looked everywhere, and your watch isn't here!" To which the drunk guy responds with a laugh and says, "Of course not! I lost it over there!", and points to a dark alley.


                                                  People measure what they can see -- but any halfway decent programmer will tell you: garbage in, garbage out.  As has been remarked upon in this thread already, reputation point systems are brutal simplifications of the overall behaviours you need to measure, and the simple fact of the matter is, no IT-based system can ever measure everything you need. Jevon MacDonald blogged about this recently here (http://www.fastforwardblog.com/2008/07/25/where-i-get-queasy-with-the-wisdom-of-crowds/), and I agree completely -- human social behaviour is messy, and cannot be reduced to a number.  Moreover, one of the pillars of Andrew McAfee's idea behind "Enterprise 2.0" was "emergent" -- behaviour is not emergent when it can be directly manipulated: the underlying mathematics of that should be reasonably obvious.  That's the difference between a reputation point system, and, say, an idea market (such as Ideagoras and the like).


                                                  Which brings me to my last point: there are better ways of achieving the nominal goal (identify the right people).  For example, in a system where tagging is widespread, and where the tags are correlated to the users, there's an easy (and emergent) alternative: design the system so that "experts" are the people most strongly correlated to a given tag.  In such a situation, the correlation between the tag and the user means something like "this is a person who writes a lot about this, who answers lots of questions about this, who asks lots of questions about this (and therefore has experience with it), or any combination of same". I would argue that this is at least as useful as the assumed value of a reputation point system, and has the advantage that it's much harder to game (not impossible, but hard enough that diminishing returns make it unlikely).


                                                  Moreover, I think the "social software" world still has a lot to learn from the example of open source software development communities.  Those are established, virtual, collaborative communities, which have groped their way, over the last 20 years, to a rough consensus of the best way to structure and manage such beasts.  For example, the "benevolent dictator" model is one that can be directly applied to a corporate environment struggling to adapt to the disruptive effects of social software.  The connection to reputation point systems is the widely adopted model of a self-selecting group of "committers" in an open source community -- these are the equivalent of the "experts" that you'd like reputation points to identify for you.  That mechanism deserves more study.


                                                  Thus, in summary, I think the negative aspects of reputation points, within the context of an enterprisey use case, far outweigh any positive ones.  And that's why I don't like 'em.  ;)

                                                  • Re: Do reputation points help or hurt?

                                                    Good points Mark, we've been grappling with many of the issues you raise for years.  We still haven't implemented a points system.


                                                    My wife and kids have recently started using the flair application on facebook.  As anyone who has seen Office Space knows (isn't it required viewing in the IT world??), flair are little buttons you collect.  Facebook flair is silly, stupid, ironic, and pointless so of course everybody loves it.  However, I've learned far more about people by looking at their facebook flair than I ever would by looking at their reputation rating.  It won't solve the reputation problem but it is one way a community in a non-competitive way can reveal the explicit and implicit knowledge, interests, and experience of its members.




                                                    • Re: Do reputation points help or hurt?

                                                      In general, reputation points are useful, but as a few have pointed out - they don't reward for everything they should and have some downsides.


                                                      I'd rather see a system that just establishes awards or statuses for accomplishing certain things.  These might work like ribbons in the military or merit badges in the boy scouts.




                                                      For example:


                                                      • Having the most useful answers (those marked as helpful or correct by other members) over a given period (month, year, etc)
                                                      • Participating in events
                                                      • Most useful posts (as determined by other members)
                                                      • Forming or leading groups
                                                      • Completing projects or tasks
                                                      • Contributing in other ways (donating, sponsoring face to face meet-ups, etc)


                                                      I'd rather look at someone's profile and hover my mouse over their merit badge icons to learn about the specific ways in which they contributed to the community.  Plus, I'd probably work at earning some myself!  It would be a much more meaningful "award" system than just points.

                                                      • Re: Do reputation points help or hurt?

                                                        Jim, I smell what you're steppin' in. Add to that the ability for your peers to "award" you X number of badges a year, for exemplary services. Gotta keep a maximum number in place, though, or gaming might occur.


                                                        So, for e.g., maybe you could receive up to 4 merit badges from colleagues in a year, and could award up to 8 to others. These badges would show up on your profile, and be accompanied by supporting text that the awarder is required to write when they give you the badge.


                                                        Sounds Facebooky, but I'm thinking it'd actually be useful in the enterprise.

                                                        • Re: Do reputation points help or hurt?

                                                          This may sound a little odd, but what about XBox Live's Gamer Point system? Games are inherantly competitive and so a point system is more readibly adopted. I think you could do something similar for a community.


                                                          On the XBox, each game has a set number of points that they can use to reward players who accomplish certain tasks. It's up to the publisher / developers to determine what the Achievements are and how to distribute the points. Each has an associated name, icon and point tally. When a player completes an achievement they get instant feedback about this accomplishment within the game and the points are added to their total.


                                                          This may be a bit of a leap for some, but imagine this in a community.


                                                          You start a project in your internal community. The project manager gets a pool of points to award folks for contributing. The manager has complete control over how they are awarded. People complete tasks or assignments and thus get points for being awesome.


                                                          This could get pretty draconian very quickly, but I imagine there are a variety of tasks in any organization that a point system could work well for.


                                                          Someone more creative than myself could probably come up with something better. Any ideas?

                                                          • Re: Do reputation points help or hurt?

                                                            Interestingly enough, we're having a parallel discussion on "how do you best motivate community participation?" on our internal platform.


                                                            Wish you could see it -- as a result, our model has evolve to a new level.


                                                            We've come up with a schema that starts with the core give/get wins that drive the community at the outset (e.g. recognition, exchange of knowledge, altruism,  competition, getting to know your peers, etc.) and then think in terms of reinforcement mechanisms that support the underlying motivation for a particular community.


                                                            Taken that way, point systems tend to work well in communities that have an underlying dynamic of either (a) competition, or (b) being recognized as a helpful, engaged person.  And, conversely, they don't work so well in communities that are based on, for example, quality of conversation or knowledge shared.


                                                            As a counter-example, we have an internal blogging community, and a while back, we ran a nomination/vote contest for best internal bloggers.  Worked better than a points system, I'd argue.


                                                            Made us think of the problem differently, it did.

                                                            • Re: Do reputation points help or hurt?



                                                              Lots of great points in your response.  I think your experience and recommendation on the tag-based solution to the reputation question offer a very valid strategy for companies adopting this technology for more widespread use.  One thing that I see as a challenge is getting older executives to consider the time spent on contributing to the community as a valuable job activity.  There is some resistance to this, and I think reputation ratings/management often get thrown up as a means to police the activity and provide a demonstrable way to measure direct contribution to company success in order to justify not just the investment in the technology by the investment by employees to contributing to a community.   However frustrating it may be, I still think many business leaders are suffering from personal-oriented social community hangovers, and as such are looking for a business metric framework to establish a punative (whether explicitly or implicitly) means of evaluating employee work activity.  But your counter argument is well made and opens a more useful way to help community members connect with the right people and provides a means for management to create communities focused on value creation.

                                                              • Re: Do reputation points help or hurt?

                                                                I think that's a good point - as soon as a newbie puts something in, they become a contributor, so that should be acknowledged immediately.

                                                                • Re: Do reputation points help or hurt?

                                                                  Mark - if I thought you'd appreciate it, I'd award you a pile of brownie points for that post ... well said.


                                                                  If we think more in tagging terms, then maybe we should be giving kudos to others via a form of attribution (I use the "via:" tag in del.icio.us for instance - anybody that knows me is probably aware it's something of a hobby horse of mine!) or link back to useful information/answers, and over a period of time people who generate useful content will become more obvious. It's not as simple as "reputation points", but (as pointed out already) this is not a simple issue.

                                                                  • Re: Do reputation points help or hurt?

                                                                    I had to doublecheck on the medal icon the first time I saw it here.


                                                                    My first thought was it was a reputation kind of marker.  The Citrix forums use icons like that for their reputation system.  Actually it's cool how it works...


                                                                    When you ask a question in the forum, you assign the question a difficulty level.  The more difficult it is, the more points it's worth.  If someone answers the question completely, you mark it answered and give them the points.  If it's more of a collaborative answer, you distribute the points among the people that helped.


                                                                    Participants get certain icons next to their name depending on how many points they have from answering open issues.

                                                                    • Re: Do reputation points help or hurt?

                                                                      Apple (another Jive customer ) uses a scale where you earn a certain number of points for an answer that is considered 'helpful' by the originator.  You can get a greater amount of points if you answer 'solves' the question.   The combination of the points earned (qualitative measure) and the total nubmer of posts (a bit of an old school quantitative measure) gives me a better feeling when I'm judging if I should follow the advice or not. 


                                                                      At a base level, its a way to get people to engage.  And if that helps participation for the target audience, then great.  If the converse is true (e.g. people just aren't into it), then turn it off.


                                                                      The key is having a sense of how your target audience will react to it.

                                                                      • Re: Do reputation points help or hurt?

                                                                        I think they work, although we haven't taken full advantage of the way to use them.  I'd like to have a contest for our users to reward (materially) the most active users.

                                                                        • Re: Do reputation points help or hurt?
                                                                          michaeljpastor Novice

                                                                          My background is with Jungian psychology for definitions of my terminology, so please bear with me if its new to you.


                                                                          When trying to design a reputation system, its important to recognize that it's ultimately derived from one's subjective opinion, despite the fact that it has objective pieces and parts. Context is also important.


                                                                          Objective in Jungian terms means "shared and compromised" to summarize a complicated concept.  Subjective means unique and personal.  Ultimately the things that I think matter in terms of a person's reputation are going to be different than yours, but ultimately I probably will, in some manner, "poll" people who also know the shared/objective third person, and ask others in shared objective terms, such as "politeness," helpfulness," "participatory-ness," or some other tag or label.


                                                                          So when we create a measurement, such as "was Joe Schmoe's answer helpful" we're creating a shared, objective measurement.  That doesn't speak to Joe Schmoe's tendency to answer questions in the most derisive and patronizing manner possible (and from my personal experience, when dealing with OpenSource forum support systems, that's usually how it happens!).


                                                                          Now Don Doozer also tends to be neck and neck with Joe Schmoe with the helpfulness of his answers, and he's a lot nicer.  It turns out he's an IT guy for a non-profit organization that deals with quite a few computer illiterate people, so there's a background to why.  There's nothing in the rating system to give him "nice" points, because I'm depending on a third party (a shared objective intermediary) to create a "niceness" meter.


                                                                          Now I could create my own niceness meter label to apply to people, but it only contains my ratings, and doesn't benefit from wisdom of other people's opinions of Don's, and thus far, there's no way for me to share it.  Now a system could allow for me to publish (share objectively) my subjective metering system, but that's not always desirable (particularly since I don't want Joe to know that I think he isn't nice).  One is entitled one's opinion, but you're also entitled to keep it to yourself; and I'm sure we wish more people would exercize that second entitlement sometimes!


                                                                          But let's say I have a peer group of people with whom I'd like to share my niceness meter, and whose opinions of niceness I hold in high regard.  I decide I'd like to incorporate their opinions as well.  But I'd like to keep my opinion as the primary meter, and only let their collective opinion "tweak" the result.


                                                                          But there's a person in my peer group who I know is a poor judge of character, and actually thinks people who are rude are actually being mercifully nice.  I'm not a fan of his opinion, despite the fact that he's a great peer in the current context.  What's actually happening is that we don't objectively share the definition of "niceness."  Do I have the ability to tweak the formula to incorporate my (low) opinion of his (inappropriate) opinion?


                                                                          But wait, let's say there's a meter that I haven't considered (timeliness of response, for example) that others have considered.  Is there a way to "suggest" that meter, or subconsciously incorporate it?


                                                                          It turns out that if I incorporate that last meter, some dark horse candidate comes around the bend and ends up winning the Kentucky Derby, and that this dark horse happens to have an extremely high reputation in the opinion of both Don and Joe!


                                                                          This is getting pretty complicated isn't it?  Well, so are reputations.


                                                                          Ultimately, any reputation system that depends on "global" objective/communal/shared systems is going to be gamed and ultimately fail in the relevance game.  Once you know the rules, its easy to break them.  But its impossible to game a system when it isn't known by the gamer, because that system is private, subjective and personal.


                                                                          Reputations algorithms are ultimately subjective and therefore any attempt to quantify them must be customizable by the person depending on their results.  In order to be customizable, they must be transparent.  In order to be useful, they must be understandable by the end user who may not understand algebraic equations.


                                                                          Reputation is going to be one of many Next Big Things in the next generation of Social Software, and unless it takes context, subjective and objective concepts into account, it will boil down to devolved popularity contests.




                                                                          • Re: Do reputation points help or hurt?

                                                                            Wow and Thank you.  Very insightful.


                                                                            • Re: Do reputation points help or hurt?

                                                                              Gil - LOL; that was pretty much my reaction too ... that was a clear, concise summary of what is wrong with reputation mechanisms at the moment, but also highlights why it would be useful to get them right.  Somebody give Michael some reputation points!!

                                                                              • Re: Do reputation points help or hurt?

                                                                                The first part of your post reminds me of Competent Jerks and Loveable Fools and the Formation of Social Networks. When I speak of expertise location, I always say, "what you're really looking for is someone who knows what you need to know, and is nice enough to help you."


                                                                                One thing we're discussing lately is the value of some sort of valid recommendation mechanism. As you say, so much goes into how we value recommendations from others that it is somewhat silly to try to assign an objective points system to such a thing. Instead, an idea is to simply make it easy to publicly recommend someone with a short explanation of why, and that recommendation shows up on their profile along with the person who made it. That way, an individual would not only be able to evaluate the reason for the recommendation, but weigh its value based on his/her relationship with the recommender, or perhaps their relationship with someone in their network who recommends the recommender. To me, that would be much more valuable.


                                                                                I wonder, however, if people would ever make those recommendations.

                                                                                • Re: Do reputation points help or hurt?
                                                                                  michaeljpastor Novice

                                                                                  My first attempt at a reply was lost to the ether, but I think my "keep me logged in" check box should stick this time.


                                                                                  Your kudos and concerns about recommendations are both spot-on.  For example, I've been stuck at 90% on my LinkedIn profile because I lack two more recommendations.  Now I actually have an issue with that, because it's not my fault - it isn't my responsibility to have a recommendation; the onus is on others to recommend me, not for me to solicit recommendations.  I hate grovelling, brown-nosing and otherwise acting as a salesman.  For me, the incentive (for the goal of 100%) should be the other way around - recommend 3 others.


                                                                                  It may seem silly to use a Likert scale numeric metric for a recommendation, but it does have some value.  If it's one of a few (or several) metrics used in the final rep algorithm, you'll be able to tweak its influence in the final "score" (presuming you have a way to view the 'rithm transparently and can change it).  How you tweak that Likert score can be influenced by the commentary that accompanies it.


                                                                                  It's pretty easy to give someone a "high five" when you're enthusiatically entranced by them, but if you're being overly gushy in your commentary (in someone else's opinion), that enthusiasm can be tempered by one's opinion of their opinion.


                                                                                  Let's presume it's a +10/-10 scale with commentary (I prefer 10 point scales over 5, because it allows for more nuance).  The final outcome of the score would be affected by


                                                                                  1) my (subjective) relationship "score" (if any) with the recommender.

                                                                                  2) my (subjective) relationship "score" (if any) with the recommendee

                                                                                  3) my (subjective) opinion of the recommendation based on the commentary

                                                                                  4) the (objectively) collected (subjective) opinions of the recommendation

                                                                                  5) pick a metric, any metric.

                                                                                  6) and for their final rep score, how much recommendations factor into the final number


                                                                                  and of course we could go on ad infinitum with other factors, and none are really necessary.  Having both a number and commentary provides for both subjective (free-form space to gush) and objective (a shared standard of measurement) factors.


                                                                                  Privacy about recommendations is important as well.  There's not much incentive for me to give a negative recommendation if my name is attached - I don't necessarily think I need to own up to my comments, if my aims are altruistic and I'm trying to give constructive criticism.  But privacy also allows for libelous commentary.  Private recommendations could be allowed with a moderator/broker approval, if keeping things civil is an important aim, and you can even allow for hand-picked brokers/moderators.  You could always subjectively "turn off" anonymous commentary but still allow for it.


                                                                                  Again, this is getting exponentially complicated, because it is.  Ultimately a remote web-hosted SAAS platform (an objective/shared space) can't handle it, because it's infinitely subjective and requires (IMHO) a directly accessable desktop application.  A peer-to-peer (or persona-to-persona) system is ultimately the only way to handle it.  That's not a reason for SAAS companies to become despondent however.  They can assist in many ways, by enabling the basic metrics and standards, by (objectively) "aggregrating" metrics on my behalf, by storing or processing via HAAS'n'SAAS.  I think the best thing that SAAS companies can do is contribute to OpenSource initiatives that enable end-users to view and tweak rep systems, and make their own inhouse rep systems compatible with others and accessable via APIs.  Short-sighted companies will fight this, but they'll ultimately lose.

                                                                                  • Re: Do reputation points help or hurt?

                                                                                    Wow, lots of responses on this!


                                                                                    I don't think that reputation systems always work well in external communities; it really depends on the theme/subject of the community itself.  On top of that, there are two different types: point systems with accompanying status level (such as Beginner, Expert, etc.), and those that specify a status level based on a specific characteristic (such as identifying Jive employees here with a medal, regardless of points).


                                                                                    I think that point systems make sense for communities where the value of a member's contribution directly correlates to their experience and credibility within the community, such as most tech-based communities....I'd be more likely to take advice from an "Expert" with 1,000 points than a "Beginner" with 2 posts.


                                                                                    A point-based reputation system wouldn't make as much sense in a community where you want everyone to feel welcome and equal, such as a community for cancer patients and survivors, which was the first type of community that I helped to plan and develop (Barry gave us great advice on this).  In that type of situation, you wouldn't want to disengage new or potential members by implementing a reputation system based on points.  Everyone is either looking to give or get support, and the value that someone has to contribute in that area is not relevant to their point level.  This is a case where a reputation system with an accompanying icon/title based on a characteristic such as "Just Diagnosed", "Survivor", "Caregiver", or a type of cancer, would be more relevant and valuable, and members would be able to more instantly recognize and connect with new and existing members.


                                                                                    That reminds me, is there any plan to include text beneath status icons in Clearspace? It would be much easier for new members here (and in other Jive communities) to identify Jive employees.  As an alternative, an image with text included could be used, but a text display would be more ideal.

                                                                                    • Re: Do reputation points help or hurt?
                                                                                      I'd rather see a system that just establishes awards or statuses for accomplishing certain things.  These might work like ribbons in the military or merit badges in the boy scouts.

                                                                                      The idea of awards/statuses mimics nicely what we see in the physical world. I can reward someone with more than just a thank you when they provide helpful information or resources. I've worked in places where people would give each other "$10,000 Bucks" or "$100,000 Bucks" bills to thank someone. Or, of course, you could really score by getting a corporate-branded paper weight -- we all need a desktop full of these.


                                                                                      I'd rather look at someone's profile and hover my mouse over their merit badge icons to learn about the specific ways in which they contributed to the community.  Plus, I'd probably work at earning some myself!  It would be a much more meaningful "award" system than just points.

                                                                                      Just as we provide emoticons to visualize our emotions in an email, let's provide a selection of awardicons or meriticons that I can provide a definition for and present to a deserving colleague.

                                                                                      • Re: Do reputation points help or hurt?

                                                                                        Interesting, I thought the medal meant you were a high level contributor or was related to the ratings of your contributions... i've only been playing on the site for a hour though... but just my first impression.

                                                                                        • Re: Do reputation points help or hurt?

                                                                                          I started reading this thread and only slowly realized how very long and winding a road it would turn out to be. I have two separate contributions to make.


                                                                                          First, we use the status points in Clearspace with our internal installation. We're about 5 months into our implementation. At first I used to barely mention the points when I did initial overview demos and training, but I quickly learned that the points really got people's attention. We seem to have many competitive people in our company! They enjoy the points, take them as nothing more than fun, and love to rib their colleagues about who has more medals or a more prestigious title.


                                                                                          While I worried about people gaming the system, it does not seem to have become an issue for us. I've published our levels and what it takes to achieve them, and that's become a popular page.


                                                                                          My second comment is on the term "reputation points." Since status level points are awarded in Clearspace primarily for creating content, I don't consider them measures of reputation. They are measures of activity, showing at a glance who has contributed the most in terms of content to the community. They are brownie points, gold stars, "attaboys," or whatever you want to call them, but they do not in any way truly measure one's reputation.


                                                                                          The only exception is that some points are awarded for correctly or helpfully answering questions. But I have no way to know when I see someone with 1000 points whether they got those points answering questions or dutifully posting a daily status report.


                                                                                          We would like to have a reputation measurement system. This thread has made clear that doing something like that well is no easy task, and perhaps is not even possible, yet. I really like the idea suggested above by Mark Masterson about using tags -- I think it has great potential to be both automated and useful:


                                                                                          Which brings me to my last point: there are better ways of achieving the nominal goal (identify the right people).  For example, in a system where tagging is widespread, and where the tags are correlated to the users, there's an easy (and emergent) alternative: design the system so that "experts" are the people most strongly correlated to a given tag.  In such a situation, the correlation between the tag and the user means something like "this is a person who writes a lot about this, who answers lots of questions about this, who asks lots of questions about this (and therefore has experience with it), or any combination of same". I would argue that this is at least as useful as the assumed value of a reputation point system, and has the advantage that it's much harder to game (not impossible, but hard enough that diminishing returns make it unlikely).




                                                                                          I'd love to see Jive work on that, and if they need any customer input or a working committee, I'd be happy to help.

                                                                                          • Re: Do reputation points help or hurt?

                                                                                            This is an interesting question.  I've sort of commented on this topic as it pertains to adoption of social networking sites.  (Blog here: http://mattmaccaux.blogspot.com/2009/05/social-networking-for-masses.html)


                                                                                            I think that recognition points, rankings, badges, etc should only be used to rank content, not people.  The reason for this (in my opinion) is that by explicitly ranking people, you end up excluding (by discouraging) those who don't have high ranks (or post counts, or rep points, or whatever.)  The idea is to encourage people to add value to the process through adding content, opinions, etc and anything you might do to discourage this (either directly or indirectly) should be avoided.

                                                                                            Ranking the content, however is a good idea.  I mean, isn't the idea of enabling the knowledge worker to find relevant content quickly the whole point of social tools for the enterprise?  Ranking the people is only relevant to the point that their rank impacts the rank of the content being posted or commented on.  However, the person's rank should be an implicit attribute rather than an explicit one so that people can not be ranked (explicitly) by their peers.

                                                                                            • Re: Do reputation points help or hurt?

                                                                                              What about sites where members are paying for access?  Any examples of that, and how did/does/should that change your decision about whether or not to use reputation points?

                                                                                              • Re: Do reputation points help or hurt?

                                                                                                Reputation points do not hurt in my opinion.. They give members of your community a feeling of involvment and a bit of a competitive spirit, because people are competitive "animals" by default

                                                                                                • Re: Do reputation points help or hurt?

                                                                                                  Coming late to the discussion...


                                                                                                  Its useful to remember that there are four generations in the workforce. Each will likely view a ranking system differently as will different personality types.


                                                                                                  As has been mentioned, some will certainly try to "game" the system if there is a perceived advantage in doing so.


                                                                                                  Others my be completely altruistic in their approach and not care one way or the other.


                                                                                                  If implemented correctly a point system will allow individuals with less experience in an area to accept and respect the input and opinions of others who have earned meaningful reputation points.

                                                                                                  • Re: Do reputation points help or hurt?


                                                                                                    Reading through the post, I did not get a clear idea of "why".  What are we trying to accomplish with the rating system?  If this is a way to recognize people that contribute (however that is defined), then you need a point degradation system like the one developed by Austen. People need a way to move up and continue adding value, not resting on the pile of stuff they added at the beginning.  If this is connected with performance or acknowledged in a review, then people will understand that this is valued.


                                                                                                    If the purpose is to highlight examples of "good' behavior or actions, then using status level icons associated with a group provides a way to do that.  Of course having a small icon next to my name that people need to hover the mouse over to find out what it means does not provide a big bang.  Creating a box on the page with their photo, name and why they are special provides more of a boost.


                                                                                                    Intrinsic rewards that get unexpected acknowledgement and recognition are more valuable.



                                                                                                      • Re: Do reputation points help or hurt?

                                                                                                        Hi Mike,


                                                                                                        Why? - Why - because we like you! (perhaps you'll know where that came from)


                                                                                                        I also re-read a majority of this thread.


                                                                                                        I saw a wealth of great information and great points made about reputation points.


                                                                                                        I also agree that Austen's add-in is a better attempt than a simple tally for participation.


                                                                                                        In the few months that I've been participating here in Jivespace, I thought that raising my point count would be of some value


                                                                                                        I recently attended a presentation by Jared Spool where he discussed some UX/UI "Secrets from the Amazon". Jared noted that you need to engage through content. On Amazon, user reviews can be viewed as content. It wasn't until Amazon provided a way to say that a specific review was helpful that their review system became useful. Oddly enough, Target uses Amazon's platform (under lease) and their review system isn't particularly helpful - due mostly to negative reviews. Amazon ends up with useful positive reviews due to the sheer volume of use (they had 86 million site hits in Dec. '09 alone - they process about 24 orders a second on average).


                                                                                                        I guess the point I'm suggesting here agrees with one made above that ranking content is possibly a better way of directly influencing the system. The content author can then benefit indirectly through a scheme somewhat more complex than a simple: number of posts count (as per Austen's add-in).


                                                                                                        By allowing user ranking of other supplied content coupled with something like Austen has devised a better system could be put in place that would allow users to evaluate content.


                                                                                                        The Q&A system on Linked in is supposed to allow the original poster of a question to rank the best answer. Individual profiles then show that a particular person supplied "n" best answers in a category. If my questions was about "x" and somebody with 200 best answers in "x" were to reply, I'd certainly place more stock in their reply than in the reply from someone with no "best" answers.


                                                                                                        The flip side of that is the silly way that Linkedin defines an "expert" in their Q&A area. Some answers are given to simply drive a user's answer count - even if the answer posted is worthless.


                                                                                                        In a social system where there is no entry criteria everyone starts out "the same". A point system attempts to provide users with a way to differentiate between a "newbie" (who may or may not have valuable content to provide) and an experienced contributor of valued material.


                                                                                                        In between are the lurkers...


                                                                                                        ...no points for lurking.

                                                                                                      • Re: Do reputation points help or hurt?

                                                                                                        I really like the idea of allowing members to determine if someone should be recognized for their assistance.  This helps "ground" the contributions to something that at least someone likes or feels is of value.  I liken it to a virtual "thank you".  I also agree that this needs to be capped or governed in some manner.

                                                                                                        We've decided to turn off both points and badges in our internal community but that doesn't mean that our culture didn't allow it.  It was more about the tone of what points and badges (in their current form) say.  We may turn them back on, but for now, its appears to be one of the smaller decisions on our plate.

                                                                                                        • Re: Do reputation points help or hurt?
                                                                                                          grantc Advanced

                                                                                                          Hi Gia Lyons, whats the latest on plans to improve the Status points system? Lost has been raised in this forum over the last few years but not much has changed. We definitely believe that the star rating given to someones content should in someway impact on their points - we need quality as well as quantity measures!



                                                                                                            • Re: Do reputation points help or hurt?

                                                                                                              Hi Grant,


                                                                                                              Please see my response to your duplicate post on this in Internal Community Managers. Discussions about feature requests or improvements should be held in Products and Feedback, where Jive's product managers are active. This area is dedicated more towards sharing best practices and asking and answering questions about the business applications of Jive (as the original post in this thread demonstrates).




                                                                                                              Sent from my iPad

                                                                                                                • Re: Do reputation points help or hurt?
                                                                                                                  adam.mertz Beginner

                                                                                                                  Grant, just a couple quick thoughts since to your question.  Actually a ton has been done on this and delivered over the past couple years.  I've been on Jive's Product Marketing team since 2007 and talk and work with our clients on questions like this regularly.

                                                                                                                  First, just to back up a bit, status points is just one aspect of the broader reputation and rewards category.

                                                                                                                  People care about their reputation, online or not, but it's not just about points.  In Jive 5 you now see things like -

                                                                                                                  Trending people (on the What Matters Activity page)

                                                                                                                  trending people.png

                                                                                                                  Trending content

                                                                                                                  Latest likes


                                                                                                                  Latest Acclaim



                                                                                                                  All of these new pieces directly correlate to showing what people like and ultimately a person's reputation and status.  And of course every social action is completely configurable as to how many points are derived from each.


                                                                                                                  But wait, there's more -

                                                                                                                  We now have some cool apps like Rypple that focus explicitly on enabling users to highlight another user's good work or thanking people, example -


                                                                                                                  In addition, we have a framework that supports badging which several of our clients leverage such as Cisco's support community or Avon's community.  We're thinking and doing more in this area as well so stay tuned.  But those are a few examples.

                                                                                                              • Re: Do reputation points help or hurt?

                                                                                                                Adam, where is the Jive 5 status points system documented? I'd like to understand exactly how it works, in admin console detail.


                                                                                                                We have been using Austen's Social Leaders plugin for one of our communities. I need to know whether to invest in upgrading that for Jive 5 or whether Jive 5's status points scheme will meet our needs.

                                                                                                                • Re: Do reputation points help or hurt?

                                                                                                                  I'd love to see points categorized by topics. So while I might have 300 points overall, maybe I could have 100 general points as well as 200 points gained through activity and feedback in areas tagged as "community management". Because a forum I'm in can be tagged with multiple terms, I might have 300 overall points, but 200 in "community_management," 75 in "reputation," and 150 in "user_adoption," etc.


                                                                                                                  This would be of greatest help when the company is trying to use Jive for expert identification. There might be some benefit though to displaying multiple badges/scores/grades for a user in contexts where particular tags are in play. So in this forum for instance, if I had 300 points overall and 200 in community_management, you'd see both scores.


                                                                                                                  Any hope of that sort of functionality coming along soon?

                                                                                                                    • Re: Do reputation points help or hurt?


                                                                                                                      I think the system does this currently.  The top participant widget shows the points for that space.  So even though I have a the top number of points for all of the communities, I am a novice with few points in specific spaces.


                                                                                                                      I am not sure if that information is captured in the analytic data.