I'm definitely a fan of Andy Sernovitz's blog, and his recent installment was particularly wonderful - if you haven't seen it yet then be sure to check it out: Home is Where Your Mug Is. The basic idea is simple:

Starbucks should give their best customers a personalized mug.

Giving out free stuff to special customers is nothing new, but a free glass or cup is only the most superficial part of this idea. The key is in the personalization - make your customer feel unique and special by doing something out of the ordinary and personal just for them. Sure, it takes more work. But your customer now knows you care enough about them that you would go to such lengths to celebrate them. Whenever they use it, they have an opportunity to talk about the gift and why they were given one. Seeing someone else with a such unique item may make other customers wonder why they don't have a relationship like that with their shop. And every visit to your place of business becomes a special event.

We know our top super users and influencers in our communities very well - can you imagine what wonderful, personalized rewards you could give them that they can talk about and display, both on-and-offline? What have you done to make them feel special?

photo by Viernest

Today we introduced Lithium Insights, a set of analytic solutions focused on enterprise online communities.    The first two solutions in the set are the Community Health Index (or CHI), and Lithium Lifecycle Benchmark Services.    I think these solutions represent a significant leap forward in the way online communities are measured, and I thought I'd give you a little insight into what they are, why they are important, and the road we took to developing them.   I'll also give you a peek at what's coming down the road from our analytics group at Lithium.

The Lithium platform has always had a pretty rich metrics tool inside.   There are currently more than 150 unique metrics in the system - everything from common metrics like posts and page views to relatively more arcane ones like wireless views, or net anonymous blog comments approved.  You can see these for the community as a whole, or for a unique category or element within a category (blog, forum, chat, or ideas).  With a single click you can see them by quarter, month, week, day, or hour, or you can choose custom ranges like last Tuesday to midnight last night.

So - lots of data.  Typically the way we've approached this data is to choose four or five high-level metrics indicative of community health - such as those Heidi Cohen shared in her column this week - and then used the remaining 145 metrics diagnostically.   That is to say, we watched for trends in high-level metrics, and then used more detailed metrics to better understand the trends we saw.   Here's a simple example I like:   In a forum, if you see post count dropping over time, you know you have a problem but you don't really know why, and you don't know what to do about it.  If you then pull data on threads and replies - essentially breaking post count down into those two different kinds of posts - you can see whether the drop is attributable to fewer questions (threads) or fewer answers (replies).   If the former, you probably have a problem with how you are promoting the community, since questions typically come from new users.  If the latter, you probably have a problem with how you are managing superusers, since they often create 30-40% or more of the replies in your community.   Pretty simple, pretty effective. 

But there are a couple of reasons why that breaks down over time.   First, almost all high-level metrics in a community are retrospective in nature.  Registrations, posts, page views, searches, visits, etc., all tell you a lot about yesterday not much about tomorrow.    So you can address problems when you see them, but you can't really predict them or prevent them.    Past performance is no guarantee of future results, as they used to say on Wall Street when there was a Wall Street.

Second, looking at high-level metrics in isolation can be very misleading.   I know from experience, for example, that page views are the most lagging of indicators - they can and do continue to scale long after your community begins to slide into failure. 

Third, high-level metrics become harder to read as a community matures.   Think about it - in the early weeks and months, a steep climb in page views and posts makes it pretty obvious you're succeeding.   But those growth curves tend to flatten over time, and in years 2, 3, 4, and 5, it can be harder to tell whether your community is really healthy or not.

At Lithium, we've been working with communities for more than 10 years.  As a software-as-a-service provider, we've followed all our communities continuously over time, and learned a lot about what a healthy community looks like.  Some of this is quantitative and some of it is frankly intuitive.   So how did we convert this knowledge into a quantitative index of health?

Here's how we did it:  we asked our statistician, Michael Wu, to create a model of community health.   He gathered the best of what we know from our most experienced community experts - I had a role in this - and he tested our knowledge and intuition against real data from our ten-year database of enterprise communities.    Essentially, we played hundreds of games of "Kasparov versus Deep Blue" over the past six months, testing our knowledge against Michael's model.

It wasn't always fun being a Kasparov - I think you can guess I didn't always win - but I learned some amazing things.   One thing I learned is that human beings always overestimate the health of large communities and underestimate the health of small ones.   Another thing I learned was that that quality we associate with communities that "feel like a community" versus pure question-and-answer actually has a statistical indicator.   After going through this exercise, it's clear to me that there's a vast amount yet to be learned - and that good tools for measuring communities are an absolute necessity for any of us working with communities today.

Today we published a whitepaper that shares our statistical model - we call it the Community Health Index, or CHI (rhymes with buy).   In the whitepaper you'll find a description of the model and all the formulas.   Like any statistical effort, it's a bit forbidding for the mathematically challenged (like me), so I'll be exploring simpler ways to describe it on this blog in the coming days and weeks.  But do check it out and let us know what you think.  To see how we're incorporating CHI in our reports, see this morning's blog post by Neil Beam, who led this effort at Lithium.  

We have created CHI as an open standard - the inputs are limited to data available from any community platform - both as an aid to anyone seeking to measure their online community and as a way to open the model up to exploration and critique by others in the field.   I'm sure we'll be tuning the model over time to increase its accuracy and relevance.   But I'm thrilled we've started on this journey, and I hope you'll join us, or simply follow along.

One final note:  community health is critically important, of course.   It is a measure of community success from the user's perspective.    But there are two large areas of measurement that go beyond community health.   One is return on investment - in other words, success from the business perspective.   The second is community insight and sentiment - what are users saying, and how do they feel?    Both are on the roadmap for our Lithium Insights program this year.   As with health, there are measures for both today but there is much, much work to be done to make them more reliable, more precise, and more useful.  You can trust that we will apply the same level of rigor to those efforts as we have with CHI.   

Something special today: a quest post from Lithium's own Neil Beam, Director of Enterprise Programs in our Client Services group. Neil's recently been spending all his time immersed in metrics and numbers to help us describe what makes your community tick (and more importantly, how to keep it going strong). Today he's here to share some of the though that went into developing Lithium's newly announced analytics offering:

We are releasing today our new Lithium Insights suite, and Scott invited me to talk about it in more detail here on his blog.

We have amazing projects in the wings but today the first thing I want to show you is the Community Health Index.

The Index is the first step towards an industry standard – focused on what community practitioners should measure, report and be held accountable to in their daily practice. It does two things: 1) serve as a absolute measure of communities where you can stand them up side-by-side and say – “okay, now I know how to compare these communities” and 2) provide actionable measures that tell practitioners what to focus on first and what to do next that are specific and relative to the individual community – 6 health factors do this. Three are predictive and three are diagnostic. Note, we picked these 6 factors (Liveliness, Interaction, Responsiveness, Members, Content, Traffic) because they are an universal denominator common to most community platforms, even Twitter could fit this paradigm.

Case in point, when I was the project owner of a community in my past position I quickly discovered that simple metrics (page views, posts, registrations) only gave my management team a partial snapshot and single dimension of a very complex and dynamic system. It never felt good. How did I compare for my executives the other 5 communities on the Lithium platform at this company which were built for different products and completely different audiences and launched at different times? The Community Health Index addresses this.

So what does the Community Health Index look like?

The atmosphere or tone of your community is hard to measure. But achieving a positive tone is critically important to your community success. Like a massive black hole you can't see it directly, but a negative tone warps everything around it and pulls it down. That's why community management and moderation are so important: effective community leadership in your terms and guidelines, the welcome messages and content you create, and in every interaction online with your members influences the overall mood of your community. Each is an opportunity for you to positively engage your members to elevate your community tone.

Don't let negativity sap the energy and light from your community:

Be transparent: Establish guidelines for member behavior and cite them when taking action.

Be positive: Don't rise to the bait when a member is frustrated, and be twice as nice online as you are in person.
Be respectful: Warn your members politely and in private, not in public. There's no need to embarrass anyone.

Be consistent: Document the actions you take with members (and be sure to read the notes of others).

What are you doing to keep your community a warm and sunny place?

photo by Dana Berry of SkyWorks Digital (found posted on flickr by thebadastronomer)

One of the things I love about being a best practices manager for a SaaS company is how we are constantly learning and refining what we know. Even though we've been building and managing successful communities for over 10 years, we are always finding out new things from our customers or finding new ways to look at old topics.

A case in point: Reputation. We used to think that every community was unique on how they should set up their ranks, the more creative the better. And for some communities, particularly gaming communities, this strategy works very well. But early on we noticed an interesting trend: the higher the member traveled up the rank structure, the more serious they became about the rank they were given. In some cases we saw members putting these achievements on their resumes! So while creativity is still good, the upper ranks had to be meaningful for these members.

Stemming from our original ideas about creativity, our product defaults at first began with just 7 ranks to cover the different areas of progression, the idea being that you start with a framework and then fill it in with your own creative names and progressions. But while some of our customers went to town with their rank structure, others were content to rename the existing ranks and leave the structure more or less intact. What we saw was a pretty dramatic difference between these two approaches over time:

1. The average number of ranks across our most successful, mature communities was 31.3
2. There is a correlation between the number of ranks you have and the ratio of super users in your population (the more ranks we saw, the higher the ratio).

To be clear, we haven't shown a clear causal relationship between these two factors but it was very interesting to see that happen across so many communities. And using this information we have been able to redesign our default recommendations for our customer's initial rank structure and assist them in implementing the improvements.

Are your ranks effective? How does your community compare with the above?

Photo by gruntzooki

It seems like a chicken and egg problem: you need more members to generate more activity, but you need more activity to attract more members. How do you get past this hurdle? If you are promoting the community well a steady influx of new members will help, but its not enough; you need to keep these folks coming back for more. A good reputation system can give members visual evidence that their efforts are appreciated and get them more and more involved in your community's success.

And as communities mature reputation becomes more important, not less. With a lot of activity going on new members want to see that their voice matters, and existing members want to continue to be appreciated for contributing. No one should ever be stuck in a rank with no hope of getting out, no matter what level they are at; keep them moving in their path as long as they are still participating. That's why you need to continually reward members both for their volume of participation as well as the different ways they contribute. Keep them involved and coming back.

Reputation is only one part of how you recognize members, but it is the most persistent and visible in the community. So it makes sense to spend time developing and fine tuning your reputation system to get it right.

How are you encouraging new and existing members to become progressively more involved in your community? Or are your members stuck in a dead-end rank?

Photo by woodleywonderworks