Setting Standards for Social Media Measurement

It has only been a few years, but it is clear to many people that social media has moved beyond experimentation and anecdotes. Public, nonprofit and commercial users now need to justify why they continue to use social media and, in some quarters, are demanding evidence that it impacts bottom-lines - whether they are financial returns, social impact or behavior change. And it is clear that the newest versions of bean-counting exercises - tabulating things such as 'followers' and 'fans' - are no longer sufficient for defining "success." Such 'beans' are easy to measure, but not particularly valuable.

There is also a need for standards in social media measurement: a common language that is transparent, meaningful and able to drive strategy across organizations. These standards should also be applicable across disciplines and communities whether they are advertising and public relations agencies and their clients, academic researchers trying to decipher the inner workings of social networks, or program planners who need to decide if, when and how to allocate their resources to social media platforms. Common standards for social measurement should also increase the reliability of the methods that are used and the data that are presented to support the use and effectiveness of social media.



These were some of the arguments for setting standards for social media measurement at the 4th European Summit on Measurement by Tim Marklein and Katie Paine in their presentation of the work of the #SMMStandards Coalition. This group has identified six priorities for standardization:

1. Content sourcing and methods
2. Reach and impressions
3. Engagement
4. Influence and relevance
5. Opinion and advocacy
6. Impact and value

Here are some excerpts from their presentation:

• Organizations need clearly defined goals and outcomes for social media.
• Media content analysis should be supplemented by web and search analytics, sales and CRM data, survey data and other methods.
• Evaluating quality and quantity is critical, just as it is with conventional media.
• Measurement must focus on conversations and communities, not just 'coverage.'
• Not all content venues, aggregators and analysts are created equal. Social media measurement success stands or falls on the quality, scope and methodology of content analyzed, as well as analyst experience.
• Multipliers should not be used to calculate social media reach; in fact, dividers are more appropriate. Few of your followers “read” every tweet; only 8-12% see Facebook posts.
• Engagement could be but is not necessarily an outcome and is manifested differently by channel.
• Influence is multi-level and multi-dimensional, online and offline; it is not popularity or a single score.
• Sentiment is over-rated and over-used; its reliability varies by vendor and approach.
• Key performance indicators and balanced scorecards are helpful to connect social media impact to business results/language.

They also presented the first interim standard for content sourcing and methods used by vendors: "All social media measurement reports should include a standard "content sourcing and methodology" table that helps clients know "what's inside" the product for full transparency and easy comparison (like a food nutrition label)."

The #SMMStandards Committee is hard at work on the next steps including the publication of discussion documents on several of the priority topics. In the meanwhile, you can check for updates at #SMMStandards and start talking with your researchers and vendors about filling out a transparency table for their social media measurement sources and methods (see pdf example). Perhaps that will start moving us from black boxes and faith to data-driven decision-making.

Comments

[Craig]

Thanks for your comments Jeffrey. I have also had the bewildering experiences of trying to decipher the SMM vendors, especially when they are so divergent. Good luck with your talk next week. We need to start shifting the marketplace dynamic of SMM measurement from the producers' push to users' pull. Broad use of the transparency table for sources and methods is a step in the right direction.

[JeffreyAThom]

Thankyou, for the introduction to #SMMStandards Craig.
Could not have timed this post better. I've been scratching around looking at various commercial SMM packages and encountering the same issue, transparency.
Attempting any sort of critical analysis was not only drawing a blank but a range of very defensive posturing especially from SMM & CRM vendors.. Current offerings are a real black-box product from which I have not been able to obtain any sort of replication in results between packages never mind business activity. I understand this could, and possibly should be expected, but it is not accounted for transparently.
Transparency of measurements and methods will be central to the acceptance and engagement of SMM in any professional sense.
I will be paraphrasing your last sentence in a presentation on this very subject during the week - "Black Boxes & Faith-Based Decison Making In Social Media Measurement".
Gold.