Archive for the ‘Measurement’ Category

Olympics/China Demo: The Final Countdown

Friday, September 12th, 2008
Posted by: esmith


With the Beijing games’ conclusion in August, the Olympics Demo we’ve been running has a lot to show for itself. Over the course of the summer, we picked up thousands of news stories ranging from athletic scandals to the cuisine of China, medals sweeps to Tibet supporters.

Overall, as noted in our previous posts, the Olympic games held a much higher favorability rating than that of host China. Nearly all articles reporting on sporting results were much more correlated with the Olympic Games or other countries, while China’s stories consisted largely of pollution, Tibet, and human rights.

As the games began, the “Phelps Fever” swept the press. With the flexibility of the ImpactWatch platform, I was able to add attributes to tag all articles related to Michael Phelps. Taking data from August, here is a graph I generated showing the topics breakdown:

graph.png

It is incredible to see that out of the articles published about athletes, Phelps-related stories comprised of nearly 1/5 of the total news about athletes. I had expected it to be slightly higher, but this is still an incredible percentage.

Our demo worked out just as we had intended. Aggregating articles about the Olympics and arranging them in intelligent and meaningful ways yielded interesting results, as well as provided an outlet to showcase the features of the ImpactWatch platform. As this demo was my first IW experience, I am convinced that the platform is a valuable tool for monitoring media (our clients agree!).

We’re throwing some ideas around within the IW team for a smaller-scale demo in the future, and so far we’ve come up with UFOs, assorted B-list celebrities… have a suggestion? Drop it in the comments, we’d love to hear it.

Olympics Demo: Topics and Issues

Thursday, August 14th, 2008
Posted by: esmith

As the Olympic games have kicked off over in Beijing, I thought it was time to update the topics and issues portion of the demo to reflect current events. We’ve removed some of the topics and issues that are no longer in the news (such as the torch run) and added some others that have come to light since the 8th. So far the new additions are medal race, journalistic freedom, and security. Have any ideas for other issues or topics we should be tracking for the report on the games’ duration? Let us know in the comments.

Olympics Countdown

Tuesday, August 5th, 2008
Posted by: esmith

With only three days left before things kick off in Beijing, our ImpactWatch demo has made itself clear: within the media, China is being portrayed at a far less favorable rate than that of the actual Olympic Games. Even with widespread buzz addressing athletic performance enhancers, coverage of China’s human rights record, air pollution, algae blooms, and authoritarian domestic policies have been pushed to the forefront of western media. While glittering articles about athletes’ preparations boost the favorability of the Olympics, China is struggling to find its own crutch topic under the strain of relentless criticism and unfavorable press.

The graph below illustrates what I’m talking about:

 Even from this raw graph of daily average favorability, the difference is quite noticeable. When I visualize by month to show overall trends, things get interesting:

Monthly Average Favorability for China, Olympics from 05/06/2008 - 08/04/2008 (Trendline)

These trend lines highlight the discrepancy between the two topics. While the Olympic Games’ average favorability has had its ups and downs, its average favorability has both remained positive and increased in positivity from May to August.

China has not fared as well in its coverage; while enjoying two relatively neutral months (May and June), its favorability rating began to falter in July (-.13) and August (-.29). Quite the opposite of the Games’ coverage, China’s trend line indicates both a negative rating and an increase in negativity from May to August.

As the Games’ opening ceremony nears, a new chapter in our ImpactWatch demo begins. With Beijing as a stage, the world — and our demo — is all eyes…anything could happen.

ImpactWatch: Why humans are better raters

Thursday, July 17th, 2008
Posted by: hades

I have spent this summer working for the Bivings Group as an intern. The main focus of my internship has been working on ImpactWatch. One feature that strikes me as particularly interesting is the use of human editors to rate the sentiment of articles.

Many media measurement companies now use automated systems to rate the sentiment of news articles. While this system may be able to read and rate the articles faster than a human, we feel that that the complete automation of the system causes the information to lose value. The Bivings Group will continue to monitor the automated sentiment analysis technology and will switch to it if and when it is warranted.

We have found that while great strides have been made in the field of sentiment analysis, things like sarcasm and colloquial speech can often throw the system off. Another weakness of automated sentiment analysis programs is that they tend to mishandle trends. If a company has made a mistake, but is being praised for their efforts to correct that mistake, automated systems will often rate that as negative, when it should be positive. Sentiment analysis software also seems to run into problems when it comes to rating the sentiment of different entities in an article.

It is for all these reasons that the Bivings Group has chosen to stay with human editors for the sentiment rating in ImpactWatch. We are confident at this point that the extra time it may take to rank the articles is worth it because we know that we are giving our clients the most accurate data possible.

Ch-Ch-Changes: Turn and Embrace the Derivative

Friday, June 27th, 2008
Posted by: esmith

It seems that everyday I see a new catchphrase coined by a social media “expert” to describe social media, hoping the concept will set the blogosphere ablaze with link endorsements and fleeting exposure on Techmeme. Amid the clutter, converting metrics such as blog views, Facebook buzz, Tweets, and del.icio.us plugs to a usable, actual value of return-on-investment (ROI) in dollars seems nearly impossible — currently the holy grail of strategic internet marketing.

I’ve noticed something as analysts try to pinpoint an exact metric to measure social media and all this talk of ROI as both return on investment and return on influence. The derivative (change over time) has taken a backburner to comparitive measurement. I’m a proponent of using this measurement, and I think it’s currently underrated and arguably more, or at least equally important, as comparitive analysis in social media.

Here’s my short list of ways to use the power of the derivative in social media campaigns (a friendly reminder, in case you’ve been distracted):

  1. Focus on change over time for your brand. Sure, it’s nice to know how your competitors are doing with their online presences and how you stack up comparitively. However, unlike traditional print media, there are both logical and logistical fallacies that don’t always make this an easy task. As Maggie Fox from the Social Media Group points out, good metric aggregation for social media is a time-consuming and manual process (read: expensive). As my boss recently pointed out in an e-mail, in terms of actionable PR work in social media it is much more important to know where you stand and how to improve than where your competitors are: in social media, there are no press release standards, and there is no official staging ground for retorts. I don’t want to sound like a self-help book, but focus on you.
  2. Keep in mind, “buzz”, by nature, is merely a rate. Internet buzz itself is a phrase coined to describe lots of conversations in a small amount of time. By identifying time spans that had the highest rate of change, you can reflect upon successes (or disasters) and find what works best for your specific brand. Comparing the rate of change for different campaigns can give you a metric as to what worked better — for your brand. Not all methods work similarly.
  3. The double derivative can affect timing. Think real hard back to high school physics… the double derivative represents acceleration. When applied to aggregated data about social media, it is easy to tell whether the cycle of buzz is coming or going. Keeping up the momentum on a social media campaign is difficult, and one of the most common downfalls of an otherwise great campaign can be timing. The double derivative (acceleration) can help with increasing, or at least maintaining, positive headway for your social media campaign. Knowing when to play your cards is key to success. The gadget/computer manufacturers have their “product-leaks” to blogs down to a science.
  4. The derivative of self boils down to constant self-comparison. It’s a tangent from point #1, but I cannot stress how much I believe that in social media, self-contained metrics are vital to a strong social media strategy. Allocate resources to where it really matters, and I think pushing said resources into creating change for your brand is the top priority. Self-comparison through time gives real results with actionable intelligence.

OlympicsWatch Update

Friday, June 13th, 2008
Posted by: esmith

Only sixteen days after my last post about our ImpactWatch demo, the media has begun drifting away from Sichuan’s earthquake in light of high-tech espionage, Paul Hamm’s broken hand, and a record-breaking year for track and field. With the Olympic trials season in full swing, positive coverage on athletes and the games itself is steadily rising. I created an issues breakdown visualization using ImpactWatch, which reveals that just as quickly as the media surrounding the earthquake sprang up, most of it seems to be over:

Issues Breakdown

While the stories surrounding the earthquake have subsided quicker than I had anticipated, it certainly set the stage for interesting coverage of the 2008 Olympic season. Stay alert, more updates to come.

YouTube: An Analytical Approach (Part 2)

Monday, June 9th, 2008
Posted by: esmith

In the last post, I did a micro-study of Kinoki Detox Foot Pads and the way they are being portrayed in the realm of YouTube. In addition to analyzing the “views” count on the website, I adjusted the quantity by minutes to give a better representation of the exposure YouTube is actually giving the Kinoki brand.

This time around, I will examine the user feedback to the videos that were part of the study. If I had been the one who had actually posted these videos, I would be able to utilize YouTube Insight, a powerful built-in tool that turns an ordinary YouTube video into something that resembles an online focus group (there will be more on this tool in the future). Seeing as how I am not interested in posting my own videos about Kinoki Detox Foot Pads, I will have to rely on the inexact method of comment analysis and “feel it out”.

First, a red flag. On the positive videos, there were only a couple of comments that were all positive and framed as customer testimonials reiterating their “refreshing feelings” and how using them had changed the users’ lifestyles for the better. After three or four comments, the commenting feature had been disabled, thereby locking in a few positive comments and keeping naysayers from commenting on the wall. That, and the videos themselves were infomercials in the guise of “health update” news reports. Rather suspect if you ask me.
On the videos that allowed commenting, the tone was overwhelmingly negative. A couple of comments would be users who have actually used the product and would claim that the Kinoki Detox Foot Pads actually made them feel better, to which users would instantly become outraged and prove them wrong with their elite high school chemistry knowledge.

In review, the coverage of Kinoki Detox Foot Pads on YouTube is quite negative, both in videos and commentary. If an undecided consumer were to consult YouTube to make a more informed decision, they would more than likely decide not to purchase these foot pads — maybe changing your diet is the way to go after all. I still swear by my twice-daily regiments of snake oil.

YouTube: An Analytical Approach (Part 1)

Wednesday, June 4th, 2008
Posted by: esmith

According to Alexa’s listing of most visited websites (sorted by country), YouTube is ranked as #4 in the United States, #6 in the United Kingdom, #4 in Japan, #5 in South Korea and #2 in Germany. It has the power to launch overnight public-relations Cinderella stories such as the Blendtec miracle (yes already, it will blend) as well as broadcast to millions a reputation-killing moment even more swiftly. These occurrences, once posted to YouTube, are available indefinitely.

How does one quantify the amount of successful or damaging exposure YouTube is causing them? Here are some measurements of a micro-case study crafted specifically for this blog post: an analysis of YouTube exposure of the television infomercial product, Kinoki™ Detox Foot Pads. Note: The Bivings Group and the ImpactWatch service are in no way affiliated with Kinoki™ Detox Food Pads.

I manually aggregated data by watching YouTube videos related to the “miraculous detox system”. You can see the raw data that I collected here in an excel spreadsheet. Using the resulting data, I created some visualizations within ImpactWatch:

 

 

Kinoki Foot Pads YouTube Graphs

 

The first graph shows a raw view of the types of videos people watch related to the Kinoki™ Detox Foot Pads. This is strictly measured in views, which is valid because YouTube only counts views once from each unique IP address, and only if an overwhelmingly large portion of the video was viewed. However, this unit of measurement is often misleading and does not give an accurate representation of the actual exposure Kinoki foot pads have received. That’s where a little spreadsheet manipulation and the second graph comes into play.

In the second graph, a new unit was analyzed. Taking the number of views for each video and multiplying them by their length in minutes, an adjusted unit that more accurately represents total “exposure” was created. This graph provides a much better metric for gauging the float-or-sink status of the product. Although in both graphs negative exposure was dominating (I wonder why?), the adjusted quantity of “YouTube minutes” shows that it wasn’t as bad as just a raw views count might have initially demonstrated.

In the next part of this two-part case study, comments for the videos and other forms of responses will be analyzed.

A Tale of Two Game Consoles

Monday, January 29th, 2007
Posted by: Chuck Fitzpatrick

Back in November, a number of us here at The Bivings Group were captivated by the Nintendo Wii and Sony PlayStation 3 launch – so much so, that we set up our ImpactWatch demo to follow the news about the two game systems. Now that both the launch and the Holiday Season have past, we decided to put together a report based on the ImpactWatch analysis and our own research. The following is a brief summary of our findings.

Sony PlayStation 3

Prior to launch, it sounded like Sony’s PlayStation 3 had captivated the imaginations of the gaming world. While the Nintendo Wii was quietly building buzz, the media was all over the PlayStation 3 like a pack of wolves on a fresh kill. The hype surrounding the PlayStation 3 launch was staggering, and going into the Game Console Wars, a betting man probably would have put his money on Sony. With the success of the original PlayStation and the even greater success of the PlayStation 2, it seemed like it would be a slam-dunk for Sony. That turned out not to be the case. Three things that stood out about the media coverage of the PlayStation 3 were:

  • PlayStation 3 Launch Sees Rampant Crime. Buying a video game system was never so dangerous. Due to massive hype-driven demand, combined with severe supply constraints, the launch of the PlayStation 3 was tainted by theft and violent crime that spanned the globe.
  • Sony PR and Management Blunders Abound. From Sony’s decision to include the Blu-Ray high-definition DVD drive, thus delaying the launch by almost a year, to Sony Australia’s General Manager, Nic Foster saying “Wii is a core gaming device. It’s a more fun, intuitive sort of product to pick up,” to Sony’s press release falsely claiming that they had won a Technology and Entertainment Emmy for the SIXAXIS controller, they managed to fumble the PR ball on every play.
  • Developers Jump Ship, Express Angst. Reports show Sony failed to cater sufficiently to non-Sony game developers. They lost the platform exclusivity of the best selling game of all time (Grand Theft Auto) and alienated their third party developers to such an extent that the launch line-up consisted of only a handful of titles. Gabe Newell of Valve Software said that, “The PS3 is a total disaster on so many levels, I think it’s really clear that Sony lost track of what customers and what developers wanted.”

Nintendo Wii

game2.gif

The Nintendo Wii can be described as the underdog going into the Game Console Wars. Initially, both Sony and Microsoft felt that it was not a direct competitor. Their machines targeted hard-core gamers, who are interested in the most realistic gaming experience, the best sound, and greatest speed. The Wii was considered more of toy targeted towards younger gamers. It turned out that the Wii was a more direct competitor than anticipated.

Three thing that helped to shape the brand perceptions consumers have about the Wii were:

  • Media Stardom. The Nintendo Wii rapidly ascended to become a pop-cultural icon. From the beginning, the Wii was a media darling, earning free coverage in editorials, radio station contests, and television shows - including being prominently featured in episodes of South Park and The Colbert Report, as well as being given away to every audience member on an episode of Dr. Phil.
  • Strapgate. In a public relations coup d’etat, Nintendo managed to turn a potentially damaging quality issue into a resounding brand victory. Within a week of the Wii’s launch, reports began to surface about Wiimotes slipping out of gamers’ hands and the security straps breaking. Nintendo responded promptly and decisively, announcing an investigation and voluntary recall of the straps. What could have turned into a significant issue managed to help solidify the Nintendo brand in the eyes of consumers.
  • Broad Demographic Appeal. Nintendo’s NES gaming console was an 80’s icon and many people that were kids playing their NES console in the 80’s are now in their 30’s and want to relive the fun of their youth. In a piece on 1up.com, Nintendo’s Senior VP of Marketing, George Harrison, cites a 2006 survey by the Entertainment Software Association revealed that 25 percent of all gamers are 50 or older. Of the three next-generation game consoles, only the Wii has been positioned to really target such emerging market segments.

If you are interested in reading the full report, click here to download it as a PDF.

If you would like access to the ImpactWatch demo set up to track this issue, click here to request a username and password.

Featured Article in Communique Magazine

Tuesday, January 9th, 2007
Posted by: Chuck Fitzpatrick

In the most recent volume of Communique Magazine , TBG’s Gary Bivings authored a featured article. “Watching the Web” is a great piece about how PR and marketing techniques are changing to incorporate the monitoring of various new forms of media, primarily the Web. The article cites several examples of how the Web has affected PR and customer involvement in the pharmaceutical industry, and discusses how content tracking platforms can be an effective addition to any PR firms strategy (this includes our very own ImpactWatch !). Among other issues, Gary listed the Top Seven Online Media Monitoring Practices in his article:

  1. Get involved sooner rather than later
  2. Implement a platform that will track online media for you
  3. Categorize online activities of target markets/information seekers–blogging, message board posting, etc.
  4. Make a list of the top sites to monitor per public/target audience
  5. Track site statistics (eg, how often the site is visited)
  6. Track bias of various sites with regard to your product
  7. Decide how you will respond to the circulation of negative information about your product

You can read the article in its entirety here.