ImpactWatch

[This is cross-posted at The Bivings Report.]

J.W.’s American Idol Twitter prediction about which contestant would end their run last night was wrong.  Brooke White did not go home despite having a disastrous restart during her song while Carly Smithson performed well this week — even garnering praise from Simon Cowell.

I think that J.W. had a novel idea, but why was his prediction wrong?

There’s an excellent chance that those who use Twitter aren’t representative of the ardent American Idol voter.  While I don’t know the demographics of the voting population, Twitter is new and geeky enough that it wouldn’t surprise me if this was the case.

Here’s an anecdote about why I know Twitter isn’t that widespread yet. Twitter inputs my tweets into Facebook and lists them in my status updates.  My friends see: “Steve Petersen is twitter: …”  Although my friends do show social media tendencies by using the site, some of them have no idea of what Twitter is.  One asked me, “What’s with all this twittering?”  While another wondered if I was constantly nervous.

Now, I’m not saying that my friends accurately represent the American Idol voter, but they probably are more like the voter than a group of people who use a geeky (I use that with pride) site.  If some of my friends understand and enjoy social media enough to use a social network but are not aware of Twitter, then tweeters are in a smaller subset of the population than my friends who use social networks. Needless to say, I have many friends who either don’t want a Facebook account or lack the desire to social media (let alone Twitter) on-line.

Further, I doubt that everyone who positively tweets about a contestant votes for that person.  Also, I know people who vote multiple times for the same person each week, and even if a Twitter user voted for the contestant whom they wrote about, how are we sure that they voted once, twice, or nine times?

My predictions were absolute rubbish, it seems.  Apparently, tweets cannot be used to accurately predict who is going home from American Idol.  In fact, two of the top three positive tweet-getters were the ones that were unceremoniously in the Bottom Two.  Syesha Mercado and Carly Smithson had the least votes after their performances on Tuesday night, despite having the most positive tweets found during my Tweet Scan.

Interestingly, Mercado and Smithson were the two contestants with the fewest positive tweets BEFORE their performances of this week.  This may indicate that overall twittering can predict the voting habits of America better than spur of the moment written notions.

Will we ever find a way to predict American Idol?  I researched some of the other methods previously mentioned in this post, and none of them predicted a Carly elimination.  With that, Fox Television continues to remain a mystery.

For the second part of my study concerning whether tweets from Twitter could be used to predict the losing contestants of American Idol, I decided to wait until noon to make sure that there were enough new tweets to equal the amount used in the analysis last week.  Fortunately, there were more than enough fresh, unique tweets discussing last night’s episode.

As I did last time, I used Tweet Scan to search for tweets about each of the remaining contestants on American Idol, stopping at 90 each.  Then, I ranked each of the tweets as positive, negative, or neutral, which is similar to the ImpactWatch system of rating articles.

Syesha Mercado had the most positive new tweets (78), despite the fact that she had the fewest just a day ago.  David Cook continued to have mostly positive tweets, while Carly Smithson nearly doubled her amount.  Contestants Jason Castro and Brooke White dipped in positive tweets, while their negative tweets rose dramatically.  David Archuleta, who was the front-runner at the beginning of the season, continued to slide in positive tweets and gained more negative ones.  An Excel spreadsheet of all the specific data is located at the end of the post, while a graphical analysis is below.

While it could be useful simply to take the most recent tweets into account, I decided that to make an accurate prediction, I should combine the previous night’s tweets with the more current ones.  It seems reasonable that voter opinions would last for about a week with the most current performance taking precedence in minds.

When looking at the combined data (graph below), Cook seems to be the frontrunner, with Mercado, Castro, Archuleta, and Smithson all behind him.  White is the only contestant to have more negative tweets than positive tweets overall.  If what I hypothesized is true, it seems from the data that the dreaded Bottom Three will contain White, Castro, and Archuleta, with White sealing her eliminated fate.

As I have said previously, I do not believe that online activity can predict offline behavior, but since many of the tweets specifically reference contestants for whom the writer had voted or was planning to vote, I believe that tweets may represent a fairly accurate portrayal of the voting population.  We will see if I am correct tonight.

Below is an Excel spreadsheet of all the data collected during the course of the study.  Also below is a report, created using ImpactWatch, showing all of the graphs in a high quality format.

AI Excel Spreadsheet All Data

AI Graphs PDF

Once again, the Internet is abuzz with predictions and theories about who is going to win American Idol.  In the early days of the competition (back when Kelly Clarkson was still a nobody singing karaoke and we only hypothesized that we hated the British without actually knowing it through Simon Cowell), there was much less web traffic about the show.  This season and the previous one, however, it’s all the Internet can talk about.

This leads people in finding numerous ways to predict who of the now-6 remaining contestants will be voted off each week.  After all, this is a show that purports that the American public gets to decide who is going to stay and who is going to go.  Polls, blogs, and fansites may all play an important role in deciding the overall victor, much like a modern day political campaign. 

TV Squad, a popular television site, uses polls from various sources as well as their own intuition to predict the next bootee.  Most of the polls incorrectly predicted Syesha Mercado’s demise, while the real loser was Kristy Lee Cook.  Obviously, this is not an accurate way to predict the contestant with the lowest votes.  The polls are simply too specific in the sense that only those Internet snoopers that come across them will actually get a chance to vote in the poll.  This does not represent an accurate view of the American public.

DialIdol.com has found a more inventive way to predict the successful contestants.  Their software measures the busy signal of each phone line to determine who is getting the most votes.  They started the program during the previous season, but achieved only moderate success in the predictions.  The company also sells software to enable one person to vote many times for a contestant.  Many sites have reported that the software is now known by the American Idol producers and rarely works anymore.

Tivo also found a creative way to measure the votes.  The company claims that they can predict who is going to be voted off by which minutes of the recorded programs are re-watched.  The theory is that Idol favorites will have their performances re-watched by their adoring public, while soon-to-be eliminees will have fewer views.  Unfortunately, the system seems to not be altogether accurate, since Tivo has incorrectly predicted Mercado two weeks in a row.

Another social media company, BuzzLogic, uses their “influencer blog” ratings to follow the entire competition via their blog.  I was impressed by the fledgling company’s efforts at first glance, but upon closer inspection realized that few, if any, of their predictions have been true.  In addition, BuzzLogic gives very little explanation when they are incorrect.  This does, however, bolster my recent opinion that Katie Paine’s connection between online activity and offline activity is flawed.  Many ‘influential’ bloggers may be writing about certain candidates for American Idol, but that does not necessarily mean that they are voting for them, or voting at all.

I decided to tackle the task of predicting American Idol, ImpactWatch style.  Instead of using news articles, I used Tweet Scan to analyze 90 tweets per remaining contestant, using two separate searches for each.  I searched for each contestant’s full name as well as their first name and the phrase “American Idol”.  I read and ranked each tweet post as positive, negative, or neutral.

There are two reasons why I believe this method to be more valid than the other ways that were described above.  First, tweets represent impulses and first impressions, which I assume mirrors the mindset of actual voters.  Secondly, this is the only method that ascribes a positive or negative take on the information.  Polls just rank the favorite, while the Tivo system lacks any real information about why certain parts of the show are re-watched.  BuzzLogic’s system has merit, but suffers from the need of personal input by its bloggers to explain anomalies in the amounts of influencer blogs.

Using my ImpactWatch inspired protocol, I found that David Cook and Jason Castro have the highest amount of positive tweets.  Sure enough, after doing some extended research, I found that the two received much praise for their performances last week.  All three females had an identical number of negative tweets (45), but Mercado has the lowest amount of positive tweets at a scant 30.  This is preliminary, but on Wednesday morning, I will post an updated tweet analysis (since Tuesday is when the contestants will perform their new songs).  Voters will most likely be tweeting away while they are waiting to vote.  Let’s see if I can accurately predict which Idol will fall.

My current results are summarized below, using a graph created using ImpactWatch.

This is the final part of my four-part analysis of Facebook applications.  (For the preceding part, click here.)  In this section, I will attempt to make some conclusions and predictions from all of the data that I collected.  For a complete list of every single one of the Facebook pages that I analyzed, check at the bottom of the page for an Excel spreadsheet link.  The names of the users have been deleted, but originally I used them to avoid accidental repetition during my research.

One of the most notable aspects when you take a look at the graphs (a PDF of all the graphs from the previous posts is included at the bottom of the post) is that not a single user had recently deleted an application.  After looking at many users, I decided to check a few extended histories, but alas, I still found no deletions.  Personally, I have deleted applications in the past, so I am aware that it happens.  My theory is that users have begun to recognize when they want to add an application or not, and as such, are becoming more ‘picky’ when they are presented with a new one.  This would explain why there are still several additions present within the data.  With so many applications now available, newer ones have to be worthwhile in order to garner interest from users.  This is still possible, as Bumper Sticker proves, being a fairly recent application itself and already in the Top Ten.

Speaking of the Top Ten, my inner predictions were accurate.  According to Adonomics.com, approximately 5%-10% of users have each of the individual applications installed, so if I am ranking ten of them, my statistics professor from college would be thrilled to know that I realized about half of the total users would have at least one of them.

It is also interesting to note that users that only have 1 to 2 applications typically had one of the Top Ten as that lone application.  This makes perfect sense, since many of these are Hug Applications.  Any user wanting to receive these pokes and hugs from other users must have the application installed; so many users probably have it simply to receive and not to give.  It’s total Christmas Stocking Syndrome.

I was pleased to find that a clear majority of users (of those who actually had applications) have 5 or less applications in their profiles.  When I began this research study, I had a gut feeling that I would find more 9+ entries than any other kind of profile.  Perhaps it is that those profiles simply stand out more.  In my personal opinion, given that some of the user-created applications are fun, and dare I say, ‘useful,’ it is perfectly reasonable to have five or fewer.

I was also not surprised to find that the majority of typical usage was for Extended Use.  Some of the notable Extended Use applications–other than the ones already explicitly mentioned in the study–were ones that allowed users to post bigger pictures and give extra information about themselves.  It’s somewhat of an old Internet cliché: people do not want to be limited in anything that they are doing, no matter what it is.  I was a tad surprised that Online Games were the least used category, but then again, users of Facebook can find free online games in other avenues.  Why use Facebook when there are better games out there?

As I was researching prior to the study, I saw many web postings comparing Facebook to its main rival, MySpace.  One of the main advantages to Facebook, according to those writings, was that it was not cluttered like MySpace profiles.  I find it ironic that people add applications when this is the popular opinion.  Many of the applications take up much space on a profile, adding a cluttered feeling to the overall page.  Forget Christmas Stocking Syndrome, Facebook users suffer from wanting to have their cake and eat it too.

Excel Spreadsheet with All Data Collected

All Pie Charts PDF

[This post is cross-posted at The Bivings Report] 

Continuing the study (see the preceding part of the analysis here), I analyzed if there had been recent activity by users regarding the addition of new applications.  Facebook applications can be added or deleted from profiles at any time, and there is a specific tab on the left-hand side of user profiles designated to the addition or removal of applications.

I used the mini-feeds (which show recent user input) to analyze if there had been recent application-related activity.  35 users had made recent additions, while not a single user had recently deleted an application.  An overwhelming majority of users had done neither in the last week.  Below is a graph showing this data, made using ImpactWatch features.

The final area of study concerned the ‘Top Ten’ applications as elected by Adonomics.com.  These are Super Wall, Top Friends, Hug Me, Super Poke, Bumper Sticker, iLike, Graffiti, Zombies, Scrabulous, and Quizzes.  These were the top ten applications at the time of the research.  With the addition and removal of applications, the top ten applications could change periodically.  More information on these applications can be found in the background post about the study.

Of the 300 users, 146 had at least one of the Top Ten applications, while 154 did not.  Below is the graphical representation of this data, made using ImpactWatch features.

The final part of the study will be posted soon.  It will include an Excel spreadsheet of all the data, as well as some conclusions drawn from the data.

The first aspect that I wanted to analyze was the sheer amount of applications contained within profiles.  To make the count simpler, I narrowed the selections down to five categories: 0, 1-2, 3-5, 6-8, and 9+.  Only 3^rd-party (i.e. not official) applications were counted.  Of the 300 profiles researched, 64 contained 0 applications, 84 contained 1 or 2 applications, and 84 contained 3, 4, or 5 applications.  On the higher side, 41 profiles contained 6, 7, or 8 applications, and 27 contained 9 or more.  Below is a pie chart with a summary of the collected information.

The application interfaces appear within the profile itself, based on where the user would like them to appear, and also as a small icon directly under the profile picture.  Below is an example of what could be found under profile picture, if the user had many many many applications.

The second usage category that I analyzed concerned the type of applications that each user employed.  Of the applications used, which type was most prevalent in their profile?  I divided the types of applications into four categories, based on my own observations.  The categories under which the applications could fall were Extended Use, Online Games, Hug Applications, and Outside Applications.  Applications under Extended Use expand the usability of the standard applications, like the Super Wall.  Online Games, such as Scrabulous, give users the ability to play turn-based games with other users.  Hug Applications grant the user the ability to send more personal messages and ‘gifts’ to other users.  Finally, Outside Applications promote an outside media, such as a charity, television show, or computer program like Skype. 

124 users had applications for Extended Use, a clear majority over the other categories separately.  21 users had applications mostly for Online Games, with these mostly being very popular ones like Scrabulous and Jet Man.  59 users mostly have Hug Applications; 32 mostly have Outside Applications; and 64 have no applications whatsoever.  It’s worth noting that eight of the ‘Top Ten’ applications fall under the category of either Extended Use or Hug Applications.  Not a single one of the Top Ten is an Outside Application.  Below is a graphical interpretation of this data.

Part 3 of the study to come shortly.  Check back soon.

[This post is cross-posted at The Bivings Report] 

The overly popular Facebook social network has recently seen a surge of ‘applications’ added to its roster.  Users hoping to enhance the experience of the social platform create these applications.  As of January 2008, there are over 14,000 applications in circulation among users.  The uses of these applications range widely; in July 2007, the first Facebook-only venture capital firm (Altura 1 Facebook Investment Fund) was released to the public.  They have gotten so popular that Stanford University recently debuted a class where the end product is Facebook application.  The great success of this class most likely means that many more schools will soon follow suit, offering more classes on social network metrics and creation.

These applications are not without their criticisms.  Many users voiced opposition when they were first established because they felt that the applications would clutter the very streamlined Facebook profile design, making it more akin to a MySpace style page.  Others complain of the sheer number of available applications.  Many of these are deemed to be pointless or copies of pre-existing applications.  Another common complaint is the way in which they have to be added.  For example, to use the application titled Bumper Sticker, you have to install the application within your profile to be able to view a ‘sticker’ that is sent to you by another user.  This process ends with a user having applications that are rarely used by them, but cluttering up what is otherwise known as a clean interface.

I decided to do some research using my networked profiles to discover in what ways and in what capacity people were using these controversial applications.  My study used 300 of my ‘friends’ profiles.  I simply took the list of all of their profiles and analyzed every other one until the number totaled 300.  This helped eliminate any bias that I may have shown towards certain types of people.

I outlined the criteria for the study before viewing a single profile in an attempt to make the study as ‘blind’ as possible.  I divided the criteria into four questions, which I ranked.

The first question asked how many Facebook applications the user had in his or her profile.  The selections were 0, 1-2, 3-5, 6-8, or 9+.  This category counted only applications created by users, not Facebook applications that come with the standard profile setup.  Specifically, the following applications were NOT included: Newsfeed, Wall, Photos, Gifts, Marketplace, Pokes, Status, Events, and Video.

The second question was which type of functionality was most prevalent in their applications.  The categories under which the applications could fall were Extended Use, Online Games, Hug Applications, and Outside Applications.  Applications under Extended Use expand the usability of the standard applications, like the Super Wall.  Online Games, such as Scrabulous, give users the ability to play turn-based games with other users.  Hug Applications grant the user the ability to send more personal messages and ‘gifts’ to other users.  Finally, Outside Applications promote an outside media, such as a charity, television show, or computer program like Skype.

The third part of the analysis is whether or not there has been a recent addition or deletion of an application.

The last question is whether or not the user has one of the ‘Top Ten’ applications as sourced from Adonomics.com, a site that does some Facebook-related analysis.  The Top Ten applications are described in the following paragraph: 

Super Wall (or alternately, Fun Wall or Advanced Wall) lets the user have more wall functions.  Top Friends allows the user to select certain friends to always appear in their profile.  Hug Me adds more creative hug features.  Super Poke adds more poking verbs (i.e. John has just bodyslammed Chris).  Bumper Sticker lets the user gift witty graphics to other users.  iLike lets the user list movies, songs, etc. that they enjoy with graphics representing their selections.  Graffiti lets the user make drawings for other users using a Windows Paint-like tool.  Zombies adds more Zombie-specific hug features.  Scrabulous is a turn-based Scrabble clone for 2-4 players.  It is played over an extended period of time.  Quizzes allows the user to take multiple quizzes on various topics for fun.

The next post will contain all of my collected data from the analysis, as well as analytical graphs displaying my findings, creating using ImpactWatch.  I will then draw conclusions based on my research about the state of applications within the Facebook online world.

[This post is cross-posted at The Bivings Report] 

To follow up on a recent post concerning Comcast’s effort to answer consumer complaints via Twitter, I used Tweet Scan to search specifically for Comcast posts and research exactly with what we are dealing. A basic one-word search found well over 1000 tweets about Comcast within just the last couple of hours, so I narrowed my focus down to the most recent 300. I read each of them, and categorized them in three different ways.

The first specification was whether the tweet was positive, negative, or neutral, overall. The results are as follows: 26 of the tweets were positive, 86 were neutral, and a majority of 188 were negative. It is a pretty negative environment for Comcast on Twitter right now.

The second category dealt with what category of complaint or praise under which the tweet fell. There were four distinctions: Not Working, Slow, Prices, and Company. “Not Working” and “Slow” deal with complaints about the Internet and cable service. “Prices” concern any complaints or praise about cost or billing issues. “Company” refers to any mention of the company that does not fall into one of those categories, or short tweets with little information (i.e. “grrr…Comcast”). 178 were about the company itself, 66 were problems with the Internet or cable completely not working, 33 were about slowdown, and 22 were about pricing concerns. It is interesting that on Twitter there is a lot of general venting about Comcast (bad for the brand), and less specific complaints.

The final category is whether or not the tweet contains cursing of any sort. From a quick skim of the 300 tweets, it seems like this is a good indicator of the level of frustration by the writer of the tweet. 35 contained curse words, and 265 did not.

Found below are some examples of Comcast-related tweets, as well as a document containing all the graphs above. This post is similar to the kind of analysis we perform through out service ImpactWatch. Interesting to note is that several of the tweets among the 300 were by the same user, who claims to be a representative from Comcast. Also, many of the tweets contained links to articles referencing the recent customer service use of Twitter by Comcast. Unfortunately, the representative could only handle one consumer problem at a time, so the use of tweets was just as effective as phone consumer services. The links below represent the tweet-by-tweet written data, some examples of Comcast-related tweets, and analytics.

All Data Collected

Example Tweets

Graphs Made in IW

<Cross Posted from The Bivings Report>

Over the weekend, two of the users I follow on Twitter, David All and Techcrunch (Michael Arrington), had separate problems with Comcast and vented about them via their Twitter accounts. Comcast apparently monitors Twitter and proactively reached out to both of them.

Here is the relevant tweet from Techrunch:

And here is the tweet from David:

An article in the Consumerist confirms that other users have received responses after complaining via Twitter. In a follow up article about his problems, Michael Arrington offers advice to folks with a Comcast service problem: “Skip the hold time on their customer service line and go on the attack at Twitter instead. You may find your problem fixed in a hurry.”

Three thoughts on this:

(1) I think it is great that Comcast is listening to people on Twitter and reacting proactively to fix problems. Based on a quick search, there appear to be plenty of problems to that need addressing. More companies should monitor and participate in Twitter in a meaningful way (we are working on doing Twitter tracking through our ImpactWatch service). It should be incorporated into the customer service loop.

(2) As a consumer, I’m bothered by the precedent of the squeaky wheels on Twitter getting preferential treatment over people who go through normal channels.

(3) Not speaking specifically about Comcast, I think the focus some companies place on social media is more about PR/crisis management than a true commitment to customer service and dialogue. Performing triage on complaints that come in through Twitter may keep the customer revolt at bay for a short time, but when that levee eventually breaks, it isn’t going to be pretty.