1. A tweet about StreamGraphs from Tim O'Reilly on August 5th to his 8,680 followers. Tim is the founder and CEO of O'Reilly Media - a publisher of excellent technical books.
2. An article written by Scott Gilbertson entitled StreamGraph Maps Twitter Word by Word for Webmonkey. A link to this showed up on the front page of Wired and drove some significant traffic.
3. A tweet about using StreamGraphs to follow references to the Olympics from Evan Williams. Ev is the Co-founder and CPO of Twitter and currently has 17,535 followers on the service.
4. An article by Tim Showers called Visualization Strategies: Text & Documents that highlighted several of my little tools. His article attracted some attention on Reddit.

Thanks very much to these people and the many others who have let people know about Neoformix !

 

In Zach's words:

Each page displays trends in the top 25 movies at the box office for each weekend in a year. The color is based on the movie's debut week. Because of that, long-running movies will gradually start to stand out from newer movies with different colors.

Related posts: Movies Ebb and Flow

 

 

The application shows a StreamGraph for the latest 200 tweets which contain the search word. The default search word is 'interesting' but a new one can be typed into the text box at the top of the application. You can also enter a Twitter ID preceded by the '@' symbol to see the latest tweets from that user. A parameter to the URL can be used to specify the initial search word. For example, use http://www.neoformix.com/Projects/TwitterStreamGraphs/view.php?q=coffee to see the latest tweets about coffee. This makes it possible to link to a StreamGraph for your own tweets from your blog or within a twitter update.

The StreamGraph shows the usage over time for the words most highly associated with the search word. One of these series together with a time period are in a selected state and coloured red. The tweets that contain this word in the given time period are shown below the graph. You can click on another word series or time period to see different matches. In the match list you click on any word to create a different graph with tweets containing that word. You can also click on the user or comment icons and any URL to see the appropriate content in another window. If you see a large spike in one time period that hides the detail in all the other periods it will be useful to click in the area to the left of the y-axis in order to change the vertical scale.

Credits go to Lee Byron for the visual ideas behind the StreamGraph (although I'm using a simpler symmetrical form), to Processing for the development tools, to Martin Porter for the Porter Stemming Algorithm, to Vaga for the two small icons, and to Summize for building a great API into the Twitter data.

Related posts are TwitArcs and Twitter Spectrum.

Happy Canada Day ! I've created a simple flag graphic using a few words that come to my mind when I think of Canada.

I've done this analysis and presented the results below. For each of the top 6 authors and top 60 names the number of times each author used each name was determined. The first graph shows the breakdown for the top 10 names. The second has the same form and shows numbers 11-60 but I've broken it into a separate graph because it uses a different scale. This lets us see more details for these names. I have also colored the bars to show proportional use of the names. A deep blue color means that the name was used proportionally much more often for that author and a deep red shows that it was used proportionally much less often. Paler colors indicate a lesser degree of high(blue) or low(red) usage.

Click on the images to see larger views
 

Some things that I spotted quickly from the larger version of the top 10 graphic include:

1. There aren't too many names that were discussed a lot more by a particular author - no deep blues
2. Perhaps the deepest blue in the graphic shows that Hendrickson discussed FaceBook proportionally more than the others
3. Unusually low (deeper red) combinations are Schonfeld-Digg, Kirkpatrick-FaceBook, and to a lesser degree Gonzalez-Microsoft and Hendrickson-Microsoft

1. Arrington discusses Life (as in Second Life, Yahoo Life, Online Life, various others), VOIP, Adsense, Silverlight, and P2P proportionally less than average
2. High proportional names for Riley include Twitter, Life, Windows, and Silverlight
3. Low proportional names for Riley include RSS, Flash, Zoho, AJAX, Salesforce, NetVibes, and Wikia
4. High proportional names for Schonfeld include Comscore, Bebo, Salesforce, and especially OpenSocial
5. Low proportional names for Schonfeld include RSS, Life, API, URL, Ning, photobucket, and a few more
6. Other high proportional pairings are Kirkpatrick-RSS and IM, Gonzalez-VOIP and Zoho, Hendrickson-Bebo,Ning,Hulu and OpenSocial

The data for this analysis was kindly provided by Yuvi from The StatBot.

Click on the image for a larger view

The graph clearly shows the companies that have dominated TechCrunch coverage over the last 2 years. Google looks biggest with FaceBook, Yahoo, and Microsoft being quite large as well. You can spot the increase in coverage for Microsoft and Yahoo in Feb of this year due to the merger talks. Notice also that MySpace and FaceBook were fairly even until July 2007 when FaceBook began dominating. If you look closely you can also tell that Twitter has become important lately with the number of references in April 2008 very similar to both Microsoft and FaceBook.

Click on the image for a larger view

The standard line graph for the same data lets you see some details more clearly. Google was indeed the most referenced company in all but a few months where it was barely exceeded by Yahoo (Sep 2006, Feb 2008) and FaceBook (Aug and Oct 2007). And references to Twitter did barely exceed Microsoft and Facebook in Apr 2008.
 
The standard line graph cannot usefully show the top 20 names because so many of the series overlap each other and can't be distinguished. The StreamGraph version for 20 names is much more useful at full size.

The above StreamGraphs show the texts delivered by Obama and McCain recently on the American economy. Click on either one to see more detail. Obama's remarks, given the title Renewing American Competitiveness, were delivered at Kettering University in Flint, Michigan on June 16th, 2008. John McCain delivered his remarks concerning America's Leadership in the Global Economy to the National Restaurant Association, in Chicago, Illinois, on May 19, 2008. Of course it's more informative to actually read the texts but these things do jump out from the graphics:

• McCain mentions 'tax' a lot more than Obama
• McCain mentions 'tax' a lot more towards the beginning of his speech than the end
• McCain mentions the large numbers 'million', 'billion' and 'trillion' a lot, they aren't prominent in Obama's remarks
• Obama mentions 'teachers', 'schools', and 'education' a lot, not McCain
• They both discuss 'jobs' and 'trade' although trade is a bit more prominent towards the end of Obama's speech
• Obama mentions 'oil' and 'energy'
• McCain mentions 'farmers' and 'subsidies'
• Obama mentions 'Bush', McCain doesn't mention him with any prominence

I have created another StreamGraph, this one for the book Little Brother, by Cory Doctorow. Click on it to see a larger version. It shows the distribution of proper noun references across the text. Here are a few things you can pick out from the graph:

• major people referenced seem to be Darryl, Ange, Van
• lots of secondary characters like Charles, Marcus, Mom, Dad, Jolu
• Jolu appears primarily around the middle of the text
• Ange referenced much more often in the second half of text
• About 1/3 through the text there are lots of references to Booger, Zit, and Pigspleen none of which seem to reappear afterwards to any great degree
• Booger, Zit, and Pigspleen seem associated with Internet, Xnet
• Masha figures prominently in the end of the text but not beforehand

See the post Tom Sawyer Character StreamGraph for a very brief description of how it was constructed. The design of the graph is based loosely on those created by Lee Byron.

Here is another for a different work by Cory, Down and Out in the Magic Kingdom.

The above image is a StreamGraph for the book The Adventures of Tom Sawyer, by Mark Twain. Click on it to see a larger version. It seems to do a pretty good job of communicating the ebb and flow of the various characters throughout the book. The Mississippi River figures prominently in the book so a stream-like representation of the text seems appropriate.

I have adapted the StreamGraph code used to create the various Twitter Topic Streams so I can create StreamGraphs from arbitrary text documents. The document is split up into 25 equal sized segments and the word counts are done within each segment. These segments are used in place of time along the horizontal axis of the StreamGraph. This document StreamGraph again focuses on capitolized words but ignores a few common ones like 'Mr' and 'Mrs'. I'm also using a longer format for the graph and showing two labels for each word series - one on the left half of the graph, and one on the right. The difference in label size for the same word can show whether it was used more frequently in the first or second half of the document. In the 'Tom Sawyer' graphic above you can clearly see that both 'Ben' and 'Mary' are more prominent in the first half of the text but that 'Huck' is more common in the second half.

The above StreamGraph illustrates the distribution of the most interesting capitolized words in the StatBot dataset of all the updates for the top 100 twitter users. I removed most place names (NY, Paris, Boston etc) and several common words like 'twitter', 'lol', 'company', 'web', and 'internet'. The interestingness of a word was quantified by a function of the total references as well as the burstiness of the word distribution.

The most 'interesting' words in this data are primarily product, technology, or technology event names with the exceptions of 'Scoble' and 'Obama'. This isn't surprising since the top twitter users are early-adopters interested in technology. I was a bit surprised at the large volume for Seesmic but discovered that it is a company founded by Loic Le Meur, the 6th top twitter user.

I also created the twitter topic stream for Robert Scoble shown below. The graphic does a pretty good job of highlighting the primary technologies Scoble focused on over the last year or so.

This StreamGraph shows the top twitter users based on the number of tweets sent during the period December 2006 until April 2008. Click the image to see a larger version with more of the labels legible.

I have mentioned before the wonderful stream-like visualizations created by Lee Byron. I've written some code so I can create my own using whatever data I want. The one above was constructed using the twitter data from The StatBot. You can click on it to see a larger version of the image. I left out the first few months which had a very low volume of data so this one runs from Dec 2006 to Apr 2008.

For a small number of series a simple line graph would be superior because you can directly see which values are larger at each point in time. These StreamGraphs do a better job of emphasizing the sum at each point and the breakdown into the various series. I think StreamGraphs are also better at showing lots of series that dominate for short parts of the timespan of interest. For example see the image below that shows movie revenues. There are a great many movies illustrated and each one is only present in a fairly small part of the overall range of time.

Lee Byron and Martin Wattenberg have written a short paper describing the design decisions and algorithms behind these types of graphics. Have a look at Stacked Graphs - Geometry & Aesthetics (pdf) if you are interested in the details.

1. chrisbrogan has the most tweets overall but his totals were eclipsed in most months by newmediajim
2. scobleizer jumped in quickly in Mar 2007 and had the most tweets of anyone in his first month of use
3. down at #21 'ev' seems to have the highest use during the first few months. This isn't surprising since 'ev' is Evan Williams, Co-founder of Twitter
4. guykawasaki and problogger are high in the overall usage rankings despite starting later than many of the others

I've constructed a graph showing how use of Twitter clients by power users has changed over time. I used a dataset containing all the tweets from the power users in the Twitterific Top 100 list which was graciously provided by Yuvi, over at The StatBot. Two full years of data, from May 1st, 2006 until April 30th, 2008 were used for the analysis. The main things that jump out for me are:

1. Web client use dominates
2. txt client use seems to have plateaued between Mar 2006 and Mar 2008
3. very rapid recent growth for Twhirl
4. decent growth for im
5. all the curves are fairly spiky

There have been some other recent posts giving statistics on the clients used most often to post updates to Twitter. One, from ReadWriteWeb, was called How We Tweet: The Definitive List of the Top Twitter Clients and was based on a random sample of over 37,000 tweets from the public timeline. The results were posted April 2, 2008 so I presume the data was collected shortly before then. The top 3 clients from their survey were:

1. Web 56% (20734)
2. IM 8% (2975)
3. Twhirl 7% (2754)

Yuvi, more recently, did a similar analysis based on the data he provided to me. He listed a number of findings by constrasting the two datasets including that the power users make use of both SMS txt messages (10% vs 5%) and Mobile Twitter (6% vs negligible) much more often than the typical user. He also claimed that the power users are using Twhirl less than the typical user (5% vs 7%). I believe this claim is incorrect.

The two studies mentioned above show an average client usage over very different time periods. The ReadWriteWeb study uses data from a 24 hour period around April 2, 2008 but the StatBot analysis uses a complete list of tweets that span a timeframe from March 21st, 2006 until May 25th, 2008. Drawing comparisons between two datasets based on such vastly different time periods should be done very cautiously. Twhirl is relatively new and Yuvi's analysis used lots of historical data before Twhirl was available.

The StatBot analysis showed that on average it was the 6th most popular client. In fact, if you look at the graph above at the point between Mar and Apr 2008, which corresponds to when the 'typical user' study was done by ReadWriteWeb, you can easily see that Twhirl was actually the second most popular client for power users. I've looked at the power user data for all tweets between Mar 31st and Apr 2nd, 2008 and there were 241 for Twhirl out of 1833 total - 13% , which is much higher than the ReadWriteWeb result of 7% for typical users. This makes sense to me - power users have more of an incentive to install a specialized client than an average user who doesn't use twitter very often.

Just for fun, (well, and to try and get them to link to me ! ), I have generated graphs for two of the top power users: Robert Scoble and Chris Brogan. Note that the clients are coloured differently in the three graphs. Ideally, for easy comparison, they should be consistent. Here are a few observations concerning their patterns of use:

1. Scoble has a huge peak for use of the web client around Mar 2007 and a lull in overall usage in June 2007
2. Scoble switched to be a heavy user of twitterific for the last half of 2007 and again had a peak in usage
3. Starting in Nov 2007 Scoble switched from twitterific to im as his primary client - web usage still common as well
4. Scoble dabbles in many other twitter clients as well
5. Brogan had a large drop in web client use at the same time as Scoble - Mar 2007, but didn't drop as far
6. Brogan has used the mobile client a great deal since Jun 2007

Thanks to Twitter and Summize for the data and Processing.org for the tools. Give TwitArcs a try !

TwitArcs (static image)

As before, one topic is coloured blue, the other red, and the associated words are coloured and positioned based on how highly they are associated with the two topics. Click on any word to see the related tweets. Give Twitter Spectrum a try ! As always, feedback is welcome.

Thanks to Twitter and Summize for the data, Processing.org for the tools, and Chris Harrison for the inspiration behind the design.

Twitter Spectrum (static image)