Lehrblogger

previously: project proposal, midterm progress, initial idea, Twitter/Scala experiments

I’ve made substantial progress on my Twitter/globe project for ICM. I rewrote and improved the Twitter search code that fetched new retweets in PHP to work with a MySQL database on my Dreamhost web server. For each retweet found by that PHP script, it uses the @username syntax and text of the retweet to find the original retweet. If another retweet is found (i.e. there is a chain of retweets) it recurses back until it finds the original. If all of the tweets in the chain have valid location data, it stores each tweet with a unique ID (assigned by Twitter), the author’s username, the text of the tweet, the latitude/longitude, and the time.

It also stores the number of retweets that a tweet has – in a three tweet chain, the original tweet will have 2 retweets, the first of the two retweets will have 1 retweet, and the final retweet will have 0 retweets. It also stores the ID of the tweet’s parent tweet – in that same example, the original tweet will have no parent, the first retweet will have the ID of the original as its parent, and the second retweet will have the ID of the first retweet as its parent. This data will make it relatively simply to choose and display chains and trees of retweets on the globe by filtering for interesting structures and long distances between tweets.

You can see that PHP script here (sorry for the lack of comments, I’ll add them later), and it’s been running a few times an hour for the past several days. I have over 10,000 tweets in my database; about 5,500 of them are leaf nodes and have no retweets, 4,400 of them have one retweet, 500 have two retweets, 100 have three retweets, 30 have four retweets, and 25 have five or more retweets. I’m getting good data and will be able to draw interesting chains and trees on the globe, especially after it has been running for a few weeks.

As a side note, it’s been interesting to watch the sort of things that people are retweeting. For example, there was a flood of heavily retweeted content after the terrorist attacks in Mumbai. The three most heavily-retweeted tweets are (with 11, 14 and 11 retweets, respectively):

• chrisbrogan: 15 year old girl needs a kidney. She’s dying. Can you help? Can you at least retweet? http://tinyurl.com/6bue7f
• SantaClaus25: Please re-tweet and make history “Ho Ho Ho @Santaclaus25 and all of us here at the North Pole wish your family a Happy Thanksgiving”
• zigzackly: #mumbai sortable list of injured/dead at http://tinyurl.com/5q23h6 – please retweet

In addition, I’ve finalized the name (formerly Retweet Tree) to TwiTerra. I registered the twiterra.com domain, and for now it currently redirects to a WordPress page for the project. A project description and one-line pitch are coming soon, since the submission deadline to be in the show is December 1st.

The courses for Spring semester were posted recently, and you can see a full listing of the exciting options here. Course sizes are limited to 16(?) students, and some courses will be more popular than others, so each student is asked to rank their top ten choices. An algorithm then assigns students to classes in a random order, with preference given to second-year students over first-years. The results will be made available on Monday, and my preferences were as follows:

I’ll be taking four classes in total, and ideally I will have a spot in each of the first three and then choose from one of the next four. I’m also interested in the subsequent ranked (and unranked) courses, as well as several that aren’t on that list. The final ordering I settled on isn’t necessarily a direct representation of my preferences, as factors such as expected popularity, expected future course offerings, time conflicts, and subject matter similarities all needed to be taken into account. Additionally, it’s difficult to judge a course from a paragraph of description, and I intend to attend as many classes as I can during the first week (regardless of whether or not I am on the waiting list), so my schedule will be in flux until then.

Details have been announced for the Winter Show! I’m hoping my Twitter project will be selected for display, but we’ll see.

ITP Winter Show 2008
Wednesday, December 17 and
Thursday, December 18
from 5 to 9pm at ITP

A two-day festival of interactive sight, sound and technology from
the student artists and innovators at ITP.

http://itp.nyu.edu/show

Founded in 1979 as the first graduate education program in
alternative media, ITP has grown into a living community of
technologists, theorists, engineers, designers, and artists. This
two-year graduate program gives 220 students the opportunity to
explore the imaginative uses of communications technologies — how
they augment, improve, and bring delight and creativity into people’s
lives.

Housed in the studios of NYU’s Tisch School of the Arts, ITP takes a
hands-on approach. Students learn to realize their ideas through a
hands-on approach of building, prototyping, and testing with people.

Interactive Telecommunications Program
Kanbar Institute of Film and Television
Tisch School of the Arts
New York University
721 Broadway, 4th Floor, South Elevators
New York NY 10003

Take the left elevators to the 4th Floor
This event is free and open to the public
No need to RSVP

For questions: 212-998-1880
email: itp.inquiries@nyu.edu

http://itp.nyu.edu/show

We look forward to seeing you!

Feel free to pass this message along far and wide.

The assignment was as follows:

Please write a 1500-2500 word doc on non-professional political media, outlining what strategic advice you would give a Presidential contender running in 2012 (which is to say, forming an exploratory committee in 2010.)

Topics can include, but do not need to be limited to: how to reach out to media producers, how to solicit research on your opponent, how to frame or benefit from media in favor of your candidate, how to frame or benefit from media against your opponent, how to neutralize media praising your opponent or attacking you, the difference between media produced in the primary vs. general campaign, and so on.

(PDF)

previously: midterm progress, initial idea, Twitter/Scala experiments

For my final project I plan to integrate my previous work with NASA’s World Wind, Twitter, and Scala into more recent work with PHP and MySQL. I will continue to focus on Twitter ‘retweets‘ and will use a PHP script to search for recent retweets (using a variety of search terms – ‘RT’, ‘Retweet’ and ‘RTWT’ as well as ‘Retweeting”) and store them all in a MySQL database. I’ll use the database information and additional searches to link retweets to previous retweets that had the same content and to the original source tweet.

I will then use the twittervision API or the Twitter API to associate geographic data with each of those tweets, and I will filter the database to keep only linked chains of tweets for which all tweets have an associated location. These chains of tweets can then be further filtered by distance between Tweets, since larger distances will create more interesting visualizations. They can also be filtered by complexity of the chain – I have some expectation of seeing tree-like branching, in which multiple people retweet a single tweet, and then other people retweet those retweets.

I’ll then visualize these retweet trees on a globe. The globe will rotate itself such that the location of the first tweet is visible, that tweet can be marked with a dot and displayed with an annotation, and then a line can be drawn to the next level of tweets and the globe can rotate itself again to illustrate the progress of the idea. After the leaf nodes of the tree are drawn, the entire tweet can fade slightly (but still remain visible), and the process can repeat for another tree. I will use the database information to always have enough retweet trees to create an appealingly dense web around the world.

I also hope to create a web visualization, using a similar but simpler Flash globe visualization with the Poly9 FreeEarth.

The third and final component will be more interactive, and will be based on Twitter’s current feature of ‘nudging’ users who have not tweeted in the past 24 hours (and have expressed that they wish to receive the reminders). I will create a new Twitter account which interested users can follow, and it will watch their tweeting patterns to determine if that user has not posted a new tweet in a certain amount of time. The twitter account can then automatically suggest a much-retweeted tweet for that user to retweet as well. Thus that user can have an awareness of current popular ideas on Twitter, can participate in the viral spread of these ideas, and can interact with the trees of retweets as they grows.

I’ve figured several things out with World Wind that I was having trouble with in the previous version, but there aren’t enough updates for a new video. Current versions of the files, however, are below:

• WWJApplet.scala
• TweetHandler.scala
• RetweetPair.scala
• Retweet.scala
• Tweet.scala