Real Time Analysis of Images Posted on Twitter Using Bluemix

In this blog post, I'll share my experience of developing a real-time image analysis application using Bluemix. The best thing is, you don't need an understanding of computer vision or image processing to develop such an application on Bluemix (thumbs up!).

Lets start! Text analysis on Twitter is a widely known and performed activity nowadays. Sentiment analysis and event detection are just a few of the things that researchers and companies do. However, analysis of images posted on Twitter has not been explored much in the past (although it's starting to pick up recently...). In this post, I'll show you how to build such an application within an hour worth of development using IBM's Bluemix platform. I hope you have enough knowledge about Bluemix before reading further. If not, you can read my introductory Bluemix blog post.

Please note that although Bluemix provides a Twitter Insight service in the catalog, it's still in Beta and therefore have some limited capabilities. Hence I opt out of Bluemix service for Twitter and directly use the Twitter API. However stay tuned for future updates on the Bluemix service.

Twitter provides a nice and well documented API on its website. In terms of the capabilities that I needed, I'm interested in getting real-time tweets about a topic (such as 'NYC' or 'Obama'). Therefore, for my case it looks like Public Streaming API is the way to go. In addition, I'd like to get only the posts that have images in it. However, it looks like current API doesn't support this kind of queries. As a result, I request all twitter posts about a specific query term and processed only the ones that have images.

To use Twitter API, first create a Twitter application on Twitter website and obtain credentials for accessing the API and note your consumer_key, consumer_secret_key, access_token and access_token_secret. Then choose a Twitter library that can make calls to Twitter's streaming API. Since I plan my app to be a Node js one, I use this one, however feel free to use another library. Once that's set up, go to Bluemix console and sign in.

On the dashboard click '+' for creating an application. Choose a web application.

Next select a runtime, I chose Nodejs but feel free to choose another one if you like. Once that's done, you need to name it. 

Once you name it, your app should be created and deployed on cloud foundry in a few seconds. Click on it. Now you should be given an option to download the starter code. Please do so and download it. 

Next go back to your application overview page (left menu) and click on add a service. Select Watson category and choose 'Visual Recognition'. 

Once you name the service and bind it to the application you created, you'll be able to see your service in the Overview page of your application. Click on "Show Credentials" and note the username and password which you'll need to make calls to the service. Please see this Github page to see how to make calls to the service.

You environment is now set up and now you can start coding. I'll not be going into the details of the code, but feel free to look at my quick implementation on github using express, passport and jade. The result of the app can be seen from this website. It basically shows a pie chart of labels of all the images posted in Twitter for the last 2 hours for NYC. I regard a category of a label as "Others" if it's less than %3 of all images. As can be seen from the chart, "Others" takes the majority due to the long tail distribution of images posted in Twitter.