Wednesday 27 November 2013

Trending topics using wikipedia

What is a Trending topic?

A word, phrase or topic that is tagged at a greater rate than other tags on twitter  is said to be a trending topic. Trending topics become popular either through a concerted effort by users, or because of an event that prompts people to talk about one specific topic. These topics help Twitter and their users to understand what is happening in the world. (from wikipedia).


How can we extract Trending topics from wikipedia?


Using the page view statistics of wikipedia, we can find out what are the trending topics during any single hour, a day, a month and an year. The page view statistics include requests for reading wikipedia, editing articles,etc. 

The advantage of extracting trending topics from wikipedia over just looking up trending topics from twitter is that sometimes you cant make meaning out of the hash(twitter) that is trending. For example: if #apple is trending in twitter, is it Apple company or is it some other apple?.. Unlike twitter handles, the page view statistics directly relate to an article on wikipedia. So we clearly know the meaning of what is trending. 

Since articles on wikipedia have categories, we can also say what is trending in a particular category. The only problem is we cannot have location wise trends, which twitter provides. 

The pageview stats for every hour are available on wikimedia site. It is based on Western European time. You can find them at  http://dumps.wikimedia.org/other/pagecounts-raw/

You can see the trending topics from wikipedia and twitter extracted at the same time. Twitter trending topics #brandchat does not make any sense, while all of the topics extracted using wikipedia stats are meaningful.  

Sometimes you dont know why some topic is trending on twitter for some vague concepts.
But this is not the case for Wikipedia, because any topic should be a wikipedia article first. This removes the chance for some vague group of words to trend. 


Trending topics based on wikipedia 1 hr page view statistics at 11:13 IST on 2013/11/27
Twitter trends in United States and India at 11:13PM IST on 2013/11/27

We can reason the presence of each of the trending topics we got from wikipedia. We take help of Google Search here.


  •  Banksy


  •  Thanksgiving: Tomorrow is the thanksgiving day for 2013. So this is obvious why it is trending


  •  Ian Watkins and Lost Prophets are trending because


  • Noida double murder case is trending because




  • Black friday(shopping)....This is obvious because tomorrow is the black friday
  • Hanukkah started today. It is a week long festival and hence might stay in top 10 for some time now.
  • Topics like Java, Online advertising trend without any reason as they are very generic and we expect these topics to trend almost 50% of the days. 
  • Expo 2020 ?? ...Here is the reason