As I’ve mentioned before, researching the culture of NaNoWriMo and Script Frenzy for Wikiwrimo is hard. Really hard. Blame the lack of Wayback Machine in recent years, blame the lack of site and forum archives in 2002 and 2004 and 2008, blame whatever you want. When a tradition starts early, it’s hard to dig through all those search results, and even if you wanted to dig through all of them, the search engine will cut you off eventually for site performance reasons.
So I started investigating the features of Google, Yahoo, and Bing. I had a hunch from the start that Google would come out as the winner, and as much as I love using Goodsearch for my searching needs, there are some things that the Yahoo-powered Goodsearch just can’t find. (Sorry.)
The primary item I investigated was searching within a timeline. If I knew that, say, Mr. Ian Woon came along before 2003 (and he must have since he’s mentioned in the 2003 forums quite a bit, though I can’t find a post where someone realized that Mr. Ian Woon is a nifty NaNoWriMo anagram), then being able to search the pre-2003 Internet would be a wonderful thing. There’s just one problem.
None of the major search engines can do that. Bing can’t do it at all from what I can tell. Yahoo and Google’s time-sensitive results can search only recent results, not exclude them. Yes, excluding recent results would likely lead to a much wider pool of results, but not if you can put a cap to when the results were created. It would be a great way to create a function of that search term. How many terms were being added in this interval, and how quickly were they being added? How many disappeared because the pages or relevant search terms were removed? This function would definitely be increasing with large derivatives when the term is getting discussed a lot. As an example, last week the iPhone would have a large derivative. Actually, the derivative would still be large and probably increasing since people have the things in their hands and rumors are buzzing even more loudly about the iPhone going to Verizon.
But what’s the long-term behavior of a given search term? Or more interestingly, how could such a function be useful?