One company that tries to do this is Icosystem, they apply massive distributed computing resources to very large problems in the hopes of finding better strategies for large corporations.

Consider Google for a moment: it is a very large distributed computing system. It's 3 billion pages represent what I will go out on a limb and call a snapshot of the world's actual business systems and strategies that's only a little behind being real time.

Google is hardly a controlled experiment conducted under pristine conditions. It is what it is, a catch-as-catch-can record of amazing proportions. My guess is that in a record of this size signal and noise begin to move to discernible positions. You may not know if any given page or site is signal or noise, but averaged over large populations, the signal may be reliable.

I tried this theory out 2 weeks ago, to amuse some colleagues at a large software company. I searched on the suffix of their most popular file format, and discovered it showed up on 3 times as many pages as the suffix of one of their chief competitors. You would need to bring a lot of context to that finding, you wouldn't call it 'scientific', but it was what it was: a snapshot in the noise. Is it a signal? Is it reliable?

Good questions. Look at the chart at the top of this post. It represents Google searches on the word 'software' and prices from $9 to $999. There are a lot of reasons that the word software and a price appear on a page, but one is that the page represents an offer of some software for a certain price. The pages that don't are probably about the same for every price: I can't imagine why there would be more noise at, say, $49 than at $59 or $119.

And, since the real world is an actual Darwinian process, the Google snapshot may be a good proxy for what's happening and what's working. Here you see that software prices at $99, $9, $49 and $199 show up more often than the other price points. As in the former example, context and interpretation are important, and caution should clearly be exercised in how the data are used. But there are a lot of interesting things floating around out there...
Comments [ ] 8:43:52 AM    

Chris Gulker's view from Silicon Valley - in words and pictures