mrry (Happy New Year)
Blog MSNBOT - The revenge 26/Jun/2003

Seems like everything I write these days is a refutation, but my technical credentials are far more important to me than my politics, so here comes my reply as a new article. Let me set the scene:

Recently, Microsoft has entered the search-engine race. This was first picked up by Jeremy Zawodny last week, and Microsoft's crawler started visiting mrry soon after. Also last week, I penned a welcome to the crawler, which criticised those who were opposed. However, Gary doesn't agree, and suggests that the cessation of Internet Explorer development is a foretaste of what would happen if Microsoft were to corner the market for searching the web.

Let's examine some Microsoft monopolies.

Windows is the number one operating system because the most popular software is written for it. Windows has the most popular software written for it because it is the number one operating system. Search engines aren't like operating systems, however. There's no issue of compatibility between the results from two different search engines. In fact, I'd bet that if all Windows software could run on Linux, then Windows' lead in the OS stakes would be far smaller, if not destroyed altogether.

Word is the number one operating system because it has a proprietary file format that also happens to be a standard format for the exchange of word-processed documents. If you want to make a Word document (at least, if you want to be able to use all of Word's features), you have to make it with Word. It's the same issue as with Windows. But since search engines all work the same way - type in a query and get a list of results - why should a user be tied to one only? Unless Microsoft creates its own Internet that only its search engine can search, this scenario should not arise.

Internet Explorer can be accounted for differently. I believe that Microsoft (rightly or wrongly) played on the inexperience of new computer users, in its quest to make IE the most popular browser. Because IE was launched before the massive upsurge in computer, to the (vast proportion of total computer users made up of) newbies, it didn't seem odd that a web browser was part of the operating system. In fact, it seemed downright handy, after all it would be an awful stress to download this "Netscape" thing, and what does that mean anyway? There's no verb in "Netscape". But because Internet Explorer established itself before the web entered the common consciousness, few people ever even knew that alternative browsers existed, so its monopoly was quickly won.

Further to that last point, consider a market to which Microsoft has come late - games consoles. Sony PlayStation was the market leader, it gained worldwide brand-recognition, Microsoft released the X-Box, and... Sony PlayStation2 is the market leader. It seems that Microsoft doesn't automatically conquer all in its path.

Google, however is to search engines what PlayStation is to games consoles (if not more so). Considering that search engines cannot be monopolised in the same way as OSs and word processors, I believe that Microsoft's search engine will have the same difficulty as the X-Box in gaining market share.

Of course, may we not forget that Microsoft already has a search engine of its own. Not only that, it is integrated into Internet Explorer (which, you will remember, is the most popular web browser), and Windows (ditto for operating system). But, unless I am most mistaken, Google is the most popular search engine. This leads me to two conclusions: MSN Search isn't very good, and, when there's a superior alternative, people won't always settle for any old crap from Microsoft.

There is one scenario I haven't considered, which might be worth consideration. Maybe Microsoft will defeat Google. But if it does so, it'll have to come up with a search engine that's at least as good as Google. And then won't we all be winners?

I apologise for the rambling nature of this reply, but it's because I'm writing it in an 80 columns by 8 lines text box, and I'm in no state to proofread. I have two final points for the boycotters.

Firstly, I don't think a boycott of the new crawler by bloggers will do anything to harm the search engine. Indeed, many are now saying that blogs have inordinate control over the Google rankings, so maybe by boycotting the Microsoft crawler, you will help create a more useful index, which will ultimately contribute to Google's demise.

Secondly, I still have no preference about who reads my site. You can be my friend, you can be a gay Australian neo-conservative called Scott, you can be a search engine bot - you're still welcome to read what I have to write. Far be it from me to criticise anyone who wants to curtail their readership: I simply take the opposite, embracing view.

Phew, there you have it. That's why I'm happy to let Microsoft in. Even if I weren't, it would be moot, because they've already crawled over 900 pages of my site. In fact, I'm getting hits from MSN Search on new pages that could only have been crawled by the new crawler, so I'd say it's working pretty well.



P.S. For your requisite dose of referrer log humour, I noticed this Useragent in the logs today:

Mozilla/4.0 (compatible; MSIE 4.01; Windows NT; MS Search 4.0 Robot) Microsoft

Needless to say, the IP doesn't trace back to Redmond, so I haven't been fooled this time.


Gary said:
I've been writing a CMS for around a year now. Development never ends. My advice (if you do decide to do it):

* Keep your presentation and content separate. Then things like changing between HTML, XHTML, XHTML 2.0 and whatever might come are irrelevant.
* Templating. Put a lot of thought into this at the beginning.
* Look at additional requirements. A great list of possible additions to a weblog system is available.

Incidentally, I use proper the proper XHTML MIME-types on my site: app/xhtml+xml for browsers that can handle it, text/html for others. The code for this is trivial.

Derek said:
It's also the reason why I haven't (yet) provided a better comment box than a 22x5 textarea....



Derek said:
I'm glad that you're lifting the ban on MSNBOT, and the "innocent until proven guilty" maxim is a very noble reason to do so.

It may turn out (and, since I lack the imagination of the Microsoft's interface designers, I can't imagine how) that the integration of MSN Search into Longhorn, and future versions of MS products, will turn out to be so popular with the masses that MSN Search usurps Google from the top position in searching. I don't think that uptake of these applications will be 100 per cent though, especially since MSIE is frozen (for legacy OSs) at version 6, and the "less-discriminating" masses may be content not to upgrade to the cutting-edge versions of the programs they use.

I hope, and I am an eternal optimist, that MSN Search does enough to challenge Google, and, in doing so, provokes Google into improving its search results in order to compete.

Maybe we can all win.

As for my shoddy markup, I had hoped that it wouldn't peek out from under the carpet. If you saw the PHP that this runs on, you may rightly wonder what claim I have to evangelise OOP.

The reasons I don't change this (at the moment) are twofold.

For one thing, there is a lot of old, downright bad HTML in the archives, and using either DTD would be a lie in these cases.

Secondly, I plan (though who knows if it'll ever come to pass) to try my hand at writing a proper CMS for this site. Beautiful class hierarchies would unfold, and it would be the most valid XHTML you'd ever seen, even down to the arcane matter of MIME types. I'd even use CSS, instead of tables, to lay the pages out.

But, to that end, I'm reluctant to touch the code upon which the site rests. Maybe by the end of summer, a better-presented site might appear in this place.

Maybe once they make an OODBMS to compete with MySQL on cost.... :P



Gary said:
Incidentally, is there any specific reason that you don't use XHTML (or even HTML 4.01 strict) and non-presentational markup for your site? It would cut the size of your pages down by a fair bit, and be a bit easier to write.

Gary said:
I still think that when MS throws their weight behind their new search features (something that they don't currently do), that more users will use them seamlessly. The biggest worry in this area is that they're asking for future hooks in other apps. They can't stop Google in the browser unless they get a better search going, I agree, but what if they just by pass the browser entirely? Seamless searching of MSN only tightly integrated into other apps is plausible and is being worked on. It's a bit like changing the playing field without letting google know.

I agree that in the early days of the browser wars IE was better than navigator. Look at the situation from around IE5 time. Mozilla is a far better browser than IE is (Firebird is better still, IMHO) but still more people use IE. Quality is not the be all and end all.

But despite my worries in this area, I've removed the block (or will do once I upload the new Robots.txt). Not because I think I'm wrong (I'm fairly certain that MS will try to lock the market with an inferior product and marketing) but because I believe in the basic old bit of law "innocent until proven guilty". Until they actually do it, I'll give them the benefit of the doubt.





Please enter the number 8485 in the box below:

CommentsTell a friend about this page

Your Name

Your E-Mail

Your friend's E-Mail


< # Scottish Blogs ? >
Technorati Profile
Listed on BlogShares

Subscribe to the mrry RSS feed
More about RSS.
Trackback URL for this article: