Microsoft's Robert Scoble Discusses Search Engine Technology

From mboxrd@z Thu Jan 1 00:00:00 1970 From: Hans Reiser Subject: Microsoft's Robert Scoble Discusses Search Engine Technology Date: Sun, 08 Feb 2004 20:13:17 -0800 Message-ID: <402708DD.6030806@namesys.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------020705010800080106040300" Return-path: list-help: list-unsubscribe: list-post: Errors-To: flx@namesys.com List-Id: To: ReiserFS List --------------020705010800080106040300 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit http://www.searchengineguide.com/beal/2004/0204_ab1.html --------------020705010800080106040300 Content-Type: text/html; charset="iso-8859-1"; name="www.searchengineguide.com/beal/2004/0204_ab1.html" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline; filename="www.searchengineguide.com/beal/2004/0204_ab1.html" Content-Base: "http://www.searchengineguide.com/beal/ 2004/0204_ab1.html" Content-Location: "http://www.searchengineguide.com/beal/ 2004/0204_ab1.html" Microsoft's Robert Scoble Discusses Search Engine Technology

	Searc= h Engine News Daily search engine news. Plus, thousands of indexed articles.	Search Engin= e Marketing Learn how to use search engines to increase traffic and sales.	Internet= Search Engines Directory of search engines categorized by topic.	Search EngineBooks & More Find books, consultants to improve your rank, copywriters, etc.
Tell = A Friend About This Site \| About \| Advertise &nbs= p;\| = Newsletters \| Site Map

Navigation: Searc= h Engine Guide Home Page > Andy Beal > Microsoft's Robert Scoble Disc= usses Search Engine Technology

Andy Beal
Articles= - Bio

Microso= ft's Robert Scoble Discusses Search Engine Technology
By Andy Beal - February 04, 2004

In my recent article "The Future of Search Engine Technology", I looked at a = lot of developments that might happen in the future, that would improve sea= rch technology. I strongly believe that we are witnessing the infancy of se= arch engine technology, but I wanted to hear what others had to say. Today,= we start a series of interviews with prominent experts, insiders and searc= h engine developers to hear what their thoughts are for the future.

If you've been online for any length of time and work in any industry conne= cted with the Internet, you would heard of, Robert Scoble. The Microsoft em= ployee maintains a daily blog= (when he's not working to get the word out about Microsoft's new "Long= horn" operating system) where he gives his thoughts on all things Microsoft= , while also casting a critical eye on the competition. Scoble does a great= job of keeping a distinct line between what's "official" and what's simply= his opinion.

I was fortunate enough to catch him taking a sabbatical from his blog and a= sked him his thoughts on the future of search engine technology. Scoble did= ask me to note that the following represents his personal opinions, not Mi= crosoft-vetted opinions.

[Andy Beal] Robert, tell me about the search engine technology being dev= eloped that you are most excited about?

[Robert Scoble] That depends on whether you're talking about Internet searc= hing, or searching on your local hard drive. If we're talking about your lo= cal hard drive, searching for files on your local hard drive is still awful= and getting worse.

[AB] Why do you say that?

[RB] Because hard drives keep getting bigger (a 60GB drive at Fry's Electro= nics is $60 now -- in three years we predict it'll be $20 and you'll see 50= 0GB drives for less than $100). It's easier to create files now than it is = to find them.

Today search tools like X1 are most interesting because they index your har= d drive and make it easy to search for email and files on your local drives= . Microsoft Research has been working on a tool called "Stuff I've Seen" to= o, which is also quite interesting (both let you search email as well as fi= les on your hard drive). But, these tools don't go far enough. First, they = are bolted on top of the operating system. So, while they are indexing, you= r system often sees slowdowns. They can't design those to work properly wit= h the operating system and with other applications that might need processo= r time.

Plus, to really make search work well search engines need metadata and meta= data that's added by the system keeping track of your usage of files, as we= ll as letting application developers add metadata into the system itself. I= n a lot of ways, weblogs are adding metadata to websites. When a weblog lik= e mine links to a web sit= e, we usually add some more details about that site. We might say it's a "c= ool site" for instance. Well, Google puts those words into its engine. That= 's metadata. (Technically metadata is "data about data"). Now if you search= for "cool site" you'll be more likely to find the site I just linked to. S= o, you can see how Google's engine is helped by metadata. But, we haven't b= een able to apply those lessons to the thousands of files on your hard driv= e. That will change.

[AB] Can you explain the problems faced with searching hard drives and w= hat Microsoft is working on to help?

[RB] What if we did the same thing on your hard drive [as Google]? For inst= ance, look at pictures. When I take pictures off of my Nikon, they have som= e metadata (for instance, inside the file is the date it was taken, along w= ith the exposure information) but that metadata isn't useful for most human= searches. For instance, how about if I wanted to search for "my wedding ph= otos?" Neither X1, nor Windows XP's built in search would find your wedding= photos. Why? Because they have useless names like DSC0001.jpg and there's = no metadata that says they are wedding photos.

Let's go forward a couple of years to the next version of Windows, code-nam= ed Longhorn. In Longhorn we're building a new file storage system, code-nam= ed WinFS. With WinFS searching and metadata will be part of the operating s= ystem. For instance, you could just start typing in an address bar "W" and = "E" and "D" and "D" and anything that started with WEDD would come up to th= e top. For instance, your wedding documents, spreadsheets, and photos.

But, WinFS goes further than X1 and other file search tools do today. It le= ts you (and developers of apps you'll use) add metadata to your files. So, = even if you don't change the name of your files, you might click on one of = the faces in a picture application and get prompted to type a name and occa= sion. So, you would click on your cousin Joe's face, type in "Joe Smith" an= d "Wedding."

Now whenever you search for wedding stuff, that photo will come up. And tha= t's just the start. If you imported a group of photos into a wedding album,= you'd be adding metadata for the search engine to use. In other words, you= 'll see a much nicer system for searching your local hard drive.

[AB] It looks like Microsoft has things mapped out for offline searches,= but can they compete with Internet search engines?

[RS] Now, if we're talking about the Internet, then Google has done an awes= ome job so far. I use Google dozens of times a day. Will MSN [search] be ab= le to deliver more relevant results than Google? I don't know. Certainly th= at's not the case today. Will that change tomorrow? I'm waiting to see what= the brains at MSN do.

One thing I do see is that in Longhorn, search will be nicer for customers.= Google is working on making its toolbar the best possible experience. We'r= e working on a whole raft of things too. I'm very excited about the future = of search, no matter which way things go.

[AB] Let's look beyond the next couple of years. What new developments i= n search do you see happening in the next 3-5 years?

[RS] For Internet searches, I see social behavior analysis tools like Technorati becoming far more importan= t. Why? Because people want different ways to see potentially relevant resu= lts. Google took us a long way toward that future as their Google's results= are strongly influenced by how many inbound links a site has. But, now, le= t's go further, even further than Technorati has gone. Let's identify who r= eally is keeping the market up to date on a certain field and give him/her = more weight.

I also see that search engines that search just specific types of content (= like Feedster) are going to be mor= e important (Feedster only searches RSS and Atom syndication feeds).

Oh, and users are going to demand new ways of exporting searches. Google sh= owed us that with News Alerts. Enter in a search term, like "Microsoft" and= get emailed anytime a news source mentions Microsoft. Feedster goes furthe= r than that. There you can build an RSS feed from a search term. I have sev= eral of those coming into my RSS News Aggregator and find they are invaluab= le for watching what the weblogs are saying about your product, company, or= market. For instance, one of my terms I built a feed for is "WinFS" -- I'l= l be watching to see how many people link to this article and if any of you= have something interesting to say I'll even link back.

[AB] Let's look at your "wish list". Assuming there were no restrictions= in technology, what new feature would you like to see introduced to search= engines?

[RS] I want to see far better tools for searching photos -- and connecting = relationships between all types of files and photos. For instance, why can'= t I just drag a name from my contact list to associate that name with a fac= e in a photo? Wouldn't that help searching later on? In just 18 months I've taken 7400 photos. But I can't search any of them very well = today without doing a lot of renaming and other work.

[AB] What impact do you see social networking having on the future of se= arch engine technology?

[RS] We're already seeing an impact over on Feedster and Technorati. It's h= ard to tell what'll come in the future, but what would happen if everyone i= n the world had a weblog and was a member of Google's Orkut? Would that change how I'd search? Well, for one, it= 'd make me even more likely to search for people on services that linked to= gether social spaces and weblogs. Heck, I can't remember my brother's email= address, but Google finds his weblog (and I can send him an email there).

One other thing I'll be watching is how Longhorn's WinFS gets used by appli= cation developers to build new kinds of social systems. Today if you look a= t contacts, for instance, they are locked up in Outlook, or another persona= l information management program like ECCO. But, contacts in Outlook can't = be used by other applications (particularly now because virus writers used = the APIs in Outlook to send fake emails to all contacts in Outlook, so Micr= osoft turned those features off).

[AB] WinFS changes that. How?

[RS] By putting a "contacts" file type into the OS itself, rather than forc= ing applications developers to come up with their own contacts methodology.= =20

What if ALL applications, not just Outlook, could use that new file type? W= hat if we could associate that file type to social software services like F= riendster, Tribe, Yahoo's personals, or Google's Orkut? Would that radicall= y change how you would keep track of your contacts? Would that make contact= s radically more useful? I think it would.

Already we're seeing systems like Plaxo<= /a> keep track of contacts, but Plaxo is still unaware that I've entered my= data into Google's Orkut and Friendster. Why couldn't I make a system that= 'd associate the data in all my social software systems? Including Outlook?= =20

[AB] Do you foresee any problems with the WinFS approach?

[RS] Developers distrust Microsoft's intentions here. They also don't want = to open up their own applications to their competitors. If you were a devel= oper at AOL, for instance, do you see opening up your contact system with, = say, Yahoo or Google or Microsoft? That's scary stuff for all of us.

But, if the industry works together on common WinFS schemas (not just for c= ontacts either, but other types of data too), we'll come away with some rea= lly great new capabilities. It really will take getting developers excited = about WinFS's promise and getting them to lose their fears about opening up= their data types.

=20 [AB] Do you foresee a time when commercial search results (product/servi= ces) will be separated from informational search results (white papers/educ= ational sites)? And do you think all commercial listings will eventually be= paid only?

[RS]I don't see the system changing from the Google-style results today. Se= archers just want to see relevant results. Paid-only searches won't bring t= he most relevant results.

[AB] What makes you say that?

[RS] Because I often find the best information on weblogs. Webloggers are n= ever going to be able to afford to pay to be listed on search engines.

Commercial-only listings might be seen on cell phones or PDAs, though. If I= were doing a cell phone service for restaurants in Seattle, for instance, = I might be more likely to list just member sites. But, thinking about it, I= still don't see such a system becoming popular enough without listing ever= y restaurant in some way.

[AB] Speaking of cell phones. How do you see search engine technology im= pacting our use of PDAs and Cell phones?

[RS] Not sure if search engine technology will impact it, but the mixture o= f speech recognition with search engines might change it a lot. When I'm us= ing my cell phone I don't want to look at sites that have a lot to read (I'= ll save those for later when I'm in front of a computer or my Tablet PC) bu= t, instead, I want to find the closest Starbucks. Look up movie listings. F= ind a nice place to have a steak dinner. Now that cell phones are reporting= e911 data (that means that the cell phone system knows approximately where= you're located, so can give you just one or two Starbucks, rather than all= of the ones in Seattle).

[AB] If search engine users gave up a little of their privacy and allow= ed their search habits to be monitored, would this allow the search engines= to provide better, customized results?

[RS] Yes. I already give Google the ability to watch my search terms (I use= the Google Toolbar). But, it always must be a choice. People really hate i= t when you don't have strict privacy policies that are easy to understand a= nd they hate it if you don't give them a choice to not report anything.

[AB] Robert, you've certainly opened our eyes to the future of search en= gine technology, is there anything else you would like to add?

[RS] To echo what I said above, I hope the industry sees the opportunities = that Longhorn's WinFS opens up. We can either work together and share data = with each other, or we can be afraid and keep data to ourselves. It'll be a= n interesting time to watch in the next three years.

Many thanks to Robert Scoble, Microsoft employee and blog extraordinaire. P= lease be sure to visit Sear= chEngineLowdown.com as we continue to highlight the thoughts and views = on the future of search engine technology.

Navigation: Searc= h Engine Guide Home Page > Andy Beal > Microsoft's Robert Scoble Disc= usses Search Engine Technology
<= b>Tell Your Friends About This Site

Andy Beal is Vice President of Search Marketing for KeywordRanking.com and ProRanking.com, global leaders in professional search engine= marketing. Highly respected as a source of search engine marketing advice,= Andy has had articles published around the world and is a repeat speaker a= t Danny Sullivan's Search Engine Strategies conferences. Clients include Al= aska Air, Peopleclick, Jos. A. Bank and NBC. You can reach Andy at andy@proranking.com and view his daily = SEO blog at http://www.sear= chenginelowdown.com/.

Copyright =A9 2004 K. Clough, Inc. All Rights Reserved. --------------020705010800080106040300--