Craig Donato’s Response…

my friend craig was nice enough to respond in the comments section of my previous post on google base and i thought i’d repost his points to make sure they’re seen (why is it that rss doesnt include comments?).  and btw, i hope everyone is starting to check out memeorandum which does a great job of aggregating conversation threads through links.

Hi Mark,

I wasn’t trying to make a value judgement on what Google is doing. I only wanted to note that they’ve crossed a line.

Up until this point, Google has take the role of the "pure" search engine (v. portal) and maintained a neutral stance with respect to the content in their index. I admired that they wanted to get users off their site as quickly as possible.

Now they will be in the business of generating content and competing with the publishers in their index. Not an evil thing to do, just very "portal-like".

To directly respond to a few of your open questions…

"arent you running a service that scrapes other classifieds sites but doesnt allow your own to be scraped?"

Nope. We are heavily crawled by google, yahoo, msn, etc.

"arent you competing with the sites that you are ‘indexing’?"

Nope. We’re not in the business of selling classified listings. Our aim is to focus on the needs of buyers who want to quickly see all the listings that are out there.

"isnt the hope for oodle that people will go to you first and then click on ads you sell while looking at classifieds from other sites."

Yes, we want to be the first stop for buyers. I think the classified experience for buyers stinks (which is why I started Oodle). If we can better serve buyers by making it easy for them to find what they’re looking for, we in turn help sellers and those in the business of selling them advertising.

ok. craig sheds some light here but not sure why you’re still not directly answering my question about being crawled. no room for politicians on this blog! are you saying that *anyone* can crawl you, or that anyone you consider non-competitive can? you mention google, yahoo, msn. could i start a competing classifieds aggregator and crawl your site?

if you are saying that you dont want to allow a competing service crawl you and reuse what you consider to be your intellectual property, then i respect that decsions, however…then you must also respect craig newmark’s decision that your service is ultimately competitive so he wont allow crawling.

this is a very hard question. if oodle’s business is to crawl and normalize data and then resell it to newspapers, how could oodle let me crawl them and offer it free? however, it doesnt seem fair to crawl other people but then restrict it yourself. catch 22. i still say that anyone who crawls my data should allow themselves to be crawled too or lose that right.

does anyone know if google allows themselves to be crawled? if not, that would be the king of all injustices. and if not, i really dont see why craig newmark would want to allow google either if they are truly launching google base.

8 thoughts on “Craig Donato’s Response…

  1. Hey Mark,
    Thanks for highlighting my response.
    People can crawl our site as long as they 1.) want to send traffic to my site and 2.) do it appropriately by following robots.txt (www.oodle.com/robots.txt). Moreover, if they want to display the results of the RSS feeds that we syndicate (which don’t send traffic to our site), I’m okay with that too as long as it isn’t excessive and they provide attribution.
    I absolutely agree that Jim Buckmaster has every right to have us not index Craiglist. I’ve said this consistently in my blog and to the press. Search is and should be an opt-out medium.
    That said, I can believe this (and honor Jim’s request) but also disagree with him. I firmly believe that what we’re doing is good for consumers trying to buy/rent things using online classifieds, good for the people/businesses placing classifieds ads, and good for the sites that publish those ads.
    I look forward to indexing GoogleBase classified listings and expect that it will be allowed. To see what you can currently crawl in Google check out http://www.google.com/robots.txt.
    Best, Craig

  2. What data do you think it’s wrong for Google to not share? Just the raw data of the classified listings themselves, or also the data about the factors Google uses in ranking them? I don’t think Google would be happy if another site scraped Google search results and rebranded them.

  3. Point #1: When MSN was developing their own search, there was speculation that they were scraping Google search results to build their own index: http://www.webpronews.com/insiderreports/searchinsider/wpn-49-20041111MicrosoftCrawlingGoogleResultsForNewSearchEngine.html
    Point #2: A friend was told by Oodle that they couldn’t spider Oodle, but could take the *very skinny* RSS feed (basically an ad title, date and link to Oodle). Now that Craigs is saying he is open to anyone’s robot, I’ll ask him to check whether they REALLY mean it.

  4. Stan,
    I believe you are referring to a conversation where I recommended that the person subscribe to our RSS feeds not crawl our site given their objectives. I never said they couldn’t crawl the site.
    That said if someone inappropriately crawls our site, we certainly reserve the right to block them by adding them to our robots.txt file.

  5. Stan,
    I believe you are referring to a conversation where I suggested that the person subscribe to our RSS feeds not crawl our site given their objectives. I never said they couldn’t crawl the site.
    That said if someone inappropriately crawls our site, we certainly reserve the right to block them by adding them to our robots.txt file.

  6. The statement that Oodle does not allow classified posting of listings on its web site (or sites) and hence does not compete with papers etc is NOT true. Oodle has co-branded websites with other online properties such as Lycos and allows FREE classified posting on those sites. Looks like, in reality the classified listings are posted onto Oodle servers and databases directly.
    *** Papers and other media partners who send content to Oodle need to be careful.
    To see this ad process,
    Go to Oodle >> “Post an Ad” (http://sf.oodle.com/post/) click on “Lycos” (http://lycos.sf.oodle.com/post/) and you will see it is nothing but a posting/listing process hosted by oodle.
    ** So although Oodle says it does not receive content directly from end users, in reality does accept free classified ads and competes directly with folks they want to partner with…
    David

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s