From: Digest To: "OS/2GenAu Digest" Date: Sat, 13 Sep 2008 00:00:33 EST-10EDT,10,1,0,7200,4,1,0,7200,3600 Subject: [os2genau_digest] No. 1708 Reply-To: X-List-Unsubscribe: www.os2site.com/list/ ************************************************** Friday 12 September 2008 Number 1708 ************************************************** Subjects for today 1 Fwd: Robots.txt files restricting msnbot at http://www.warpspeed dot com dot au/robots.txt and http://firedragon.warpspeed dot com dot au/firedragon/robots.txt : Chris Graham [WarpSpeed]" **= Email 1 ==========================** Date: Fri, 12 Sep 2008 08:38:44 +1000 (EST) From: "Chris Graham [WarpSpeed]" Subject: Fwd: Robots.txt files restricting msnbot at http://www.warpspeed dot com dot au/robots.txt and http://firedragon.warpspeed dot com dot au/firedragon/robots.txt They are persistent aren't they? -Chris ==================BEGIN FORWARDED MESSAGE================== >Return-Path: v-amwilc at microsoft dot com >Received: from smtp.microsoft dot com (mail1.microsoft dot com [131.107.115.212]) by warpspeed dot com dot au > (Hethmon Brothers Smtpd) id 20080912025112-27754-8 ; Fri, 12 Sep 2008 02:51:13 +1000 >Received: from tk5-exhub-c104.redmond.corp.microsoft dot com (157.54.88.97) by > TK5-EXGWY-E801.partners.extranet.microsoft dot com (10.251.56.50) with Microsoft > SMTP Server (TLS) id 8.1.291.1; Thu, 11 Sep 2008 09:51:12 -0700 >Received: from NA-EXMSG-C121.redmond.corp.microsoft dot com ([157.54.62.192]) by > tk5-exhub-c104.redmond.corp.microsoft dot com ([157.54.88.97]) with mapi; Thu, 11 > Sep 2008 09:51:08 -0700 >From: "Amy Wilcox (Murphy & Associates)" >To: "chrisg at warpspeed dot com dot au" >Date: Thu, 11 Sep 2008 09:51:05 -0700 >Subject: Robots.txt files restricting msnbot at > http://www.warpspeed dot com dot au/robots.txt and > http://firedragon.warpspeed dot com dot au/firedragon/robots.txt >Thread-Topic: Robots.txt files restricting msnbot at > http://www.warpspeed dot com dot au/robots.txt and > http://firedragon.warpspeed dot com dot au/firedragon/robots.txt >Thread-Index: Acj76L7BxO2EMrXSSp2zEoogyoG5Nw== >Message-ID: >Accept-Language: en-US >Content-Language: en-US >X-MS-Has-Attach: >X-MS-TNEF-Correlator: >acceptlanguage: en-US >Content-Type: multipart/alternative; > boundary="_000_BFDD4CFB59C12F4A84197A31EDDB328408958B0638NAEXMSGC121re_" >MIME-Version: 1.0 >Return-Path: v-amwilc at microsoft dot com > Hi, I am contacting you from the Microsoft Corporation and its Internet search engine Live Search (http://search.live dot com) in regards to your robots.txt files located at http://www.warpspeed dot com dot au/robots.txt and http://firedragon.warpspeed dot com dot au/firedragon/robots.txt. Our customers have alerted us that your sites were not visible in our results and we have discovered that you are hiding your sites to our search engines crawler, named msnbot, via your robots.txt files. We would be pleased if you edit your robots.txt files to allow our search engine to index your content which additionally will increase traffic to your sites via our search results. The only step needed on your part is to remove the following restriction for msnbot within the robots.txt files. User-agent: msnbot/* Disallow: / If you feel that msnbot is crawling your sites too frequently, you can use the crawl delay directive in robots.txt http://blogs.msdn dot com/webmaster/archive/2008/04/18/ramping-up-msnbot.aspx For more information about robots.txt http://blogs.msdn dot com/webmaster/archive/2008/06/03/robots-exclusion-protocol -joining-together-to-provide-better-documentation.aspx Also our Webmaster Central website (http://webmaster.live dot com/) can assist you further in obtaining the best results possible for your business or website. If you have further questions please let me know. Best regards, Amy Wilcox Web Analyst, Microsoft Live Search v-amwilc at microsoft dot com ===================END FORWARDED MESSAGE=================== -Chris WarpSpeed Computers - The Graham Utilities for OS/2. Voice: +61-3-9395-1504 Internet: chrisg at warpspeed dot com dot au FAX: +61-3-9395-7633 Web Page: http://www.warpspeed dot com dot au Postal: WarpSpeed Computers, PO Box 4293, Hoppers Crossing DC, VIC 3029, AUSTRALIA ----------------------------------------------------------------------------------