# WARNING: DO NOT EVER VISIT DISALLOWED DIRECTORIES MARKED AS TRAP, OR YOUR IP WILL BE BANNED User-agent: ia_archiver User-Agent: Googlebot-Image User-Agent: yahoo-mmcrawler User-agent: psbot User-agent: Wget User-agent: Microsoft URL Control User-agent: grub-client User-agent: Nutch User-agent: MJ12bot User-agent: Mnogosearch User-agent: NetResearchServer User-agent: IncyWincy User-agent: InnerpriseBot User-agent: BecomeBot User-agent: Faxobot User-agent: Teleport Pro User-agent: WebZIP User-agent: Check&Get User-agent: LinkWalker User-agent: TurnitinBot User-agent: NPBot User-agent: http://www.almaden.ibm.com/cs/crawler User-agent: semanticdiscovery User-agent: SurveyBot Disallow: / User-agent: * Disallow: /novisit/ # TRAP Disallow: /aabbccd/ # TRAP #Disallow: /forum/ # infinite virtual URL #Disallow: /auc/ # infinite virtual URL #Disallow: /membership/ # member only #Disallow: /pm/ # member only #Disallow: /htmlmsg/ # including #Disallow: /errors/ # including #Disallow: /cgi-bin/ #Disallow: /archive/ #Disallow: /images/ # images only