Crawler918, you suck.
We blocked all access from crawler1.crawler918.com to the www.ecyrd.com site. They don't respect /robots.txt, so we don't respect their efforts (whatever they are - they have no web site, nor an URL in the referrer logs we could check). Checking the robots.txt would be polite, you know...
|
Comments?
Back to weblog
|
| Published by Janne Jalkanen. |
Comments
No comments yet.