Skip Menu |
 

Preferred bug tracker

Please visit the preferred bug tracker to report your issue.

This queue is for tickets about the WWW-Mechanize CPAN distribution.

Report information
The Basics
Id: 12960
Status: rejected
Priority: 0/
Queue: WWW-Mechanize

People
Owner: Nobody in particular
Requestors: mark [...] summersault.com
Cc:
AdminCc:

Bug Information
Severity: Wishlist
Broken in: 1.12
Fixed in: (no value)



Subject: wish: respect robots.txt by default
Download (untitled) / with headers
text/plain 237b
Let's be a good netizen and support robots.txt by default (or at least as an option!). jeffa has already written the code for this. It's listed here with more details: http://sourceforge.net/mailarchive/message.php?msg_id=7171956 Mark
It's not Mech's job to respect robots.txt.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.