LWP::RobotUA LWP::RobotUA is a Perl class for well-behaved Web robots.
Download

LWP::RobotUA Ranking & Summary

LWP::RobotUA Tags

web Perl module Robots web robots Perl class LWP

LWP::RobotUA Description

LWP::RobotUA is a Perl class for well-behaved Web robots. LWP::RobotUA is a Perl class for well-behaved Web robots.SYNOPSIS use LWP::RobotUA; my $ua = LWP::RobotUA->new('my-robot/0.1', 'me@foo.com'); $ua->delay(10); # be very nice -- max one hit every ten minutes! ... # Then just use it just like a normal LWP::UserAgent: my $response = $ua->get('http://whatever.int/...'); ...This class implements a user agent that is suitable for robot applications. Robots should be nice to the servers they visit. They should consult the /robots.txt file to ensure that they are welcomed and they should not make requests too frequently.But before you consider writing a robot, take a look at http://www.robotstxt.org.When you use a LWP::RobotUA object as your user agent, then you do not really have to think about these things yourself; robots.txt files are automatically consulted and obeyed, the server isn't queried too rapidly, and so on. Just send requests as you do when you are using a normal LWP::UserAgent object (using $ua->get(...), $ua->head(...), $ua->request(...), etc.), and this special agent will make sure you are nice. Requirements: · Perl

LWP::RobotUA Related Software