The most common action that a crawler takes is to follow links on a page. This module helps you with that task. WWW: http://search.cpan.org/dist/GunghoX-FollowLinks/