rubygem-webrobots helps writing robots.txt compliant web robots in Ruby.

WWW: https://github.com/knu/webrobots
RG:  https://rubygems.org/gems/webrobots