blob: 7f95d1f309e8a1da50f1bc3be7399734b823c8f4 (
plain) (
blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
|
Larbin is a powerful web crawler (also called [web] robot, spider...). It
is intended to fetch a large number of web pages to fill the database of a
search engine. With a network fast enough, Larbin is able to fetch more than
100 million pages on a standard PC.
Larbin was initially developed for the XYLEME project in the VERSO team at
INRIA. The goal of Larbin was to go and fetch XML pages on the web to fill
the database of an xml-oriented search engine.
The following can be done with Larbin:
o A crawler for a search engine
o A crawler for a specialized search enginer (xml, images, mp3...)
o Statistics on the web (about servers or page contents)
Larbin is created by: Sebastien Ailleret
WWW: http://larbin.sourceforge.net
WWW: http://www.sourceforge.net/projects/larbin
WWW: http://www.ailleret.com
|
