Lycos
: Hunting WWW Information
Features
Uses random search to walk the Web documents
Acccesses HTTP, FTP, and GOPHER documents
Uses the
SCOUT indexer
to do information retrieval
Written in Perl; uses C and libwww to fetch URLs.
Keeps a
summary of each URL
(here is a
sample document summary
).
Findings
Tracks
word frequencies
as it searches. word frequency
Distribution of
document types
found mostly http documents.
These URLs had the
most references
in the documents searched.