HarvestMan HarvestMan is a full-featured, multi-threaded Web crawler and offline browser.
Download

HarvestMan Ranking & Summary

HarvestMan Tags

web crawler web crawler offline browser multi-threaded web crawler HarvestMan

HarvestMan Description

HarvestMan is a full-featured, multi-threaded Web crawler and offline browser. HarvestMan is a multithreaded off-line browser.It has many features for customizing offline browsing through URL filters, word filters, domain filters, URL priorities, depth-fetching, fetch levels, file limits, time limits, robot exclusion protocols, and many more.It is useful to download an entire Web site or certain files from a Web site to the hard disk for offline browsing later. It supports HTTP/HTTPS and FTP protocols and can work across proxies.What's New in This Release:· Fixed bugs in the setup.py and install scripts so that they work with Python 2.4.

HarvestMan Related Software