Spider is a complete standalone Java application designed to easily integrate varied datasources.
- XML driven framework for data retrieval from network accessible sources
- Scheduled pulling
- Highly extensible
- Provides hooks for custom post-processing and configuration
- Implemented as a Avalon/Keel framework datafeed service
Included Core Connectors:
- Files and Zip Archives via HTTP/FTP/HTTPS/FileSystem
- Supports access via links described as literals or regular expressions
- Supports sessions/cookies/form parameters
Included Optional Connectors:
