Spindle is a web indexing/search tool built on top of the
Lucene toolkit. It includes a HTTP spider that is used to build the index, and a search class that is used to search the index. In addition, support is provided for the Bitmechanic
listlib JSP TagLib, so that a search can be added to a JSP based site without writing any Java classes.
See Other Best Web Crawlers Projects in Java:
6