Brief: A focused-crawler for web information extraction, etc
Jump To: Parent Description

  • Able to crawl websites, storing multiple versions over time like the wayback-machine. It is especially desired that this system be able to deliberate as it spiders, by using the nlu capabilities of FRDCSA. Ideally it should work similar to a web/unix softbot.