Abot by Steven Jones

<PackageReference Include="Abot" Version="1.5.0.2" />

 Abot 1.5.0.2

Abot is an open source C# web crawler built for speed and flexibility. It takes care of the low level plumbing (multithreading, http requests, scheduling, link parsing, etc..). You just register for events to process the page data. You can also plugin your own implementations of core interfaces to take complete control over the crawl process.

<package xmlns="http://schemas.microsoft.com/packaging/2011/08/nuspec.xsd">
  <metadata>
    <id>Abot</id>
    <version>1.5.0.2</version>
    <title>Abot Web Crawler</title>
    <authors>Steven Jones</authors>
    <owners>Steven Jones</owners>
    <licenseUrl>http://www.apache.org/licenses/LICENSE-2.0</licenseUrl>
    <projectUrl>https://code.google.com/p/abot/</projectUrl>
    <iconUrl>https://code.google.com/p/abot/logo?cct=1364531651</iconUrl>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <description>Abot is an open source C# web crawler built for speed and flexibility. It takes care of the low level plumbing (multithreading, http requests, scheduling, link parsing, etc..). You just register for events to process the page data. You can also plugin your own implementations of core interfaces to take complete control over the crawl process.</description>
    <summary>Open source C# web crawler built for speed, flexibility and extensibility.</summary>
    <releaseNotes>1.5.0.0 release notes may be found on https://code.google.com/p/abot/</releaseNotes>
    <copyright>Copyright 2013</copyright>
    <tags>crawler robot spider</tags>
    <dependencies>
      <dependency id="AutoMapper" version="3.2.1" />
      <dependency id="CsQuery" version="1.3.4" />
      <dependency id="log4net" version="2.0.3" />
    </dependencies>
  </metadata>
</package>