User-agent: LinkChecker
Crawl-delay: 1
Allow: /