User-agent: * Crawl-delay: 10 Disallow: /cp/ Disallow: /api User-agent: Googlebot Crawl-delay: 5 User-agent: Bingbot Crawl-delay: 20