Bingbot in robots.txt
Bingbot is Microsoft's crawler. Here's exactly what it does, whether it respects robots.txt, and the rules to control it. To apply a policy in one click, use the AI Crawler Manager.
What Bingbot is
Bingbot is Microsoft's search crawler. It powers Bing Search and, by extension, the web results in Microsoft Copilot and other Bing-backed products. For most sites it is the second-most important crawler after Googlebot.
| Property | Value |
|---|---|
| User-agent | Bingbot |
| Operator | Microsoft |
| Category | Search engines |
| Honors robots.txt | Yes |
| Affects search ranking | Yes |
Official documentation: Microsoft crawler docs.
What Bingbot does
- Crawls and indexes pages for Bing Search.
- Feeds web results into Copilot and Bing-powered partners (e.g. DuckDuckGo, Yahoo).
- Supports the IndexNow protocol for instant change notifications.
Why site owners care
- Bing drives a meaningful share of desktop and enterprise search traffic.
- Bing results increasingly surface inside AI assistants, so Bing visibility now also affects AI answer presence.
- Blocking Bingbot removes you from Bing and every product that uses Bing's index.
How to allow or block Bingbot
Add a group targeting the Bingbot user-agent. Disallow: / blocks it from your whole site; an empty Disallow: allows it.
User-agent: Bingbot
Disallow: /User-agent: Bingbot
Disallow:This crawler affects your search ranking
How to verify Bingbot
Verify via reverse DNS (genuine Bingbot resolves to a search.msn.com host and forward-confirms) or Bing's published IP list / the Bing Verify Bingbot tool.
Does it honor robots.txt?
Recommendation
Recommended: Allow
Does Bingbot support crawl-delay?
Yes. Unlike Googlebot, Bingbot honors the Crawl-delay directive in robots.txt, so you can throttle its request rate. You can also set the crawl rate in Bing Webmaster Tools.
Does blocking Bingbot affect Google?
No. Bingbot and Googlebot are separate crawlers from different companies. Blocking one has no effect on the other.
Robots.txt Validator
Catch syntax errors and best-practice issues, with a health score.
Robots.txt Studio Editorial · Technical SEO & crawling
We build robots.txt tooling and parse thousands of real-world files. Guides are written by practitioners and reviewed against the Google and RFC 9309 specifications.