What Is Crawl Budget?

Crawl budget sounds like something only a search engineer could love, and honestly, for most sites it is. But if you run a large site, it is the difference between your important pages getting found or getting ignored.

The short version

Crawl budget is the number of pages a search engine is willing to crawl on your website within a given timeframe, shaped by your site's size, health, and authority.

Who actually needs to care

• Large sites with tens of thousands of pages

• Ecommerce sites with many filtered or duplicate URLs

• Sites with lots of low value or orphaned pages

How to use it well

Stop wasting crawls on junk. Block low value URLs, fix broken links and redirect chains, keep your sitemap clean, and make sure your best pages are easy to reach. Small sites can mostly relax here.

FAQ

Should a small website worry about crawl budget?

Usually no. If you have a few hundred pages, search engines can crawl you comfortably. It becomes a real concern at large scale.