Crawl Budget
The number of pages a search engine will crawl on your site within a given period.
Key takeaways
- Crawl Budget is the number of pages a search engine will crawl on your site within a given period.
- Check the Crawl Stats report in Search Console to see what Googlebot actually requests, and how often.
- Avoid: worrying about crawl budget on a 50-page site, where it almost never matters.
What is Crawl Budget?
Crawl budget is the combination of crawl rate limit and crawl demand. Large sites and sites with many low value URLs are most affected. Improve it by removing thin pages, fixing redirect chains, and ensuring fast server response.
Google explains when crawl budget actually matters in its guide to managing crawl budget for large sites.
Why Crawl Budget matters for SEO in 2026
Most small sites never hit a crawl budget limit, but large or messy ones do. When Google spends its visits on faceted URLs, redirect chains and thin pages, new and updated content waits longer to be found, and rankings lag behind the work you've done.
For a wider view of where this fits, see our full SEO glossary or explore the SEO and GEO service we run for clients.
How to apply Crawl Budget on your site
Knowing the concept is only half of it. Work through these steps in order to put Crawl Budget into practice on your own pages:
- Check the Crawl Stats report in Search Console to see what Googlebot actually requests, and how often.
- Remove or noindex low-value URL patterns such as filters, internal search results and tracking parameters.
- Keep your sitemap limited to canonical, indexable pages so crawl effort goes where it counts.
Common mistakes to avoid
Most sites that struggle with Crawl Budget fall into the same traps. Check your pages against each of these before moving on:
- Worrying about crawl budget on a 50-page site, where it almost never matters.
- Blocking URLs in robots.txt that you also want removed from the index; Google can't see a noindex it isn't allowed to crawl.
- Leaving long redirect chains in place after a migration.
Questions about Crawl Budget
The number of pages a search engine will crawl on your site within a given period.
Most small sites never hit a crawl budget limit, but large or messy ones do. When Google spends its visits on faceted URLs, redirect chains and thin pages, new and updated content waits longer to be found, and rankings lag behind the work you've done.
Start here: check the Crawl Stats report in Search Console to see what Googlebot actually requests, and how often. Then remove or noindex low-value URL patterns such as filters, internal search results and tracking parameters.
The most common is worrying about crawl budget on a 50-page site, where it almost never matters. Another is blocking URLs in robots.txt that you also want removed from the index; Google can't see a noindex it isn't allowed to crawl.