Crawl Waste Sampler
Sample potential crawl waste patterns.
Indexation and crawl
Last reviewed Nov 11, 2026
Free, no signup
Enter a URL to combine robots, meta robots, and canonical signals.
Results
on
Based on status, robots meta, and X-Robots-Tag signals.
Grade
Generated output
Markup
Apache
Nginx
UUIDs
API keys
NAP line
JSON-LD
Checklist
- •
Suggested queries
| Bot | Mentioned | Disallow / |
|---|---|---|
Heading outline
| Step | Status | URL |
|---|---|---|
| Header | Value |
|---|---|
Response headers
User-agent rules
Review
SERP preview
| Term | Count |
|---|---|
Page / post type
Top-level schema types
Rich-result candidates
All nested @types
Raw JSON-LD
Shared:
Only A:
Only B:
FAQ
Yes. Fair-use rate limits keep the suite available for everyone.
Yes. CDNs, registry redaction, and heuristics mean you should verify critical findings in your own environment.
Runs are processed to return results. Avoid pasting secrets or personal data you would not put in a public tool.