Documentation Menu↓
Troubleshooting
Troubleshooting Failed Audits
How to resolve crawler timeouts, sitemap loops, or memory caps.
Difficulty:
|Intermediate
Est: 9 mins
|Updated July 2026
Problem
Your website scan fails or hangs before completion, displaying a "Failed" or "Stuck" status in the Audits Dashboard.
Symptoms
- Audit progress percentage remains stuck at a specific number (e.g., 42%) for over 15 minutes.
- The scan card displays a red "Failed" badge with a "Site Unreachable" or "Timeout (504)" error message.
- Credits are deducted, but no optimization report is generated.
Possible Causes
| Error Code | Likely Cause | Recommended Action |
|---|---|---|
| CRAWL_TIMEOUT | Server response times exceed 30s timeout thresholds. | Switch from Full Audit to Quick Scan mode. |
| BOT_BLOCKED_403 | WAF (Cloudflare/Sucuri) blocks the crawler user-agent. | Whitelist the CrawlSageBot user-agent or IPs in firewall rules. |
| SITEMAP_LOOP | Circular references inside nested sitemap links. | Verify sitemap files index structure manually. |
Resolution
- Open the **Audits History** tab and click on the failed run to open the diagnostic logs panel.
- If the error log reports `BOT_BLOCKED_403`, go to your hosting control panel or CDN settings (like Cloudflare Rules).
- Add a firewall rule to whitelist requests matching:textUser-Agent contains CrawlSageBot
- If the run reports `CRAWL_TIMEOUT`, increase your server's memory allocation limits or configure your caching plugin to decrease TTFB speeds.
- Return to the CrawlSage dashboard and click **Retry Audit**.
Verification
Once resolved, the new audit run will complete within minutes, displaying a green **Success** badge and updating your health score metrics.
Contacting Support
If your audit remains stuck in the "Queued" state for over 30 minutes, it indicates a background task worker queue crash. Contact support with your Project ID to reset the queue.
Related Guides
Was this page helpful to your workflow?