How to Run a Website Scan

Overview

Running an audit directs the CrawlSage bots to scan your website sitemap, check headers metadata, test loading metrics, and isolate technical SEO issues. This guide details how to configure and execute your first scan.

Prerequisites

• A registered project slot with a verified domain address.
• Available usage credits in your dashboard account balance.

Step-by-Step Guide

1. Open the Audits Panel: Log in to your workspace. Click the "Audits" tab in the left sidebar navigation panel.
2. Configure Scan Scope: Choose the crawler parameters based on your goals:
   • Website Select: Choose the target project domain from the active selector list.
   • Audit Type:
     • Full Audit: Traverses the entire internal site hierarchy. Best for weekly baseline reports.
     • Quick Scan: Audits up to 50 key entrance URLs. Best for fast code deployments validation.
   • Page Limit: If running a Full Audit, declare the maximum pages limit bounds (e.g. 100 pages for Starter users; custom sizes for premium workspaces).
3. Launch Scan: Click the primary "Start Audit" button. The task is sent to the queue.
4. Monitor Progress: The page will load the execution console. Review the live logs stream (e.g., [crawler] Fetching sitemap.xml, [parser] Validating HTML on /about) to track progress.

Expected Result

Once completed, the logs view is replaced by the completed report. Your dashboard KPIs (Health Score, Issues Found, Pages Crawled) will instantly populate with updated parameters, and the recommendations grid will display generated tag suggestions.

Best Practices

• Schedule Weekly Baselines: Enable scheduled scans in project settings to run automatically during off-peak hours (e.g., Sunday at 02:00 UTC).
• Audit After Code Changes: Always run a Quick Scan immediately after updating your website templates, plugins, or redirects to ensure no rendering issues occurred.

Common Mistakes

• Exceeding Credit Quotas: Running large crawls (10,000+ pages) consumes credits relative to the number of pages successfully scanned. Ensure your balance has adequate credits before initiating deep audits.
• Double Scanning: Do not click "Start Audit" repeatedly if a scan is already running. Parallel crawls on the same domain are blocked to prevent overloading your host server.

Troubleshooting

• Audit remains "Queued" or "Failed":
  1. Verify that your site host is online and responsive.
  2. Check if your firewalls (Cloudflare, AWS WAF) are blocking requests from User-Agent: CrawlSageBot.
  3. Confirm your robots file has not blocked our crawler. See the Failed Audit Troubleshooting manual.