Browse by category
Frequently asked questions
On the homepage, there's a form to enter website URL and crawl limit. Fill these two fields and click the crawl my site now button.
You can find all our subscription plans on the pricing page. You can purchase a plan only after signing in. Log in to your account, visit the pricing page and click the upgrade button displayed below the plan name.
WebsiteCrawler has the capability to stop an ongoing crawl and saving the data which it had fetched while crawling. When a site is being crawled, you'll see a button "Status" in the dashboard next to the project name. Click this button to open the crawl status which is displayed below the list of projects. In the "status" section, you'll find a button "stop crawling". Click this button.
Go to Settings → API Integrations, Find the Google Search Console section and click "Connect to GSC" button and follow the instructions displayed on the screen.
Click "Forgot password" on the login page and enter your email. Now, click the submit button. WebsiteCrawler will set a new password for your account and send the same to your registered email address.If our email adoesn't arrive, please check your spam folder.
Users of our free plan can work on a single project only. If you've purchased a plan, you can add new websites to monitor/crawl. Visit the dashboard page and click the "create a new project" button. Add a new website URL and click the "Add Website" button. Once a new project is created successfully, visit the homepage to run the crawler by entering the same website URL and the number of URLs to crawl and clicking the "Crawl my site" button.
Generate an API key from Settings page → API. Please refer the WebsiteCrawler API documentation for further steps.