About Website Crawler
A website may have several pages. Analyzing the meta/heading tags, content length, page speed, links, etc on each page on a website manually is a waste of time. Also, it is a frustrating task.
Website Crawler is a cloud based website analyzer, scraper, data extractor that saves your time. Once WC crawls your website, you can quickly find out the pages that are having SEO issues or download the website data in JSON or CSV format.
How to use Website Crawler?
To use WebsiteCrawler, enter the domain name and the number of URLs you want WC to analyze. Then, click on the submit button. If the domain/url you have entered does not redirect or is not blocked in the country where websitecrawler.org is hosted, or the site is not protected by ant-bot tools, or doesn't generate links, HTML tags or adds attributes to the URLs through JavaScript, Website Crawler will begin crawling your site. Also, you'll see a new button with the label "Status" at the right side of the "Submit" button.
When you click the "Status" button, you'll see the crawl progress and and the list of URLs that have been crawled.
Note: The status button will be displayed when the domain name and the crawl limit is valid. The user will see the list of crawled URLs in real-time. WebsiteCrawler.org doesn't follow/crawls redirected URLs in the pages to make users aware of the redirect.
Once Website Crawler finishes crawling your site, it will display a form that asks you to enter the email address or log in with Google. Enter your email ID and then click the submit button or use the sign in with Google option to see your reports. Website Crawler will now send a three digit verification code to your inbox. Enter this number in the new text box displayed in the form and hit the "Verify" button.
Website Crawler will now attempt to create an account for you. Once our account is ready, you'll see a message to log in to your account. Log in to see the SEO reports, download the data, see charts, etc.
Refund Policy
Users who want their money refunded should write to us within 24 hours of buying any of the Website Crawler Plans . Don't ask for a refund if you have been using our product for more than 24 hours.
Support
We provide email support to the users of the basic/free and silver plan on weekdays and weekends. To get support or if you've some questions, don't hesitate to write to us. Our email address is: help@websitecrawler.org
About the founder
Website Crawler has been built by
Pramod Choudhary.
Pramod is web application developer (Full Stack) and a blogger.
He has been building applications since 2010. What made Pramod build WebsiteCrawler? Pramod was analyzing one of his websites that had 100s of pages. Frustrated of noting down issues in file, one day he decided why not use his technical expertise to automate this task as this will save time. This SaaS is the result of this idea.
Once WebsiteCrawler was built, Pramod started testing it. While testing, PC found that this SaaS did what it had been designed for. The reports of this SaaS made PC aware of issues he wasn't himself aware of. He then thought why not make this SaaS available to everyone? He launched WebsiteCrawler.org in 2019. Over the 6 years, PC spent a great deal of time making the data available in the reports accurate. Try WebsiteCrawler! It will reveal dozens of issues on your site.