Logfile generated by web-servers such as Nginx, Apache, etc contains information that the widely used Analytics tools i.e. Google Analytics won’t display. For example, Google Analytics won’t display the IP address of the bot or the user. Log files contain a timestamp, HTTP protocol, request type, URL, status code, IP address, etc. Analyzing the log […]
As you might know, Website Crawler – On Page SEO Checker is capable of extracting URLs from sitemap and processing them. From now on, you can now create XML sitemaps for any website with WC. For those who don’t know, a sitemap is a file that contains all links the search bots should crawl and […]
“rel” is one of the attributes supported by hyperlinks. Before Google introduced the “sponsored” value for the “rel” attribute this month, nofollow and dofollow were the commonly used values. Links using the rel=”nofollow”/”dofollow” attributes are widely known as nofollow or dofollow URLs. DF links pass page rank to linked documents. Searchbots follow these URLs. NF […]
Thin content is one of the major reasons why websites suffer from Google Panda algorithm penalty. For those who are not aware, Google’s Panda algorithm has been designed to lower the rankings of websites that have On-Page SEO issues. Thin content is one of several On-Page search engine optimization issues. The Panda algorithm is now […]
A sitemap, as you may already know, is the most important part of a website. It contains a list of links of a website and helps search engines in crawling/indexing pages which it may not find it. Today, I have good news for the users of Website Crawler. WC can now crawl the links it finds […]