So, you are conducting an SEO audit. Maybe you were inspired by the article we posted on SEO audits a couple of days ago.
You did your research. You found some checklists and made a list of things you want to check to get started. Your goal is to get a basic SEO health checkup for your client’s website.
So, you started accumulated a list of tools you need to get started. Does that list include a website crawler?
Well, today we are going to take a look at a few website crawlers to get on your way.
Screaming Frog is almost everyone’s favorite choice for a web crawler. It allows you to do a ton of things. While you can download the crawler for free, you are able to crawl an unlimited number of pages for £149.00 per year.
There are many other things included in the premium, but if you are working with smaller sites, the free is more than enough. You are limited to crawling 500 URLs.
The tool is a desktop application. It is actually called The Screaming Frog SEO Spider. Screaming Frog is the name of the company behind the tool and they offer other various digital marketing services.
If you want to see more about how to use the spider, Seer Interactive has created a fantastic guide you can see here:
Another guide for learning Screaming Frog is this one at Big Leap:
Vary is a very good website crawler that provides basic information about your page including the SEO, the speed, and the mobile SEO. The interface is really nice and clean.
See more tools: https://varvy.com/toolbox/
I learned about Vary from John Locke. Vary is a great tool to provide a great starting point for your audits.
Originally built and owned by Patrick Sexton, there are several tools on the site. In the past couple of years, it was acquired by the Internet Marketing Ninjas. IMN has its own suite of tools, so this is another in a really nice portfolio of SEO tools.
According to a representative at IMN, they haven’t touched the site since it was acquired.
Internet Marketing Ninjas
The longtime SEO company has several tools available for free in addition to owning Vary (see above). Taking advantage of these tools allow you to find many of the apps you need in the same location.
Beam Us Up
This one had potential. Many recommended this tool as an alternative to Screaming Frog. It is also a desktop based app. Unfortunately, according to their Twitter account, the tool has been discontinued.
To confirm, we do not offer support for our tool. It is completely discontinued.
— Beam Us Up (@BeamUsUpCrawler) January 30, 2018
It looks like you can still download the tool as the site is still online, but will have no support. I decided not to download the app because of that and the fact that the browser lists it as “Not Secure.”
SEO Crawler by Ron Hammond
SEO Site Check
SEO Tools Resources
Wrapping it up
There are lots of tools available for SEO. We just mentioned a few of the crawler-based tools. I believe these tools are best if you can compare the results to other tools and verify the information based on your own testing.
What SEO tools do you like to use?