Robots, Crawlers and Codes: The Critical Importance of Training in Technical SEO

Robots, Crawlers and Codes: The Critical Importance of Training in Technical SEO
By Molly Tyler

Marketing and web design using SEO is more important than ever. With the ubiquitous nature of the Internet through all devices, customers are increasingly ever present, and your business needs to keep this in mind when marketing and creating your website.


SEO (search engine optimization) will help you reach more customers more quickly by tailoring your website to fit the important search terms that engines like Google use. The good news is there are hundreds of tools out there for you to do so yourself, and there are even more professionals for you to find if you need help.

The bad news is that if you want to stay competitive, you need to know more than just how to plug in certain settings and use relevant keywords. You need to know exactly how search engines crawl the web to find your website and rank it among the millions of others.

How The Web Works

When a search engine wants to index websites it sends out robot crawlers, or “spiders,” to browse the World Wide Web and gather data. They then return to the server and report the results. This is where they get their rankings from. As you may know, the number one rule of the Internet is get on the first page, then get to the top. It should come as no surprise that the highest earning companies always rank highest in Google searches and naturally with the spiders.

As a business owner you want to control the information that those spiders seek. You want every page and piece of code to reflect what it is you do, where you are located, and the audience you are looking for. This is a sure fire way to move higher in the rankings, and it’s not that hard to learn how. Many eLearning websites offer courses on how spiders work, like at Market Motive.

Redirect The Robots

The first thing to know is that you can tell the spiders exactly what to do and what not to do. There are special “tags” that you can add to any part of your website and ask the search engines to follow or not follow certain links. Why would you want to do this? Because you then have some say in what the search engines know about your business. So why not make it all good things? Have the follow tag on posts and pages you want search engines to know about and the no follow tag on those you don’t.

Typical links that are assigned as no follow are posts to forums, paid links and “untrusted content.” These are typically places where spam and other annoyances grow, so assigning the nofollow tag makes sense. Of course, you want the engines to follow links you approve of and you think represent your business well. And that’s great, because we all know…

Content Is Still King

This is the oldest website mantra in the world: Content is king. Always has been, always will be. Websites that constantly turn out great original content that represents their business will always be, eventually, at the top of the rankings. Hopefully, if you have a website, you already know all of this. Hopefully you’re already turning out great content on a regular basis. But did you know there’s content in the code as well?

Have you ever dug into the coding of your website? If you designed it yourself, maybe you have. If not, then chances are the code remains a mystery. But within that code lies many opportunities. There are places that would be perfect for your business name and your location. Or a chance to make your keywords visible one more time. Never miss a chance to improve your standing in the rankings.

Speed Is Good

When customers visit your site, you don’t want them to get frustrated with slow performance. Three seconds seems like a lifetime to a visitor and they are more likely to leave. Learning the technical aspects of SEO will help your site run optimally, helping your sales and keeping your customers happy. Not only does speed help your customers, it helps your rankings as well. Google recently confirmed that website speed is a ranking factor. So the faster your site works, the better everything will be.

Knowing the SEO essentials is important for any business owner. When you keep churning out content and making an impression on the spiders, you should have no problem climbing higher in the rankings. But don’t forget the technical aspects as well.

Molly Tyler is a geeky girl. She is a coder who also freelances, building websites and doing SEO tasks outside of her day job to earn extra cash (she’s saving for a deposit on her first home).