The first step after creating a blog or a website is to submit it to Google so that your page can be discovered by your potential readers.
But, what if your page is not visited by Googlebots
or
What if it is never shown in the Google search results.
It’s scary. Right!
Well, read this guide to make sure how you guide GoogleBots to crawl your site effectively so that you are not left unmapped by Google.
Before, we start, let’s get familiar with two words: Crawling and Indexing
Crawling is the process by which Googlebot discovers new and updated pages to be added to the Google index. When a Googlebot visits your website it detects links on each page and adds them to the list of pages to crawl. To learn more about it, read the below guide:
- What is Google crawling and indexing
If your site has got a very authority your pages will get indexed by Google within few minutes. Below are some useful tips to get your pages indexed by Google effectively.
How to make Search engine bots crawl your site effectively:
1. Create and submitting a Sitemap to Google:
Even if your site is already indexed in Google, submitting a sitemap will give Google a better way to understand about your site.
If you are using WordPress use any of the plugins (Google XML sitemap)that can create a sitemap for you. Also, place your sitemap link in your robots.txt and in your sitemap file, as it will help Google bots to crawl your website effectively.
2. Register for Google Webmasters Tools and check the crawl error page frequently:
The crawl errors page provides details about the URLs in your site that Google couldn’t access. So you can fix the errors once you know the problem area. Harsh has covered a detailed article on understanding Google WebMaster tools crawl errors, and using it, you can find why bots are facing issue with crawling of your website.
3. Robots.txt:
Make sure that the required URL’s are not blocked by robots.txt file or meta tags. If you are not sure about robots.txt file use the Webmasters Tools to create a robots.txt for your website/blog. Also, block bots access to those part of Website, which is not important for search engines. Hint: your admin folder.
- Robots.txt stops search engine bots to crawl part of your blog
- Optimize WordPress Robots.txt file for SEO
4. Image and video sitemap:
Search engines cannot understand images and videos. So if your website has a good number of images or videos try to provide relevant information about them. For images, you need to add ALT tag, which could easily by done with the help of “SEO friendly images” plugin. For videos, you can create a video sitemap and submit it to Google.
- Read: Useful tips to increase Google bot crawl rate
Other articles which will help you to crawl your website effectively are :
Link building ,Anchor text technique, Blog post roundup
These are just some of the tips which will help you to get your site crawled by Google effectively. If you have any other tips then do share them with us.
The first step after creating a blog or a website is to submit it to Google so that your page can be discovered by your potential readers.
But, what if your page is not visited by Googlebots
or
What if it is never shown in the Google search results.
It’s scary. Right!
Well, read this guide to make sure how you guide GoogleBots to crawl your site effectively so that you are not left unmapped by Google.
Before, we start, let’s get familiar with two words: Crawling and Indexing
Crawling is the process by which Googlebot discovers new and updated pages to be added to the Google index. When a Googlebot visits your website it detects links on each page and adds them to the list of pages to crawl. To learn more about it, read the below guide:
If your site has got a very authority your pages will get indexed by Google within few minutes. Below are some useful tips to get your pages indexed by Google effectively.
How to make Search engine bots crawl your site effectively:
1. Create and submitting a Sitemap to Google:
Even if your site is already indexed in Google, submitting a sitemap will give Google a better way to understand about your site.
If you are using WordPress use any of the plugins (Google XML sitemap)that can create a sitemap for you. Also, place your sitemap link in your robots.txt and in your sitemap file, as it will help Google bots to crawl your website effectively.
2. Register for Google Webmasters Tools and check the crawl error page frequently:
The crawl errors page provides details about the URLs in your site that Google couldn’t access. So you can fix the errors once you know the problem area. Harsh has covered a detailed article on understanding Google WebMaster tools crawl errors, and using it, you can find why bots are facing issue with crawling of your website.
3. Robots.txt:
Make sure that the required URL’s are not blocked by robots.txt file or meta tags. If you are not sure about robots.txt file use the Webmasters Tools to create a robots.txt for your website/blog. Also, block bots access to those part of Website, which is not important for search engines. Hint: your admin folder.
4. Image and video sitemap:
Search engines cannot understand images and videos. So if your website has a good number of images or videos try to provide relevant information about them. For images, you need to add ALT tag, which could easily by done with the help of “SEO friendly images” plugin. For videos, you can create a video sitemap and submit it to Google.
Other articles which will help you to crawl your website effectively are :
Link building ,Anchor text technique, Blog post roundup
These are just some of the tips which will help you to get your site crawled by Google effectively. If you have any other tips then do share them with us.
Subscribe on Youtube
Like this:
Support