However, to see success on the search engines, you must know exactly how to check whether your site’s pages can be rendered and indexed, identify issues, and make it search engine friendly.
You can see this process visualized in more detail below:
Image credit: Google
Let’s look at this process in a little more depth, comparing it to how Googlebot crawls an HTML page.
It’s a quick and simple process that starts with an HTML file being downloaded, links extracted, and CSS files downloaded before these resources are sent to Caffeine, Google’s indexer. Caffeine then indexes the page.
And really, it comes down to three things:
- Making sure Google can crawl your website’s content
- Making sure Google can render your website’s content
- Making sure Google can index your website’s content
Let’s take a look at what these are.
Ensure That Google Can Render Your Web Pages Using Google Search Console
While Googlebot is based on Chrome’s newest version, it doesn’t behave in the same way as a browser. That means opening up your site in this isn’t a guarantee that your website’s content can be rendered.
You can use the URL Inspection Tool in Google Search Console to check that Google can render your webpages.
Enter the URL of a page that you want to test and look for the ‘TEST LIVE URL’ button at the top right of your screen.
After a minute or two, you’ll see a ‘live test’ tab appear, and when you click ‘view tested page,’ you’ll see a screenshot of the page that shows how Google renders it. You can also view the rendered code within the HTML tab.
Add the following code to this file to ensure that no crucial resources are blocked from being crawled:
But let’s clear one thing up; Google doesn’t index .js or .css files in the search results. These resources are used to render a webpage.
There’s no reason to block crucial resources, and doing so can prevent your content from being rendered and, in turn, from being indexed.
If you’ve confirmed that your web page is rendering properly, you need to determine whether or not it’s being indexed.
And you can check this through Google Search Console as well as straight on the search engine.
Head to Google and use the site: command to see whether your web page is in the index. As an example, replace yourdomain.com below with the URL of the page you want to test:
If the page is in Google’s index, you’ll see the page showing as a returned result:
If you don’t see the URL, this means that the page isn’t in the index.
Again, use the site: command and include a snippet of content alongside this. For example:
site:yourdomain.com/page-URL/ "snippet of JS content"
Here, you’re checking whether this content has been indexed, and if it is, you’ll see this text within the snippet.
This time, rather than testing the live URL, click the ‘view crawled page’ button and view the indexed page’s HTML source code.
- The content cannot be rendered in the first instance
- The page times out while Google is indexing the content
- Google determined that the JS resources do not change the page enough to warrant being downloaded
We’ll look at the solutions to some of these commonly seen problems below.
Server-Side Rendering vs. Client-Side Rendering vs. Dynamic Rendering
So what are these different types of rendering, and what do they mean?
According to Free Code Camp, here’s how SSR works: “Whenever you visit a website, your browser makes a request to the server that contains the contents of the website. Once the request is done processing, your browser gets back the fully rendered HTML and displays it on the screen.”
The problem here is that SSR can be complex and challenging for developers. Still, tools such as Gatsby and Next.JS (for the React framework), Angular Universal (for the Angular framework), or Nuxt.js (for the Vue.js framework) exist to help to implement this.
When you understand how CSR works, it becomes easier to see why SEO issues can occur.
This is something that was introduced by Google’s John Mueller at Google I/O 2018:
Image Credit: Google
To clear up a question that many SEOs will likely have: dynamic rendering is not seen as cloaking as long as the content that’s served is similar. The only time when this would be considered cloaking is if entirely different content was served. With dynamic rendering, the content that users and search engines see will be the same, potentially just with a different level of interactivity.
You can learn more about how to set up dynamic rendering here.
- Blocking .js files in your robots.txt file can prevent Googlebot from crawling these resources and, therefore, rendering and indexing these. Allow these files to be crawled to avoid issues caused by this.
- Setting up pagination where links to pages beyond the first (let’s say on an eCommerce category) are only generated with an on click event (clicks) will result in these subsequent pages not being crawled as search engines do not click buttons. Always be sure to use static links to help Googlebot discover your site’s pages.
- Make sure that static URLs are generated for your site’s web pages, rather than using #. That’s ensuring your URLs look like this (yourdomain.com/web-page) and not like this (yourdomain.com/#/web-page) or this (yourdomain.com#web-page). Use static URLs. Otherwise, these pages will not be indexed as Google typically ignores hashes.