How to Get Your Content Indexed with @TomekRudzki #vcbuzz

One of the SEO concepts that cause constant confusion is indexing.

What does it mean “indexed by Google” and what does it mean to your SEO strategy?

Let’s discuss!

***Add #VCBuzz chats to your calendar here.

***Please sign in here to follow the chat -> twchat.com/hashtag/vcbuzz

About @TomekRudzki

TomekRudzki is author of The SEO Playbook and Technical SEO @OnelyCom

He is also speaker at @BrightonSEO and @SMX.

Questions we discussed

Q1 How did you become a digital marketer? Please share your career story!

When I was in high school, I created a few websites with my friends. We wanted to rank Google. But at this time, I didn’t enjoy SEO that much… I entered the SEO field again when SEO was a much safer place – after Panda and Penguin algorithms 🙂

A1) Started playing online in 1993, did a newsletter in 1998, did LinkedIn in 2005?/ then Twitter Facebook etc. #VCBuzz https://t.co/JKBYT4qg5k
— Ivana Taylor (@DIYMarketers) August 23, 2022

Q2 What is indexing and why should we care?

You cannot rank your content if it’s not indexed in Google. That’s why we all should care about indexing.

In the case of popular ecommerce stores, 26% of their pages aren’t indexed in Google. They lose lots of opportunities of reaching new consumers. Here is where indexing SEO helps.

A2: Agreeing w/ @TomekRudzki –
Understand differences between:
?”Crawled” vs indexed
?”Discovered” vs currently not indexed
? How JS rendering affects Google Indexing
?Why getting a website crawled & indexed by Google in 2022 is far more complicated than 5-6yrs ago #vcbuzz https://t.co/JUcMARVZVf
— Jeannie Hill (@essentialskill) August 23, 2022

I believe that with the latest news about dynamic rendering not being a long-time solution people will talk more about How JS rendering affects Google Indexing. This topic needs consideration.

Q3 How to get your content indexed by Google and should we try and make the process faster?

There are two basic rules of indexing:

your server should be fast enough to handle Google requests (in 2022 it’s not that common!) without any problems.
your content should be helpful so Google will likely index it. That’s the short answer.

A3: Make sure your site is *ready* to be indexed. Page titles and descriptions. Alt tags on key images. Site map (often automatically generated). No broken links.

Then yes, use Google Search Console to inspect and submit – once.#vcbuzz
— Don Dingee (@don_dingee) August 23, 2022

There is a lot to discuss about content quality and helpfulness. Just a small hint: at @ziptiedev we noticed a page is very likely to be unindexed if headers aren’t relevant to the page title and content. Semantic SEO FTW 🙂

Wait..what?? It isn’t common for servers to be fast enough to handle Google requests?

Oh, that’s not good. Any tips for determining whether yours is?

Or how to find hosting that is fast enough? #vcbuzz https://t.co/gTc9ZHvg1Q
— Gail Gardner (@GrowMap) August 23, 2022

I use two tools to check if a server can handle Googlebot requests: 1) http://K6.io – stress testing 2) Any crawler – if a page cannot handle 10 requests per second that almost certainly means issues with the crawl budget. Hint: use JS mode.

I love checking Google Search Console data – crawl statistics and number of pages classified as Discovered – currently not indexed. The high number of valuable pages reported may indicate server health issues.

If you want to crawl A PAGE fast: – Add internal links to it – Add it to a sitemap – Request it in Google Search Console. A queue is still a queue but you will increase your chances.

Install Google Search Console.

Not only does it have the tools to spot problems with a crawl, it tells Google you’re serious about things.

Easy install from Google Tag Manager.#vcbuzz
— Don Dingee (@don_dingee) August 23, 2022

Google has got a priority queue – you need to convince Google to visit your website. Look for articles about crawl budget optimization to let Google crawl your website.

#VCBuzz A3.2

G does not want 5,000 copies of a wiki page about penguins, or 300 copies of a SO page about JQuery smooth scroll on 300 sites!

Then there's the whole "crawl budget" issue.
You have to "prioritise" the content,
making sure G can see what content is important.

>>>
— Lyndon NA (Darth Autocrat) (@darth_na) August 23, 2022

#VCBuzz A3.4

The more popular a URL is, the higher the priority G see's it as having.
Links from popular pages (internal/inbound),
that get crawled frequently also tend to improve things.

Sitemaps can be handy – esp. if you split them and have a dedicated "new content" sitemap.
— Lyndon NA (Darth Autocrat) (@darth_na) August 23, 2022

Q4 What are some common indexing myths to leave behind and mistakes to avoid?

I believe the most common myth is that Google has no problems with indexing. Data from our clients and complaints from website owners tell a different story. If you want to check for indexing issues check Google Search Console data.

A4: start with the easy one … what’s in the index? In your browser, type:

site:siteyouwannacheck.tld

That’s it. Shows the pages that are in the index.#vcbuzz
— Don Dingee (@don_dingee) August 23, 2022

Another common myth is that JavaScript rendering doesn’t affect indexing. Rendering counts toward crawl budget – if rendering consumes your all crawl budget, many pages will remain unindexed. Check for “Discovered currently not indexed” in GSC.

Another myth: there is ALWAYS AN EASY fix for indexing. To fully fix indexing issues you need to dig deeper, look at the content quality, and the health of the server, and find crawl budget issues. Sometimes you need to fix an engine to run a car.

When it comes to things to avoid: 1) avoid infinite scrolling 2) make sure you don’t make obvious mistakes (block in robots.txt, canonicalization of all pages to a single page or noindex on a page.

There is also a mistake that is not getting lots of attention – when Google is getting a totally different version of your pages and you don’t have control over it. It may happen in dynamic rendering or when your mobile version is totally different than desktop).

Q5 What are your favorite SEO tools?

My favorite SEO tools: 1. Google Search Console (Ofc), 2. SEO crawler 3. Analysing Google SERPs to find patterns (it’s free yet undervalued). 4) A shameless plug: http://ziptie.dev (a tool helping with indexing issues). 5. SEO Minion Chrome plugin