You paid for a new website. It looks good, it works on a phone, and months later the phone still isn't ringing from Google. Search your company name and you find your Facebook page, a directory listing and a competitor, but not your site.
Often that isn't a ranking problem. The site is telling Google to stay away, usually because of one setting that made sense during the build and never got switched off. Here are the common ones and how to check each.
Is my site blocked, or just not ranking?
Start with a quick test: type site:yourdomain.com into Google, with your real domain. It lists pages Google has indexed for that domain. It's a rough view, but it tells you which problem you have:
- Nothing comes back, or only a page or two. Something is probably keeping your site out of Google. Keep reading.
- Your pages come back, but you don't show up for your services. Google can see you; it just isn't ranking you yet. That's a different job, covered in how long local SEO takes.
For a real answer, use Google Search Console. It's free, and it's where Google tells you what it thinks of each page.
What's the difference between "blocked from crawling" and "not indexed"?
Crawling is Google's bot visiting a page and reading it. Indexing is Google storing that page so it can show it in results. Different switches control each one:
- robots.txt controls crawling. It's a file at
yourdomain.com/robots.txtthat tells bots which parts of the site not to visit. - noindex controls indexing. It's a robots meta tag in the page's code, or an instruction sent by the server, that says "don't show this page in results."
That leads to two common mistakes. First, blocking a page in robots.txt doesn't remove it from Google. If other pages link to it, Google can still index the address without ever reading the page. It may show up in results with no description, just a note that no information is available.
Second, noindex only works if Google can crawl the page. Google has to visit the page to see the tag. If you add noindex and also block the page in robots.txt, Google never sees the noindex, and the page can stay in the index. To take a page out of Google, let it be crawled and use noindex, or remove the page entirely.
How do I use Search Console to find the problem?
Set up Search Console and verify your site. If you can, choose a Domain property, which covers every version of your site at once: with and without www, http and https. Then use two tools.
URL Inspection
Paste any address from your site into the search bar at the top of Search Console. It tells you whether that URL is on Google, and if not, why. You'll see messages like "Excluded by 'noindex' tag", "Blocked by robots.txt", "Page with redirect" or "Duplicate, Google chose different canonical than user".
Click Test live URL to have Google fetch the page right now, which is how you confirm a fix. Request indexing asks Google to recrawl it.
The Pages report
Under Indexing, the Pages report shows how many of your pages are indexed and how many aren't, grouped by reason. Click a reason to see example URLs.
Not every "not indexed" page is a problem. Redirected addresses, deliberate noindex pages like a form's thank-you page, and old URLs that now return a 404 belong there. What matters is whether your homepage, service pages and town pages are on the wrong list. After you fix a reason, click Validate fix so Google rechecks those pages.
Did a staging setting or noindex tag survive launch?
This is the first thing we check. While a site is being built, it should be hidden from Google, usually with a noindex tag or a builder setting. The trouble starts on launch day, when the site moves to the real domain and the setting comes with it.
Say a roofing company's new site was built with the hide switch on, as it should be. It gets copied to the live domain and looks perfect. Google visits, reads "don't index this," and does what it's told.
Where to look:
- WordPress: go to Settings, then Reading. If "Discourage search engines from indexing this site" is ticked, untick it.
- SEO plugins and site builders: most have a similar site-wide or per-page switch. Check your key pages.
- The page code: open a page, view the source and search for "noindex." If it's in a robots meta tag, that page is asking to be left out.
- The server: noindex can also be sent as an HTTP header called X-Robots-Tag, which you won't see in the page source. URL Inspection catches it either way.
Is robots.txt telling Google to stay out?
Open yourdomain.com/robots.txt in a browser. Look for a Disallow: / line under a User-agent line that applies to all bots, written with an asterisk. That single slash means "the whole site." It's a normal staging setting that sometimes survives launch.
Also look for lines that block the folders holding your styles and scripts. Google loads pages much like a browser does, and without the CSS and JavaScript it may see a broken or half-empty page. URL Inspection's live test shows a screenshot of what Google rendered and any resources it couldn't load.
The robots.txt report in Search Console, under Settings, shows the version Google last fetched and any errors. A file that fails to load is worth fixing quickly, because Google may hold off crawling until it can read it.
Are canonical tags pointing at the wrong page?
A canonical tag tells Google, "the main version of this page lives at this address." It's useful when the same content appears at more than one URL. It's harmful when it's wrong.
Common mistakes:
- A template sets every page's canonical to the homepage. Google then treats your service pages as copies of the homepage and may leave them out.
- Canonicals still point at the staging domain, or at the old http version, after launch.
- A town page's canonical points at a different town page, because one was copied from the other.
To check, view the page source and search for "canonical," or use URL Inspection to compare the user-declared canonical with the Google-selected one. If a lot of your service pages appear as "Alternate page with proper canonical tag" in the Pages report, look closer. Keep in mind that a canonical tag is a hint. Google can pick a different version if your other signals disagree.
Is Google indexing the wrong version of your site?
Staging and preview addresses
If a staging address like staging.yourdomain.com, or a site builder's preview address, was ever open to the public, Google may have indexed it. Then it competes with your real site, or gets picked as the main version instead. Run a site: search on the staging address to check. The cleanest fix is to put staging behind a password, since Google can't index what it can't open, then redirect or remove whatever is already indexed.
The reverse happens too. A "coming soon" page, a maintenance mode or a site-wide password left switched on means Google sees that page, or nothing, instead of your real one.
http, https, www and non-www
Your homepage can exist at four addresses: with and without https, and with and without www. Pick one, and permanently redirect (a 301) the other three straight to it. Make sure your canonical tags and sitemap use that same version. To check, type each of the four into a browser. All four should land on the same address.
Redirect chains and loops
A chain is when one address redirects to a second, which redirects to a third, and so on. Chains pile up after redesigns, when new redirect rules get stacked on old ones. Google follows a limited number of hops before it gives up, and every hop slows the page down. A loop is worse: page A sends you to page B, and B sends you back to A, so the page never loads. Search Console reports these as "Redirect error." A free redirect checker, or the Network tab in your browser's developer tools, shows each hop.
Can Google reach your pages without filling in a form?
Google finds pages by following links. It doesn't type in a zip code, log in, pick from a dropdown or click a button to load more. If a page can only be reached that way, Google may never find it.
Say a property manager's site shows available rentals only after you pick a city from a dropdown, and the dropdown loads the listings with a script. To Google, those rental pages may not exist. The same goes for town pages that can only be reached through a "check your service area" form.
The fix is plain links: services and towns in the menu or footer, each town page linked from a service area page, and every page you care about reachable by clicking ordinary links. Google can run JavaScript, but anything that depends on a visitor doing something first is a gamble. URL Inspection's live test shows the HTML Google actually saw.
Is your sitemap missing, or listing the wrong URLs?
A sitemap is a file that lists the pages you want Google to find, usually at yourdomain.com/sitemap.xml. It helps Google discover pages. It doesn't force them into the index.
Problems to look for:
- No sitemap at all, or one that was never submitted in Search Console
- An old sitemap left over from the previous site, full of addresses that now redirect or return a 404
- A sitemap that lists staging addresses, http versions or noindexed pages, which sends Google mixed signals
Your sitemap should list only the final, live, indexable version of each page. Submit it under Sitemaps in Search Console, which shows whether Google could read it and how many pages it found.
What should you do once you find the problem?
Step 1: Remove the block. Untick the setting, fix robots.txt, correct the canonical or clean up the redirect.
Step 2: Confirm the fix. Run URL Inspection's live test on your homepage and a few key service pages.
Step 3: Ask Google to come back. Request indexing for your most important pages and resubmit your sitemap.
Step 4: Validate in the Pages report. Click Validate fix on the reason you fixed.
Then be patient. Google recrawls on its own schedule, and nobody can tell you the exact day your pages will come back. Getting indexed is the floor, not the finish line. Ranking for your services is the next job: for local search, months one to three are usually foundations, the map tends to move in months four to six, and after that it compounds.
The same mistakes can hide a site from AI tools like ChatGPT and Perplexity too; more on that in what AEO is.
Want a second set of eyes?
These are among the first things we check when we take on a site, because everything else in local SEO sits on top of them. If you'd like a person to look, our free website review checks whether Google can find, read and index your pages, and sends back the three changes we'd make first. If your site is indexed and you want to know where you stand on the map, get a free SEO baseline: your map position, how your reviews compare and which listings have errors.