NILEX
Digital Systems
INITIALIZING0%
SEO & Google

Online Store Filter URLs, Canonicals and Crawl Budget: How to Keep a Big Store Clean in Google

How to keep a large online store clean in Google: which filter, sort and search URLs to index, when to use canonical tags, robots.txt or noindex, what belongs in the sitemap, and what "Alternate page with proper canonical tag" means, with two live Egyptian stores as examples.

Nilex cover with a screenshot of the Bashayer El Khair store category page and the title about filter URLs and indexing

To keep a big online store clean in Google, let search engines index only the pages a shopper would actually search for (products, categories, brand pages and the paginated listings that lead to them) and stop filter, sort and search variations from competing with them. That means a canonical tag on every page pointing to its clean address, robots.txt rules that keep worthless parameter URLs and private pages out of the crawl, noindex on pages that must be crawlable but never indexed, and an XML sitemap that lists only real, canonical pages. When this is right, "Alternate page with proper canonical tag" in Search Console is a sign that things are working.

This guide is for owners and managers of stores with hundreds or thousands of products, especially if Search Console shows more "not indexed" URLs than you have products. It uses two live stores Nilex built as examples: Bashayer El Khair, a home-appliance store whose sitemap lists about 1,660 URLs in Arabic and English, and Dar El Ahram, a legal publisher with about 790 book pages. What we describe comes from their public robots.txt files, sitemaps and page tags as of November 2026.

Key takeaways

  • Filters, sorting and on-site search can create far more URLs than your store has products. Decide for each URL type whether it deserves to be indexed.
  • robots.txt controls crawling, not indexing. A blocked URL can still appear in Google without its content, so use noindex for pages that must stay out of the index.
  • A canonical tag names the main address. Sorted and filtered versions should point to the clean category page.
  • Some filters are real landing pages. A brand filter that answers "Samsung washing machines" deserves its own title, canonical and sitemap entry.
  • The sitemap should contain only canonical pages that return 200 and may be indexed. Google ignores priority and changefreq.
  • "Alternate page with proper canonical tag" means Google followed your canonical and indexed the main page. For filter and sort URLs there is nothing to fix.

Why do filter, sort and search URLs cause trouble?

Every time a shopper ticks a filter or changes the sort order, most stores add a parameter to the address: ?sort=price, ?price=1000-5000, ?brand=samsung, ?q=kettle. Each address shows a slightly different version of the same list, and search engines see each one as a separate URL to crawl. As an illustrative example only: 30 categories, four sort orders, five price ranges and twenty brands give 30 × 4 × 5 × 20 = 12,000 category variations before anyone combines two filters or types a search, and none of them has content the plain category does not already have.

Google's guidance on managing faceted navigation URLs names two costs: overcrawling, because a crawler cannot tell a filter URL is useless until it has fetched it, and slower discovery of new, useful pages.

Bashayer El Khair Arabic irons category page with breadcrumb, a filter button showing one active filter, a newest-first sort menu, a category filter chip and iron product cards with brands and prices
The irons category on Bashayer El Khair. Filters and sorting change the list for the shopper; the page keeps one clean address for Google.

Does a store with a few thousand pages need to worry about crawl budget?

Usually not in the strict sense. Google's crawl budget guide is written mainly for sites with around a million unique pages, sites of 10,000+ pages whose content changes daily, and sites with many URLs stuck as "Discovered, currently not indexed". For other sites, Google says keeping the sitemap up to date and checking the Page indexing report regularly is adequate.

Both example stores are well below those sizes. The reason to care anyway is that filter URLs are close to an unlimited supply, and the goal is that the real pages are the ones Google sees, chooses and shows.

Which URLs should Google index?

List every URL pattern your store produces and decide what each one is for. This table shows the usual decisions and what the two example stores do today.

URL typeExample patternIndex it?How it is handled
Product or book page/product/model-name, /books/titleYesSelf-referencing canonical, in the sitemap
Category page/shop/washing-machines, /categories/civil-lawYesSelf-referencing canonical, in the sitemap
Brand or author page/shop?brand=samsung, /writers/author-nameYes, when people search for itIts own title and canonical, in the sitemap
Paginated listing/shop?page=2, /books?page=2Yes, as part of the seriesEach page keeps its own canonical, not page 1
Sorted listing?sort=priceNoCanonical to the clean category; also disallowed on Bashayer
Price or spec filter?price=, ?energy=NoCanonical to the clean category; also disallowed on Bashayer
On-site search results?q=, ?search=Nonoindex, follow with a canonical to the main listing
Cart, checkout, account, login/cart, /checkout, /loginNoDisallowed on Bashayer; crawlable with noindex on Dar El Ahram
Admin, API, private files/admin, /api/, paid book filesNever fetchedDisallowed in robots.txt

The rule: if a URL answers a search someone actually types, give it a clean address, a unique title and a sitemap entry; if it only reorders or narrows a list, point it back to the main page.

robots.txt, noindex or canonical: which tool does what?

Most store indexing problems come from using the right tool for the wrong job:

  • robots.txt tells crawlers which URLs they may fetch. Google's introduction to robots.txt says it is not a way to keep a page out of Google: a disallowed URL can still be indexed, without its content, if something links to it.
  • noindex, as a robots meta tag or an X-Robots-Tag header, keeps a page out of results, but only if Google may crawl the page and read it (block indexing with noindex).
  • rel="canonical" says which of several similar addresses is the main one. Google's page on specifying a canonical URL calls it a strong signal, ranks redirects as stronger and sitemap inclusion as weaker, and advises against using robots.txt for canonicalization.

The classic mistake is to disallow a page and also give it noindex. Google never fetches the page, never sees the noindex, and may still index the bare URL because the menu links to it from every page. Dar El Ahram's robots.txt records exactly this lesson in its comments, as described below.

How Bashayer El Khair keeps a bilingual catalogue clean

Bashayer El Khair's sitemap in November 2026 lists about 1,660 URLs: 635 product pages in each language, 33 category pages plus the main shop page in each language, brand pages, paginated shop pages, 100 Arabic articles, and the home, offers, policy and contact pages in both languages. Each entry carries hreflang links to its Arabic and English version. The project itself is in the home appliances store case study. What its robots.txt and page tags do:

  • Transactional pages stay out of the crawl. Cart, checkout, account, order, login, register and email-verification pages are disallowed in every language, under a comment that reads "Never list any of these in the sitemap". The admin area and API are disallowed too.
  • Sort, price, energy and search parameters are disallowed with rules such as Disallow: /*?*sort=.
  • Filtered pages still point home. The irons category sorted by price declares the plain irons category as canonical; search results send noindex, follow with a canonical to the main shop page.
  • The brand filter is the deliberate exception. The file explains that "غسالات سامسونج" is a real commercial search, so /shop?brand=samsung is a landing page, not a permutation. Brand pages have their own titles, such as "Samsung Appliances — Prices & Offers", their own canonical and a sitemap entry in both languages.
  • Images stay crawlable for Google Images, and the sitemap is declared on a Sitemap: line.

Deciding which filters a store needs in the first place is covered in product categories, brands and filters; this article is about keeping them from flooding Google.

How Dar El Ahram handles about 790 books with a short robots.txt

Dar El Ahram's sitemap in November 2026 lists 988 URLs: about 790 book pages, author pages, 20 legal-branch category pages, blog posts, paginated book listings and pages such as events, legal questions, latest releases and policies. Its robots.txt takes the opposite style to Bashayer's: very few rules and a long comment explaining why.

  • Only the API, the admin interface and the folder of paid book files are disallowed. Cover and article images are explicitly allowed so link previews and Google Images keep working.
  • Login, signup, cart, checkout and profile pages used to be disallowed while also sending noindex, and the file notes they then appeared in Search Console as "Indexed, though blocked by robots.txt". They are now crawlable, so Google can read the noindex.
  • Old book addresses, the same comments say, now redirect permanently to the current book pages and are no longer blocked, because a redirect on a blocked URL is invisible to Google.
  • A branch page sorted by price names the clean branch page as canonical; book searches send noindex, follow. Listings such as /books?page=2 keep their own canonical and a "page 2" title, as Google's pagination guidance recommends.
Dar El Ahram labour and social insurance law category page with a dark red header, a short description of the books it covers, a breadcrumb, a book count and book cover cards
A legal-branch category on Dar El Ahram. This address is the one in the sitemap; sorted or searched versions point back to it.

Both styles work: Bashayer blocks parameters it never wants fetched, Dar El Ahram relies on canonicals and noindex, and neither blocks a URL that depends on noindex. How the book, author and branch pages were planned is in SEO for a large book catalogue.

What belongs in the sitemap?

A sitemap is the list of pages you want shown in results. Google's guide to building a sitemap says to list canonical URLs and, where content is reachable at several addresses, only the one you prefer. For a store:

  • Include products, categories, brand or author pages, pagination pages with their own canonical, articles and public information pages.
  • Leave out sorted and filtered variations, search results, cart and account pages, redirects, error pages and anything marked noindex.
  • Generate it from the catalogue, so added products appear and removed ones leave. Dar El Ahram's sitemap updates itself as books, authors and articles are added.
  • Declare it in robots.txt with an absolute URL and submit it in Search Console. Google ignores priority and changefreq, uses lastmod only when it is consistently accurate, and limits one file to 50,000 URLs or 50MB uncompressed.

What does "Alternate page with proper canonical tag" mean?

In the Search Console Page indexing report, this status (often searched as "alternative page with proper canonical tag") means Google found a page that names another page as canonical, agreed, and indexed the canonical instead. Google's help page says there is nothing you need to do. On a store with filters, hundreds of URLs in this group is normal. What to check:

  • Open a sample. If the group holds only filter, sort and parameter variations, the setup works.
  • Look for real pages. A product, category or brand page in this group has a canonical pointing to the wrong place. That is a bug.
  • Watch "Duplicate, Google chose different canonical than user". Google disagreed with your canonical; on real products, look for near-identical descriptions.
  • Watch "Indexed, though blocked by robots.txt". A page that should carry noindex is probably also blocked. Unblock it.

Our Google Search Console guide covers the other monthly reports. For everything else a store needs before launch, see the e-commerce SEO checklist for Egypt, and before moving a store to new addresses read redesigning without losing SEO.

Sources

How Nilex helps

Nilex builds stores where this is part of the build, not a clean-up job later: canonical tags on every page, filter, sort and search URLs handled according to a written list of URL patterns, private pages kept out of the index the right way, a sitemap generated from the catalogue, hreflang for Arabic and English, and Google Search Console connected at launch. Our SEO work is technical only: we do not write your product content or articles, build links or promise rankings. You publish the content; the store is built so it can be found and indexed.

See our e-commerce store service and SEO-ready website service, and the project pages for Bashayer El Khair and Dar El Ahram. If your Search Console report is full of filter URLs, send us your store address and what you see.

Frequently asked questions

Should I block all filter URLs in robots.txt?

Block those that never answer a real search, such as sort order, price ranges and on-site search, if you never want them crawled. Keep any filter that works as a landing page, such as a brand page, crawlable with its own title and canonical. Never block a URL you rely on noindex to remove.

Is "Alternate page with proper canonical tag" an error?

No. Google followed your canonical and indexed the main page. It needs attention only if a real product, category or brand page appears in that group.

Should page 2 of a category point its canonical to page 1?

No. Google advises giving each page in a series its own canonical, as both example stores do.

Can brand or filter pages go in the sitemap?

Yes, if they are real landing pages with their own canonical, title and content, like Bashayer El Khair's brand pages. Do not list sorted or narrowed versions whose canonical points elsewhere.

My store has 2,000 pages. Is crawl budget my problem?

Probably not. Google's crawl budget guide targets much larger or faster-changing sites. At your size, a clean sitemap, correct canonicals and a monthly look at the Page indexing report matter more.

LET'S BUILD

YOUR VISION.
OUR TECHNOLOGY.

Tell us what your business needs. We'll build the system around it.

or email us at info@nilexdigitalsystems.com