Looking for old forum posts and information? View the old forum archive here ยป
Welcome to Vortx Community Forum, where you can ask questions and receive answers from the staff at Vortx and other members of the community.

If you had a user account on our previous forums website, you will need to register a new account here.

A static copy of our previous forums website is available online for reference. Click here to access the forum archive.

Learn more about...

AspDotNetStorefront
DotFeed

How to stop Google from indexing category pages that have no items?

We have noticed that Google is indexing category pages that have no items, pages that shouldn't exist. This is diluting our website.

Let's say you have a category that has enough items to need 5 pages such as:

http://www.aspdotnetstorefront.com/c-156-all-add-ons.aspx?pagenum=5

There should not be any pages after that...

But if you change the pagenum to something beyond, you get a page, a page with no items. So pages 6 through 21 are all duplicate pages.

http://www.aspdotnetstorefront.com/c-156-all-add-ons.aspx?pagenum=21

This should be a 404. or a 400? or a 308 to page 1?

How do we stop this from happening?
asked Jun 12, 2014 in MultiStore by Vegarari (350 points)

1 Answer

0 votes

I think step one should be for you to eliminate any processes on your pages that might be generating excessive links to non-existent pages. Your page-visible pagination links shouldn't be creating invalid links, which is good, but the absence of link elements in the page head showing the beginning and ending of a paginated series may lead to search engines incrementing the pageNum querystring.

The first page in your series should have a pagination <link> element in the document head with a rel=next attribute, and no prev. 

The last page in your series should have a pagination <link> element in the head with a rel=prev attribute, and no next.

All of pages in between should have two <link> elements, one with the rel=prev attribute and one with the rel=next attribute.

Your question uses two category pages, so I'll continue with that.

Assuming the Xml Packages that generate your category pages use the GetProducts stored procedure, then the number of pages in that category should be exposed to the XSLT as an XML node. On my site, it's at /root/Products2/Product/pages.

If you've got that, great. Otherwise, you can add:

SELECT cast(ceiling(@rcount*1.0/@pagesize) as int) pages, @rcount ProductCount  

Just before the final END of the GetProducts procedure and you'll be receiving that value. You could also calculate it based on the number of products divided by the number of pages, but XSLT really isn't a place you want to try to do math.

With that, you can add some code like this to the Xml Package used by your category page.

        <xsl:if test="/root/Products2/Product/pages &gt; 1">
          <xsl:choose>
            <xsl:when test="(/root/QueryString/pagenum = 1) or (count(/root/QueryString/pagenum) = 0) ">
              <xsl:variable name="paginationNext" select="concat(/root/System/StoreUrl,/root/System/PageName,'?pagenum&#61;2')" />
              <xsl:value-of select="ryan:AddPagination($paginationNext, 'next')" />
            </xsl:when>
            <xsl:when test="(/root/QueryString/pagenum = 2) and (/root/Products2/Product/pages > 2)">
              <xsl:variable name="paginationNext" select="concat(/root/System/StoreUrl,/root/System/PageName,'?pagenum&#61;',(/root/QueryString/pagenum)+1)" />
              <xsl:variable name="paginationPrev" select="concat(/root/System/StoreUrl,/root/System/PageName)" />
              <xsl:value-of select="ryan:AddPagination($paginationPrev, 'prev')" />
              <xsl:value-of select="ryan:AddPagination($paginationNext, 'next')" />
            </xsl:when>
            <xsl:when test="(/root/QueryString/pagenum) = (/root/Products2/Product/pages)">
              <xsl:choose>
                <xsl:when test="(/root/Products2/Product/pages)-1 = 1">
                  <xsl:variable name="paginationPrev" select="concat(/root/System/StoreUrl,/root/System/PageName)" />
                  <xsl:value-of select="ryan:AddPagination($paginationPrev, 'prev')" />
                </xsl:when>
                <xsl:otherwise>
                  <xsl:variable name="paginationPrev" select="concat(/root/System/StoreUrl,/root/System/PageName,'?pagenum&#61;',(/root/Products2/Product/pages)-1)" />
                  <xsl:value-of select="ryan:AddPagination($paginationPrev, 'prev')" />
                </xsl:otherwise>
              </xsl:choose>
            </xsl:when>
            <xsl:otherwise>
              <xsl:variable name="paginationNext" select="concat(/root/System/StoreUrl,/root/System/PageName,'?pagenum&#61;',(/root/QueryString/pagenum)+1)" />
              <xsl:variable name="paginationPrev" select="concat(/root/System/StoreUrl,/root/System/PageName,'?pagenum&#61;',(/root/QueryString/pagenum)-1)" />
              <xsl:value-of select="ryan:AddPagination($paginationPrev, 'prev')" />
              <xsl:value-of select="ryan:AddPagination($paginationNext, 'next')" />
            </xsl:otherwise>
          </xsl:choose>
        </xsl:if>

 

My site uses a custom-built XSLT Extension, so you won't have the 'ryan:' prefix, but you can add the following function to whatever XSLT Extension codefiles you may have. I'd strongly advise against adding to or modifying the aspdnsf one, create your own instead and add custom code there.

        public void AddPagination(string paginationHref, string paginationRel)
        {
            HtmlLink pLink = new HtmlLink();
            pLink.Attributes.Add("href", paginationHref);
            pLink.Attributes.Add("rel", paginationRel);

            Page page = HttpContext.Current.Handler as Page;
            page.Header.Controls.Add(pLink);
        }

 

answered Jun 12, 2014 by Chris (3,685 points)
...