Prevent google from indexing ajax loaded content

Question

On our site we load identical content via Ajax calls (when the users click on the menu, just to prevent reloading the entire page again, so as to improve user experience). So this is works well, but actually this Ajax loaded content is actually a copy of the original content. May I prevent Google from indexing this content? http://dinox-h.hu/en/gallery.php In the left menu you can see the links: For example: http://dinox-h.hu/puffer_tartalyok_galeria.php?ajax=1

Accepted Answer

Try adding the following on your Ajax-delivered pages:

```
<meta name="robots" content="noindex,nofollow" />
```

This will tell site crawlers to not crawl the page. You could also add the pages in `robots.txt`, like this:

```
User-agent: *
Disallow: /*?ajax=1
```

That would block any URL with `?ajax=1` from being indexed (providing a robot honours your `robots.txt`). A better solution would also involve creating a sitemap and telling various search engines about it.

Edit A better way of delivering Ajax content IMO would be to send the following header when requesting your pages via Ajax:

```
X-Requested-With: XMLHttpRequest
```

jQuery will do this by default, so provided you can check for it on the server side, you could deliver your usual content e.g. without the template. You could then very easily deliver different content from the same URL depending on what the type of request is. This should also solve your crawling issue as I doubt a crawler would stumble across it.

Prevent google from indexing ajax loaded content

Solution

Problem