harassing me for not showing my ID and video recorder in a public airport and calls for backup
I'm not sure if this is a problem specific to how my hosting is structured or not.
Let's say the main domain for my account is
maindomain.com (for purpose of discussion).
I am building other sites, e.g.
The folder/directory structure on my hosting would therefore be:
maindomain.com (root) + +---> anothersite.com (subfolder) + +---> website.com (subfolder)
The domains are mapped such that they map to their respective folders. So for example,
anothersite.com takes the visitor to the
index.php in the folder
However, in Google, as well as just search results for these sites and their content -
website.com (what I want), I also see results that include:
Obviously this is not the way I want search data for these sites to occur. I would like them to be omitted from the search results.
How can the hosting be configured so that the results for each of these respective domains are separate, i.e. not concatenated, so if I search for items relating to these respective sites, I only get results for
anothersite.com relevant to it, same for
website.com and same for
maindomain.com - also used itself.
Such a solution to this problem should not damage the results that I do want to see.
I'm aware of
robots.txt but I'm not sure what I need to put in these to stop the mixed-up/concatenated searches without damaging the real results. Or even if all search engines obey them. Is there something I can also do with
As it is shared-hosting, I don't have complete control over how the folder/directories are structured for this hosting so I would have to stick with the structure but ideas as to how to get desirable search results are welcome.
I'd advise you to add a rewrite rule to your webserver configuration so that URLs are rewritten thusly:
maindomain.com/anothersite.com --> anothersite.com maindomain.com/anothersite.com/page.html --> anothersite.com/page.html etc....
With a 301 Moved Permanently redirection.
This should cause Google to correct this as pages are recrawled.
You could also add these folders to your main domain's robots.txt file and forbid Google from crawling them (and rely on it finding that material again or promoting the 'direct' version). You can speed this up by using Google Webmaster to remove these folders from the main domain (although it is anyones guess as to how long it will take for the 'direct' sites to show up on their own in Google).
- +1 @Kris thanks - this looks good! If I do this, how long before Google etc update their listing/indexing? I hope to accept your answer after trying it out, I've edited Apache directives in .htaccess before.
- Specific .htaccess directives needed might be these: webmasters.stackexchange.com/a/16816/3738
- 1 @therobyouknow How long it takes depends on how often Google crawls your site. Could be a few days, could be a few weeks. No good way to tell.
- I think your suggestion is better than that shown in webmasters.stackexchange.com/questions/16805/… because I would think yours stops the unwanted paths appearing in results in the first place, whereas theirs doesn't - rather it presents the visitor with a 404 page, it seems.
- I found out that my provider lets me move the sites' content up a level to avoid this scenario. But nonetheless this answer is good to know, to learn a bit more about Search Engine Optimisation and crawling to give certain results. Thanks again.