Tips for Website Redesign: Performing Search Engine Surveillance

After consulting with a few potential clients recently, I found inspiration to write a series of tips on the topic I am asked about regularly. While every redesign project is different and has it’s own challenges, the next few posts in this series will give you a few things to try out as you start the process.

Here we go…

Tip 1 of 10:  Identify how much the Search Engines know about your current site.

Type: into Google.
Look at this example where I’m looking at how well Google knows itself ‘’.

Google with www prefix

with www prefix

Try without www and see if there is any difference in the # of indexed pages.

Google without www prefix

without www prefix

As you can see from the results there is a difference. In this case Google is showing you the index for sub-domains ( as well. Your site may or may not use sub-domains as heavily as Google but following this step on a regular basis will tell you how much of you’re site is being indexed by Google.

Now for some fun… Let’s try it in Yahoo! Go to: and see what you get. See my example for the same search below.

Yahoo! Site Explorer

Site Explorer from Yahoo!

Notice to the right, Yahoo! allows with a click of a button to switch between all sub-domains & only this domain. This way you don’t need to remember to remove the www, Yahoo! will do it for you.

Want to see what the up-and-comers think? Try playing around with some newer search engines that seem to be making waves of innovation throughout search. – cool slash the web technique for search. (method: works, also check out the ‘seo’ tag directly beneath the result title. Kinda cool!)

Blekko Search Results

/slash the web Blekko style

or – quick answers to common search questions without the junk, focused on privacy. (method: * – Do not forget the * it makes all the difference)

DuckDuckGo Search Engine or

For the other search engines, such as Bing, do what you’ve done for Google, the method ‘’ works the same.

Why is it so important to know how much of your site has been indexed by the search engines?

Well for one, it’s good to know how many of your pages have been indexed. It’s also very important to know how the search engines indexed your site by paying attention to what URLs are showing in the search index. Are the number of pages listed for your site in the results close to the number of actual pages in your site? (If yes, great!) With some CMS’s this is a complicated process, but after clicking through the indexed results you will get a very good idea of what the search engine has indexed.

Secondly, in the case of a redesign… When you overhaul a website, the URL structure might receive an overhaul too. Having an idea of how many pages are indexed in the search engines might give you a good idea of how much work is ahead, if you choose to change your URL structure and want to keep your traffic and search rankings.

old way:
new way after redesign:
It would be a shame to have traffic click on an ‘old’ link in Google’s index and be presented a 404 error. You can actually take that traffic and direct it to the appropriate page with some knowledge about 301 redirects. The best method to avoid this is to keep your URL structure the same as the old structure. If that is impossible for some reason, you will want to use a 301 redirect to tell Google that the page it has indexed has moved permanently. If you are familiar with Google Webmaster Tools, you can even tell Google what pages should not be in the results at all.

If Google shows 100,000 results for your website – you might choose to rethink changing your URL structure, as the process of redirecting URL’s will be a long and tedious one. If you have 100, then redirecting your old URLs to new ones will not be nearly as difficult.

Sure, over time the results will work themselves out but we don’t recommend completely ignoring this often overlooked step. Dips in traffic are common after a redesign, you wouldn’t want to add to the problem by missing this very important step.

Next up
Tip 2:  Learning from your competitors before starting a website redesign.

Tagged under: