There are many reasons why you may have problems with getting your web pages indexed, and most have issues to do with making it difficult for search engines 'robots' accessig your web pages.
Search engine robots are simple programs - if they can't find your site content immediately, they will skip our site and move on to the next site. It is therefore critical that your site is setup to be search engine friendly and 'crawable' by search engine robots.
Below are the top 5 reasons for robots not crawling your site:
1. Your robots.txt file is setup incorrectly or corrupted.A robots.txt file tells robot which files/directories you don't want it to index. If you have any typos or errors in your robots.txt file, then a robot may completely ignore your entire site. Log into Google Webmaster Tools and use their robots.txt testing facility.
2. Your URLs contain too many variables
Long URL strings, URLs with question marks, that are dynamically generated, or that contain multiple symbols can be difficult for search engines to follow. Make your URLs as simple and descriptive as possible.
3. Session IDs in URLs
Session IDs cause duplicate problems issues and are a big no for Google. Session IDs should be avoided at all costs - look to use cookies instead.
4. Excessive onsite code
If you have too much code on your site and the real content of your site is difficult to find, robots may just give up. Make sure your CSS, javascript etc is moved off your pages and into separate files.
5. Poor website navigation
When we say poor website navigation, we're talking about code that Google can't follow - that means flash based menus, Ajax, some Javascript and DHTML.
Crucial to any chance of being ranked on Google is to ensure that its robots can find and index all of your web pages. Don't put barriers in the robots path, bt just try to make their job as easy as possible. Make sure you have god navigaiton, that your robots.txt file is working and that you have an on-page sitemap. Once your pages are indexed, you can then see what additional work is needed to get your pages the exposure you need, whether that's adding more content or working on developing more good quality links to those pages.
Google gices a Quality Score to each keyword or phrase that you have setup in your PPC campaigns. Factors included in Quality Score including landing page quality and load times.
Google has today announced some improvements to its quality scores:
- Quality Score will now be calculated at the time of each search query
- Keywords will no longer be 'inactive for search'
- Minimum bids will now be replaced with a 'first page bid' indication
First page bids will be based on the exact match version of the keyword, the ad's Quality Score, and current advertiser competition on that keyword.
According to Google, "these changes will mean that AdWords will use the most accurate, specific, and up-to-date performance information when determining whether an ad should be displayed. Your ads will be more likely to show when they're relevant and less likely to show when they're not. This means that Google users are apt to see better ads while you, as an advertiser, should receive leads which are more highly qualified." For example, Quality Score will now take location based information into account, so a phrase that performs better in one location than another, will have a higher Quality Score for the better performing location.
Until now Google has always had issues with flash and understandably so - Google reads text so how could we expect it to be able to understand a flash file, that contains no text!
A month ago, Google announced that it had improved the way it indexed flash files, so that it can now index textual content within SWF files including buttons and menus and it can also discover URLs appearing in flash files.
What hasn't changed? Welll Google still can't recognise text that appears on images, FLV files can't be indexed and Google will still have problems properly indexing websites built purely in flash. Google also has problems executing some types of JavaScript, so if your website loads flash via JavaScript, you may have problems.
So flash is still a big problem for search engines such as Google and there are several fundamental reasons for this:
- Flash websites usually present the entire website content on the same URL, so search engines find it difficult to find relevant sections of flash sites.
- It's difficult to get inblund links to relevant pages, because fo the above URL issue.
- Flash sites don't use basic SEO methods - ie title tags, H1s, link text etc.
- It is difficult to divide text into meaningful sections since flash doesn't use h1 or p tags. Also woprds are often broken down into their individual letters to create the effects that designers are looing for, making Google's job even harder.
Flash has not been designed for search engines and it is extremely difficult to get high rankings with pure Flash sites. Our advice is to use flash for what is was created for - not to build an entire website, but to enhance and to bring some life and movement to an HTML site.
