Drubby World Wide Web Search

What Do Search Engine Spiders Do - How Do They Work?

With so many different Search Engines online today you need to take time to understand how their spiders work so that you can truly optimize your domain in order to get the best out of your web presence. In this article you will learn how some of the more popular Search Engine spiders actually work.

One of the smaller, popular Search Engines is InfoSeek. This Search Engine only indexes the first 200 words on your web page. This is why it is very important to make sure that you have good meta tags on your site. These meta tags should list the most important information first because this is the information that will be used as a description for your site. You should realize that most meta tags can only contain about 200 characters of text. However, you can place 1,000 characters within your keywords meta tag.

You really should keep these simple rules about meta tags in mind for all of the Search Engines that you are working with. At the same time though, you should also realize that a lot of Search Engines do not even look at meta tags. This is why it is so important to have the same information in your web site?s body as is in your meta tags. Of course you cannot enter a list of keywords without this being detrimental to your site?s content.

The next Search Engine worth looking at is Alta Vista. Its spider is named Scooter and it may take three months before it fully indexes your site. This is different from the average spider which only takes between 6 and 8 weeks to index your entire site. However, normally Scooter will only spider between 2 and 10 pages of your web site each week. This is why it takes it longer but at the same time the longer you have your web site up the better it will do on Alta Vista.

While Excite was once a search powerhouse it is no longer the provider of AOL and Netscape search, and thus it is a lot less important than it once was. Excite uses a very complex algorithm to determine your site?s keyword relevance. The way it works is that once it indexes your pages it attempts to summarize them by selecting what it thinks are the most relevant sentences. In order to keep this up to date, Excite will then continue to review your page approximately every two weeks. Meta tags do not mean anything to Excite?s rankings but your description tags and relevant words will carry a lot of weight with this Search Engine.

Excite also owns WebCrawler but WebCrawled is operated independently of Excite. If you are able to get listed with WebCrawler, then you should do your best to keep your listing because this is the easiest Search Engine to get listed with. However, you should know that its submission process is tough, but not impossible. This is because it uses hit-and-miss standards combined with sporadic indexing methods.

Lycos has fully integrated DMOZ (the Open Directory Project aka ODP) into its mainstream results pages. They also use search results from AllTheWeb. It is important to do well on Lycos though because this is the site that runs click-throughs to HotBot, which is their sister site. All of this makes it very hard to understand Lycos. Generally though your site is going to eventually be indexed by Lycos if you are indexed by DMOZ and AllTheWeb. Of course, the biggest Search Engine out there today is Google. This Search Engine uses a page ranking system that is often referred to as PR. It is their central basis for their index. At one time it was almost impossible to manipulate PR but webmasters soon discovered that the more links they had the higher their PR and the better their Google ranking. This Search Engine?s spider is regularly sent out whenever Google re-indexes its own database.

Another important Search Engine to get listed in is MSN. If you can get a good ranking in Google, MSN and Yahoo! your site will do well because these 3 Search Engines provide the majority of Search Engine traffic. Usually MSN will be the first Search Engines to index your site. It will also list the most pages the fastest.

While nobody can tell you precisely when a specific Search Engine will index you, it is a good idea to check weekly. One thing that you should never do is re-submit your site more often than every 2 months. Re-submitting your site more often may keep you from getting indexed at all.

Now that you have a basic understanding of how some of the numerous Search Engines online today actually work, you need to begin working to optimize your domain so that you can make the best of your web presence. Hopefully this article has started you on your way to learning how to use these Search Engines to your benefit.

New to SEO? learn SEO, learn search engine marketing our highly rated search engine optimization video membership site "High Rank Directories Plus". Visit us today - http://www.high-rank-directories-plus.com

MORE ARTICLES:

Andrew Hazen, CEO and Founder of Prime Visibility, Will Hold Search Engine Optimization Seminar
Andrew Hazen, Founder and CEO of Prime Visibility, leading Search Engine Optimization and Internet Marketing firm, is presenting a seminar on Wednesday, November 14, 2007 from 8:00am to 11:00am at the Huntington Hilton, on Route 110 in Melville, New York. There will also be an optional Live Site Clinic from 11:00am - 12:30pm where participants can choose to have a live analysis of their website done for free, which has an actual value of $1,200.00.

Website Breakthrough Takes Search Engine Optimization Services to the Next Level
Website Breakthrough has developed a way for their clients to get higher rankings in Google, Yahoo and MSN. Using tools they developed, they can now acquire links for their clients that are stronger in the eyes of the search engines than reciprocal links, directory submissions, or purchased text links.

Pepperjam Named Top 10 Search-Engine Optimization (SEO) Firm by Promotion World
Pepperjam, an Inc. 500 Company and the industry's leading full-service internet marketing agency, announced today that they have been named one of the top 10 organic search-engine optimization (SEO) firms in the United States. The award, coordinated by PromotionWorld.com, recognizes the best promotion companies providing SEO, Pay-Per-Click Advertising, Organic SEO and Link Popularity solutions.

Achieve Improved Search Engine Ranking
If you are an owner of a website, then you will know all too well that getting optimal rankings in the search engines can be quite a challenge The keywords you use play a big part in improving your search engine rank

Off-Page Search Engine Optimization (SEO) Primer
In the earlier years of search engine optimization, it was all about on-page optimization. Things like the page title, Meta tags, heading tags, comment tags, and anchor text were the only things that mattered. Keyword stuffing could easily get your website to the top of the search engines results page. If I wanted the number one position, all I had to do was copy my competitor?s page, add my keyword one more time, and viola ? I?d rank first. These days, things aren?t quite that easy. In order to obtain top positioning in the search results, your website has to gain to credibility or ?web cred? in order to be recognized as important.

Search Engine Optimization Strategies Guaranteed to Skyrocket your Rankings
The point of optimizing your website is so that you will get ranked higher in the search engines and receive more visitors to your site. As a result, you will increase sales and revenue.

Designing a Website So the Search Engines Will Like You
Before you go and spend big money on a professional website designer, or start designing yourself, read through this article and make sure that you or your designer knows how to design a website that the search engines will like.Being a web designer myself, I know firsthand what they teach you in college about being a good designer.

Search Engine Optimization - A Must
As you surf the web take a look around at many of the sites you see. Do you notice anything that seems strange? Well, let me point it out to you.

Aha! I'm Having a Baby : New Pregnancy Site Combines the Power of a Search Engine With Customization to Deliver a Delightfully Personal Online Reso
Aha! Baby launched today Ahababy.com, a delightfully personal online resource that provides the absolute best qualified information for all things to do with pregnancy. The site combines the power of a search engine with customization to provide expectant parents with the exact information they are looking for all in one place. Using ahababy.com, expectant mothers and fathers get the best quality information to become instantly informed on topics relating to their specific situation.

The Faq's Of The Spa
With one out of five Americans having visited a spa, chances are those who have not yet experienced the spa are at least aware of its benefits and services To take the guesswork out of an experience that has been designed for relaxation, the International SPA Association (ISPA) has compiled a list of Frequently Asked Questions: How do I find the right spa for me

What's Better For Your Business...Search Engine Optimization Or Pay-Per-Click Sponsored Search
LEARN THE DIFFERENCES AND ACHIEVE HUGE RESULTS

New Research: Search Engine Marketing, Press Releases Are Tactics of Choice for Human Resource Suppliers
Latest in “HR Marketing Trends” series reveals where HR suppliers spent their budgets in 2006, and what’s ahead for 2007.

Get Better Search Engine Rankings with RSS
RSS is the latest craze in online publishing. But what  exactly is RSS?RSS or Rich Site Syndication is a file format similar to XML, and is used by publishers to make their content available to others in a format that can be universally understood.

Internet Marketing Expert Shows How Search Engine Optimization and RSS Can Boost Public Relations Visibility
TopRank CEO Lee Odden to Speak at Bulldog Reporter's Advanced Public Relations Technology in Practice Event