How Search Engines Work

Most search engines have three parts: a crawler, an index, and a search interface. Lets look at each part individually, to get a better understanding of them. Each part has its own role to play in the process, with all the parts working together to make searches possible.

The Crawler.

Also known as a spider or bot, this part of the search engine wanders the web, following links and picking up information for its database. Crawlers do most of their work at times of the day when search engines are less busy, but they typically visit frequently updated pages more often. This is something to keep in mind when youre working on your pages. As you may want to perform updates locally and update them when they have been finished rather than updating bits and pieces and hoping that the search engine runs into the correct version.

Also, crawlers ignore some things: your sites code, for example. Your sites title and text - your content - is the most important thing to a crawler. The fastest way to raise your sites search engine ranking for specific key words is to implement them into your title and your content.

The Index.

Once the crawler has collected all that text, it is then stored and indexed. This allows people searching for keywords and phrases to get results relating to what they were searching for - their search results. Most sites will incorporate rating systems such as Google Page Ranks or Alexa rankings in positioning your site. These ratings are used to attempt to ensure that sites that are important receive more traffic than unimportant sites.

To see this in action, go to a search engine and type in a word. Youll see some text on the page saying something like results 1-10 of 345,000. This means that the search engines index contains 345,000 pages it believes are related to the word you typed. If you wanted to, you could look through all these pages to find the information youre looking for.

In order to understand rating systems more thoroughly consider your own site. When you place links on your site you generally due so in order to increase your users understanding of the content of your site. If every site in a particular field links to a particular site, this site is probably very important to that field and should, therefore, be listed highly in the lists of search engine results. Thus the basic ideology of Google Page Ranks.

Consider again, a site that receives a great deal of traffic. If a site is receiving loads and loads of traffic, it probably has some information or service that is very important to its users. Alexa ratings attempt to estimate the amount of traffic that a particular site gets and compare it to the amount of traffic that other sites get. The closer that a site is to the most trafficked site on the internet, the more likely it is to have important content if it is relevant to the search query.

The Interface.

Search engines provide a public interface for users who want to find information on the web. They can type the word or phrase theyre searching for, and the interface will run an algorithm to find the pages relevant to their search and display them.

These algorithms are an important part of the SEO (search engine optimization) business, and the search engines are constantly changing them. Youll notice when the algorithms change, as the rankings of your website will change with them.

No two search engines are the same. They all work differently, with their own unique features, and they will all respond to your website in their own way. You should familiarize yourself with the most popular search engines, to better understand how each of them works.

The most popular search engines today include Google, Yahoo, AltaVista, AllTheWeb, MSN, and Ask Jeeves. There are many other search engines available, though, and you shouldnt ignore them altogether.

When you submit your website to the search engines, theres no way of knowing when they might add it to their indexes. Since each search engine has its own crawling and indexing methods, you cant be sure how long it might take. In some cases, you might see results within a week, but dont count on it - it may take several weeks or even months before you see anything.

Its not easy to get a high ranking unless you spend some time on it, and learn the proper methods. When you take the time and do some research, youll find that its not as confusing as you first thought. Learning the basics will enhance your experience more than you would have thought possible.

About The Author:

Lawrence Andrews is an ePublisher, software developer, consultant, and author of numerous books. Visit his Private Label Content and Software site at for more information about SEO and PRL.

A Guide to SEO Resources and Discussion Groups

Finding the right SEO tools and resources can be challenging but were here to make it a little easier on you. SEO forums and newsgroups can be very confusing to a person with no SEO experience. These forums and newsgroups are just so heavily populated that they can intimidate many new users.

The first rule regarding these situations is to just slowly try to integrate yourself into the community. Follow these rules closely and concisely so that you do not offend anybody:

1. Do not spam the forum with your problems. If you have a few things that you would like to discuss you will probably get help, but do not try to hog all of the assistance for yourself. For one thing, most of your questions have already been answered, believe it or not. There is generally a search option. Always use this before posting your problem.

2. Do not Flame. Flaming is a term used among forum dwellers to describe the behavior of yelling at people via the forum. This is generally associated with cursing, constant argument, typing in all caps to try to convey anger, and dismissing other peoples posts in an undignified fashion. There are other problems that are included in flaming, but I have given you the jist of it.

3. Never take a post off topic. If there is something that you would like to discuss that was inspired by a post on another topic, post a new topic in the appropriate board and explain where the topic was conceived. You may still want to reply to the post just to let people know that the topic has been created so that somebody else doesnt take it off topic.

4. Do not bump your thread (or post). Bumping a post is when you reply to your own post in hopes of getting a response sooner. The only time that this is acceptable is if your thread has laid dormant for about a week. Bumped threads are very annoying to most forum users. Bumping posts on a regular basis will probably lead to a lack of interest in your posts and will probably contribute to a lack of support in your ventures.

6. Be friendly. There is no reason that you cant be completely polite when posting on a forum. You will get very good response if you are simply polite in your post. Politeness in the world of forums includes making sure that you have communicated your problem carefully so that people attempting to help you can understand and provide answers to the best of their abilities.

7. Join in on the community favorites. Many forums now include games which are occasionally pretty fun. They are pretty much just there for if you are bored and waiting for a response to your other posts, but they are enjoyable if you keep up with them as many forum dwellers are quite articulate and witty.

Heres a list of resources that you can use when you need help but you dont want to pay for it.

Finding Groups.

Google makes discussion groups easy to find with their Google Groups tool (groups.google.com). Go there and type in seo to see whats on offer. Two popular groups are alt.internet.search-engines and alt.www.webmaster. If you dont like Googles groups, try Yahoos instead (groups.yahoo.com). Either of these sources will provide you with a pretty substantial list. Remember, regular search engine listings seem to apply here so generally you will see the best results towards the top of the listing.

dmoz.org is a good place to start: from their home page, you can browse down to any subject you want, and you should find at least one mailing list, discussion forum or message board in the listing.

There are several forums out there on the web that focus on SEO, and you can learn a lot from all of them, even if some of the discussions are over your head at first. Even ordinary searchers can learn a lot from following these discussions, as they tell you a lot about how search engines work.

Here are some useful forums: WebmasterWorld, Search Engine Watch Forums, ThreadWatch, Best Practices Search Engine Forums, cre8asite forums and the High Rankings Forum. Check them out.

About The Author:

Lawrence Andrews is an ePublisher, software developer, consultant, and author of numerous books. Visit his Private Label Content and Software site at for more information about SEO and PRL.