Many are dead set against the Gaming of search engines, yet those who do not play the game correctly are in fact buried by their competitors who do nothing but game the search engines. Often such companies are de-listed eventually, but in the mean time they do out rank your content and even when they are de-listed they are replaced by ten more.
Gaming the search engines can mean many things and there are some reasons why many tend to try to figure out where the line is to gaming the search engines. Some believe that recycling content or articles is gaming the search engines, but in reality it might be very good business if done ethically. For instance if your content on your website is no longer doing well in the search engines, rather than removing those pages, simply rewrite the content and change the titles, key words and meta tags and then when the spider comes back around you have new pages and content available.
Some might say this is gaming the search engines, by you must realize this; if your content, web pages and articles are not read by those needed to read them, then how will they reach their full potential. Meaning your customer or the entire human race Internet Surfing public goes un-served. For your website, assuming you have something valuable to tell the world or offer to the us all, this is a travesty. So if you do not put up continuous content or write new articles with new ideas you sites traffic will diminish and all the old content, information and ideas are lost. Thus you need to put up new content as well as plan a strategy of modification, renewing old articles and recycling your content.
Now then if one wishes to call that gaming the search engines so be it. But I submit to you. What if we consider Teslas stuff on electromagnetic energy, lightning harvesting power and sending energy like quantum computing to any point on the planet where needed. What if your content was lost like his experiments for over 100-years You see we are only now re-discovering what could have stopped all this pollution, which is basically killing people.
I had recently noticed on my website that some of my concepts were no longer easily found in the search engines, they had been buried in the rubble similar to the Pakistan Earthquake. Maybe one of these futuristic concepts could provide a scientist or researcher or even a University grad student an idea to propel him to think and come up with a new a discovery of great magnitude and impact to the human race you see The future of the human race is at stake in my opinion and so is your website, small business and potential viewers who could benefit from your content that no one can find anymore. So we can discuss Gaming the search engines perhaps or call it whatever we wish, but indeed in the end it is mankind who loses and you are amongst that grouping. Please think on this.
Lance Winslow - Online Think Tank forum board. If you have innovative thoughts and unique perspectives, come think with Lance;
The robot text file is used to disallow specific or all search engine spiders access to folders or pages that you dont want indexed.Why would you want to do this
You may have created a personnel page for company employees that you dont want listed. Some webmasters use it to exclude their guest book pages so to avoid people spamming. There are many different reasons to use the robots text file.
How do I use it
You need to upload it to the root of your web site or it will not work - if you dont have access to the root then you will need to use a Meta tag to disallow access. You need to include both the user agent and a file or folder to disallow.
What does it look like
Its really nothing more than a Notepad type .txt file named robots.txt
The basic syntax is
User-agent: spiders name here
Disallow:/ filename here
If you use
User-agent: *
The * acts as a wildcard and disallows all spiders. You may want to use this to stop search engines listing unfinished pages.
To disallow an entire directory use
Disallow:/mydirectory/
To disallow an individual file use
Disallow:/file.htm
You have to use a separate line for each disallow. You cannot for example use
Disallow:/file1.htm,file2.html
You should use
Use-agent/*
Disallow:/file1.htm
Disallow:/file2.htm
For a list of spider names visithttp://www.robotstxt.org/wc/active/html/ Make sure you use the right syntax if you dont it will not work. You can check you syntax here http://www.searchengineworld.com/cgi-bin/robotcheck.cgi
For help on creating robot text files there is a program call robogen. There is a free version and an advanced version, which costs $12.99 http://www.rietta.com/robogen/
Alan Murray is a Certified Internet Webmaster Professional and Provides SEO Services and Website design.