Search engine optimization isn’t about being number one for a specific word or phrase — it’s about ranking well and driving traffic with many different words and phrases. The term “search engine” is often used generically to describe both crawler-based search engines and human-powered directories. These two types of search engines gather their listings in radically different ways.

Crawler-Based Search Engines

 The search engines use robot web crawlers which go out periodically to all the pages the search engine already knows about (frequency based on importance!) and collects the latest information on that page (since pages change over time). Among other items they collect are the links on the page. The links are compared to the list of known pages and any unknown pages are put into another queue (based on the importance of the referring page!) to be crawled by the robot.

Human-Powered Directories

A human-powered directory, such as the Open Directory, depends on humans for its listings. You submit a short description to the directory for your entire site, or editors write one for sites they review. A search looks for matches only in the descriptions submitted.

Changing your web pages has no effect on your listing. Things that are useful for improving a listing with a search engine have nothing to do with improving a listing in a directory. The only exception is that a good site, with good content, might be more likely to get reviewed for free than a poor site.

There are three items combine to most assure that search engines will discern the keywords you want applied to a page. These elements are the page title, the text on the page, and the text of the link going to that page. 

The title is a standard tag found in the head section of an html – or web page – file. This text can be read in the window frame for the internet browser and is also displayed as the link name in search engine results.

The text can be displayed within a large variety of tags within the body section of the html file. It is important that there be text one way or another, sites with emphasis on design may render text into images, and in this case alternatives must be found to make the text accessible to the search engines.

The link text is the most often overlooked of the BIG THREE. How often do you still see the words click here? Most search engine ranking algorithms follow a logic which equates link text with actual content. In other words if the link says click here the search engine believes the page is about clicking here, but if the link says Benny Hinn Ministries – from Living Word, then, as far as the search engines are concerned, the page probably has something to do with – you guessed it, Living Word and Benny Hinn Ministries.

Remember that sometimes the title will contain more than just the keywords you are targetting. These examples show how title, text and link text have been so ordered to indicate the content of the pages and the target audience, at least in search terms.

One other important aspect of search engine success is your link strategy – or how you get other sites to link to your site. Links from other sites to your site are important in the first place for discovery, secondly for relevance, and finally for importance – three terms used widely in the discussion of search engine ranking.

Bookmark It

Hide Sites

Search engines are, for the most part, entities that rely on automated software agents called spiders, crawlers, robots and bots. These bots are the seekers of content on the Internet, and from within individual web pages. These tools are key parts in how the search engines operate.

Remember one thing in your mind that a spider, also known as a robot or a crawler, is actually just a program that follows, or “crawls”, links throughout the Internet, grabbing content from sites and adding it to search engine indexes.

Spiders only can follow links from one page to another and from one site to another. That is the primary reason why links to your site (inbound links) are so important. Links to your website from other websites will give the search engine spiders more “food” to chew on. The more times they find links to your site, the more times they will stop by and visit. Google especially relies on its spiders to create their vast index of listings.

Spiders find Web pages by following links from other Web pages, but you can also submit your Web pages directly to a search engine or directory and request a visit by their spider. In fact, it’s a good idea to manually submit your site to a human-edited directory such as Yahoo, and usually spiders from other search engines (such as Google) will find it and add it to their database. It can be useful to submit your URL straight to the various search engines as well; but spider-based engines will usually pick up your site regardless of whether or not you’ve submitted it to a search engine.

How Do Search Engine Robots Work?

Think of search engine robots as very simple and automated data retrieval programs, traveling the web to find information and links. They only absorb what they can see, and while a picture is worth a thousand words to a person, its worth zero to a search engine. They can only read and understand text, and then only if its laid out in a format that is tuned to their needs. Ensuring that they can access and read all the content from within a web site must be a core part of any search engine optimization strategy.

When a web page is submitted to a search engine, the url is added to the search engine bots queue of websites to visit. Even if you don’t directly submit a website, or the web pages within a website, most robots will find the content within your website if other websites link to it. Thats part of a process referred to as building reciprocal links. This is one of the reasons why it is crucial to build the link popularity for a website, and to get links from other topical sites back to yours. It should be part of any website marketing strategy you opt in for.

When a search engine bot arrives at a website, the bots are supposed to check to see if you have a robots.txt file. This file is used to tell robots which areas of your site are off-limits to them. Typically these may be directories containing files the robot doesn’t need to concern itself with. Some bots will ignore these files. However, all search engine bots do look for the file. Every website should have one, even if it is blank. Its just one of the things that the search engines look for.

Robots store a list of all the links they find on each page they visit, and follow those links through to other websites. The original concept behind the Internet was that everything would organically be linked together, like a giant relationship model. This principle is still a code part behind is how robots get around.

The smart part behind search engines actually comes in the next step. Compiling all the data that the bots have retrieved is part of building the search engine index, or database. This part of indexing websites and web pages comes from the search engine engineers, who devise the rules and algorithms which are used to evaluate and score the information the search engine bots retrieved. Once the website is added into the search engine database, the information is available for customers who are querying the search engine. When a search engine user enters a query into a search engine, the search engine performs a variety of steps to ensure that it delivers what it estimates to be the best, most relevant response to the question.

How Do The Search Engines Read Your Website?

When the search engine bot visits a website, it reads all the visible text on the web page, the content of the various tags in the source code (title tag, meta tags, Dublin Core Tags, comments tags, alt tags, attribute tags, content, etc.), as well as the text within the hyperlinks on the web page. From the content that it extracts, the search engine decides what the website, and web page is about. There are many factors used to figure out what is of value and what matters. Each search engine has its own set of rules, standards and algorithms in order to evaluate and process the information. Depending on how the bot was set up by the search engine, different pieces of information are gathered, weighted, indexed and then added to the search engine’s database. Manipulation of the keywords within these webpage elements form part of what is know as search engine optimization.

After it is added, the information then becomes part of the search engine and directory ranking process. When the search engine visitor submits their query, the search engine digs through its database to give the final listing that is displayed on the results page.

The search engine databases update at varying times. Once a website is in the search engine database, the bots will keep visiting it regularly, so as to pick up any changes that are made to the websites pages, and to ensure they have the most current data. The number of times a website is visited will depend on how the search engine sets up its visits, which can vary per search engine. However, the more active a website, the more often if will get visited. If a website varies frequently, the search engine will send bots by more often. This is also true if the website is extremely popular, or heavily trafficked.

Sometimes bots are unable to access the website they are visiting. If a website is down, the bot may not be able to access the website. When this happens, the website may not be re-indexed, and if it happens repeatedly, the website may drop in the rankings.

Bookmark It

Hide Sites