Instead of relying exclusively on page titles and meta tags, Search engine now index the entire page contents. Since Search Engines have been able to view entire pages, the hidden inputs (such as meta tags) have lost much of their importance in relevancy algorithms.

The best way for search engines to provide relevant results is to emulate a user and rank the page based on the same things the user see and do (Do users like this website? Do they quickly hit the back button?), and what other people are saying about the document (For example, does anybody link to this page or site? Who is linking at it? What is the link text? And so on).

Search engines make billions of dollars each year selling ads. Most search engine traffic goes to the free, organically listed sites. The ratio of traffic distribution is going to be keyword dependent and search engine dependent, but I believe about 85% of Google’s traffic clicks on the organic listings. Most other search engines display ads a bit more aggressively than Google does. In many of those search engines, organic listings get around 70% of the traffic. Some sites rank well on merit, while others are there due exclusively to ranking manipulation.

In many situations, a proper SEO campaign can provide a much greater ROI than paid ads do. This means that while search engine optimizers—known in the industry as SEOs—and search engines have business models that may overlap, they may also compete with one another for ad dollars. Sometimes SEOs and search engines are friends with each other, and, unfortunately, sometimes they are enemies.

When search engines return relevant results, they get to deliver more ads. When their results are not relevant, they lose market share. Beyond relevancy, some search engines also try to bias the search results to informational sites such that commercial sites are forced into buying ads.

Many effective SEO techniques may be considered somewhat spammy. Like anything in life, you should make an informed decision about which SEO techniques you want to use and which ones you do not.

You may choose to use highly aggressive, “crash and burn” techniques, or slower, more predictable, less risky techniques. Most industries will not require extremely aggressive promotional techniques.