|
Many animals, including some species of birds, feed
their young pre-digested food. The young animals cannot break down the
food’s molecules enough by themselves to gain access to the nutrition,
so the adults do it for them. Without the pre-digested food, the young
would starve.
Search engines cannot gain access to many Web pages, commonly including
dynamic e-commerce product-level pages, leaving these pages to starve
from lack of traffic. However, the practice of search engine optimization
can provide workarounds that help get content in front of the spiders,
feeding them great amounts of data the engines can use in response to
search queries. It may seem like an unappetizing analogy, but it is
an apt one.
There are a couple ways to get inaccessible content to search engine
spiders. One common method is to remove spider blocks that prevent spiders
from doing their crawl of the Web site, such as undertaking the project
of dynamic URL rewriting. One such approach is called a Mod Rewrite,
a process that removes problematic symbols in the URLs that may have
prevented a spider from accessing the content for fear of a spider trap.
Some Webmasters undertake the Mod Rewrite themselves, essentially republishing
their product or information pages in a static format that is easier
for spiders to crawl effectively. Others outsource the project to a
search engine marketing firm while some look to a specialized technology
firm whose solution may include placing pages on its own server and
charging you for traffic on a pay-per-click basis. Though that temporarily
gives spiders access to your product pages, when you stop paying, the
traffic stops, too. More importantly, you haven’t resolved the
core issue: the inaccessibility of your site to the spiders of the most
important search engines, leaving you perennially at the starting block
while your competition moves forward.
Remember that removing spider blocks will not influence how high in
search rankings any particular page will appear. It will provide spiders
a pathway to access your pages so that they now may be included in the
search engine database. Search engine optimization techniques are required
for the pages to perform their best in rankings and, thus, in traffic
generation. But at least removing the spider blocks, or applying the
Mod Rewrite workaround, gets you on the playing field.
Another common way to help a search engine include all the pages of
your site in its database is to give a “direct feed” of
your site to the engine. Yahoo, AltaVista and HotBot — through
the Yahoo Search Marketing Solutions (formerly called Overture) SiteMatch
Xchange program — invite you or your SEM agency to submit a feed
of your entire Web site, including an online catalog, into its database,
removing the engines’ need to gain access to these pages through
spidering. You then are charged on a cost-per-click basis determined
by your category of product.
To start this program, Webmasters or their SEMs can provide Yahoo a
tab-delimited file of their products that fulfills certain criteria,
including required fields, or they can build an XML file to upload to
the engines. A few SEMs have developed their own spiders, spiders that
can crawl even challenging client sites to relieve the client’s
IT team from the need to build a feed in-house. Then the SEM spider
crawl can be packaged and optimized before being sent to the Yahoo program
— pre-spidered and highly digestible.
Note once again that just giving a feed to the Yahoo SiteMatch Xchange
program does not guarantee traffic, just inclusion in the database.
However, a well-constructed feed has an excellent chance to perform
well and deliver an attractive, consistent ROI.
Google, MSN and other, smaller engines do not participate in a direct
feed program. This leaves the first option of removing spidering obstacles
and including SEO techniques and the pay-for-placement programs as your
only alternatives for keeping your site from traffic starvation from
these engines at this time. But even when you have both of these elements
of search engine marketing working for you, still consider adding the
SiteMatch Xchange program because it provides two benefits:
• You can upload a fresh feed or get re-spidered frequently, ensuring
that your newest products are included in the search engines quickly
and that changes in pricing or inventory are up to date at all times.
• Your guaranteed inclusion in these important engines provides
cushioning against the uncontrollable aspects of your “organic”
search engine campaign, such as when Google changes its algorithms,
and from cost spikes in the pay-for-placement engines, such as when
a competitor starts a bidding war.
|
