Freepages-Help-L Archives

Archiver > Freepages-Help > 2002-11 > 1038537592


From: Rod Dav4is <>
Subject: Re: Fw: [FreeHelp] FINDING MY WEBPAGE ON ROOTSWEB
Date: Thu, 28 Nov 2002 21:40:01 -0500
References: <016101c29597$1ba82580$0100007f@pat> <3.0.1.32.20021127115347.0125db90@mail.augustmail.com> <5.1.0.14.0.20021128190620.022b17d0@pop3.norton.antivirus>


Pat Asher wrote:

> Rod,
>
> It is not a "problem". It is the difference in the way a spider works
> (http), and the cron job that reads the TITLE tags for the Freepages index
> from the server (not http).
>
> Pat

I consider it a problem in that it is an undocumented "quirk", a deviation from
the way users expect the title tags to be found. IMO, this needs to be either fixed
-- so the TITLE tags are extracted in the same way that users experience it with
their user agents -- or documented as an exception to the usefulness of the ssi
stuff.

Also, I have not yet received a definitive answer about the spider that
constructs the indexes for SearchThingy: Does it or does it not use the HTML
protocol? I don't want to wait 6 months only to discover then that, oh yeah, the
spider doesn't use the HTML protocol so your pages will never be indexed.

-R.

This thread: