SECONDSITE-L Archives

Archiver > SECONDSITE > 2003-05 > 1052489746


From: Christopher Brooks <>
Subject: [2NDST] Incremental update strategies
Date: Fri, 9 May 2003 10:15:46 -0400
In-Reply-To: <200305090502.h49529H5005954@lists2.rootsweb.com>


Darrell Martin wrote:

:I would *like* to see the equivalent of "Static Pages" for Source
:Pages, but that is because of a procedure I am working on to
:identify changed files in my output

Bingo. I have spent the last month trying to work out a procedure for
incrementally updating and adding to my web pages with Second Site --
without necessarily overwriting each previously generated page.

Background parameters:

-- 37,000+ individuals in TMG 4.0d dataset
-- Perhaps 10,000 of these are destined to go online, a couple
dozen at a time as they're finally edited and polished
-- I add a family at a time ("family" meaning parents + children +
perhaps an additional generation or two of descendants) to the
existing online population
-- Probably 75% of my sentences are custom, making post-upload
tweaking (like removing extraneous software-generated commas)
necessary, and requiring that these revised pages NOT be overwritten
(permanently) by the next site generation cycle.

Strategy worked out so far:

-- Set a SCRATCH flag to Y in TMG for each newly edited person I
plan to upload.
-- Open SS and generate a site containing only those persons with
the value SCRATCH=Y.
-- FTP the new person pages and any charts for these persons to the
web, but none of the rest of the SS output..
-- Back in TMG, change the SCRATCH=Y value to ONLINE=Y to match
those persons already uploaded.
-- Back to SS, where I regenerate the site to include all persons
with ONLINE=Y. (Former population + those just added.)
-- FTP the new (ONLINE=Y) index, person index, source and main
pages to the site, as well as all the chart pages (to update the
links and properly assimilate the new persons).

This is not without major headaches, though the hassles may be due to
sloppiness on my part. Keeping everything synchronized is a constant
challenge. The biggest problem is broken links due to various
portions of the site being out of synch. I'm keeping read-only copies
of those person pages which needed post-SS. After each site update, I
resend these "reserve" pages to the site, overwriting the
last-generated versions.

So far this is feeling like brain surgery. I'm not sure whether I'm
the surgeon or the patient, or both. :-) Would love to hear from
others also working on incremental update strategies. Has anybody got
a less Rube Goldberg approach than mine?

One helpful feature would be the ability to specify Second Site's
output path. I could have one path for the SCRATCH bunch and another
path for the ONLINE bunch, to minimize the chance of premature
commingling. John, is this on your wish list?

Thanks,

Chris


This thread: