For the Technically Curious (How It Works) ...

"Script-Driven":

At its core, ChimneySweep® is a script-driven system. It implements a fully-featured programming language, loosely based on Paradox for Windows® ObjectPAL,™ and makes the low-level functionality of the system available to that language. Meanwhile, the entire high-level functionality of the system is implemented in that language:

• ChimneySweep "jobs" are programs ("scripts").
• Jobs work by setting up data-structures corresponding to the job's settings, then invoking a "system script."
• Jobs are built, in source-code form that is then compiled, by another script – the "builder script."
• The version-6.0 "job catalog," and the recording of job results as they run, is handled by yet another script (library).

The "system script," as it has evolved over these many years, now consists of more than 3,500 lines of source-code, now spread over a number of files, yet it continues to be backward-compatible with the very earliest releases of the system. Believe it or not, a job created by ChimneySweep 1.0 still runs today, although of course today it runs in a vastly different manner.

Professional Edition users (only ...) are provided with the underlying source-code to all of these scripts, as well as several hundred pages of thorough technical documentation. They are also provided with an "API" binary library, and several distinct ways to access ChimneySweep functionality within their applications.

Users of all other Editions simply use so-called "delivered" versions of these scripts, which omit the source-code. (But then again, they don't care.)

"Very Simply, Better Algorithms":

The most-fundamental difference between ChimneySweep and TUTIL32 is that ChimneySweep undertakes to solve problems either by repairing them in-place, or by replacing them outright (using faster and more-efficient algorithms which it knows that it can apply in this special case).

The approach that was taken by TUTIL32 – and by its DOS-based predecessor – was one that was "always known to work," never-mind that it might be crushingly slow:

1. Use the information in the bad table's header (if it was determined to be valid) to construct a new temporary table with the same structure.
2. Scan through the bad table, one physical block at a time, and insert all of the apparently-good records within all of the apparently-good blocks into the new table, (of course) one record at a time.

While this "ultimately conservative" approach could of course "be guaranteed to work," it presented two obvious problems:

1. "An Awful Lot Of Copying!" The original table was never considered to be "a file that could, in fact, be fixed." Instead, the file was duplicated – one record at a time.
2. "A Prohibitive Cost 'Per Record'!" Like most database systems, Paradox (and xBase) use B-Tree index structures which incur a variable (and sometimes, very expensive) cost per record-insert/delete in order to maintain near-optimal performance for record access, which quite-understandably is assumed to be the most important consideration. However, if you find yourself now throwing hundreds of thousands of records against these algorithms, all at once, this cost becomes prohibitive.

ChimneySweep therefore re-considered the entire problem, and subdivided it into two categories: those which could be "repaired in-place," and those which called for "replacing the entire data-structure, by more-efficient means."

• Repair In-Place: Each "Paradox-formatted file" is fundamentally a linked-list data structure, with singly or bidirectionally-linked lists as the case may be. Therefore, any anomalies in this data structure can be repaired by specific modifications to the file's content – without replacing it.
• Replace the Entire Structure: This applies generically to all forms of index files. The first order of business is to determine – as quickly as possible – whether or not any(!) portion of the index is incorrect. Then, having determined this, the fastest solution is to replace the entire index structure, in its entirety.

A crucial – and, fully exploitable – aspect of the "replace the entire structure" concept is that "this is not 'a dynamic situation.'" We are not dealing with "an entirely-new, therefore entirely-unpredictable, brand-new key value." Instead, we are merely dealing with an existing set of key-values that are known to be unchanging. This opens-up an entirely new set of considerably more-efficient algorithms, which ChimneySweep implements.

And so, perhaps, "it really all comes down to this. Very Simply, Better Algorithms."