I would appreciate some advice on a pilot project to create TEI-Lite
like transcriptions from OCR texts of novels from the 18th to the
early twentieth century. The basic idea is to create manipulable and
interoperable transcriptions that lend themselves to linguistic
annotation but maintain a connection to the layout of the printed
source. The resultant procedures should be simple enough for
individual users to create editions of their own with appropriate
guidelines. In some ways it's a kind of "Project Gutenberg plus."
There was some talk of TEI versions of Gutenberg texts, but nothing
seems to have come of it so far.
The procedures start from an algorithmically produced TEI version
that is derived from an algorithmically produced 'white space XML'.
The source text in every case is the equivalent of what an old-
fashioned gardening book called 'shrubs of merit', first editions or
other public domain editions that for one reason or another have
standing as 'good enough' texts. There is no ambition to capture all
minutiae of typography or layout. But I assume that users will find it
helpful to be able to align a line of transcription with a line of
printed text. On the other hand, soft hyphens can be ignored, and the
second part of a hyphenated word will be added to the line where the
Lines with running heads will be ignored, as will be lines that carry
information that have more to do with where the paper came from than
with the text, such as "B2" and the like.
Are these reasonable principles? The trickiest business involves
hyphenation at line or page breaks. Some encoding projects ignore line
breaks. Others observe hyphenation when it occurs at the end of the
page but ignore it at the end of a line. Hyphenation is not a problem
if you think of a text as something to be displayed for readers. If
you tokenize or add linguistic annotation, hyphenated words create
problems of a solvable but pesky kind. It is certainly simpler to
deal with word tokens if you can assume that a line will always
consist of whole words only, and you do not really interfere with the
reader's abiliy to align the transcription with the facsimile, which
is a non-trivial benefit. But what analytical or critical affordances
are sacrificed by ignoring hyphenation?