I'm currently transforming some texts which were prepared by
someone else into something approaching TEI-conformant SGML, and
expect to be doing quite a lot of that sort of thing over the
next year or two. Most of the job is fairly straightforward --
what the person who prepared the text in the first place and what
the TEI proposes should be encoded are not a million miles apart
(if they were something would be seriously wrong somewhere) --
and involves little more than some string twiddlings, for which
languages like Icon or Snobol are perfect (I use the latter
though if I were young again I'd use the former). But
occasionally...
 
For example, here's a problem which has just turned up on which
I'd appreciate comments from the collective wisdom, and of which
I should like to warn the collective unconscious.
 
The texts on which I am working were originally prepared for a
concordance. Consequently, they have a very detailed reference
scheme (which I can handle) and also go to some trouble to
distinguish homographs. This is done by adding a coded suffix to
a fairly haphazard selection of words, some 8% of the total of
different words in the text I'm looking at, some 12% of the
running text length. For example, `associate' (noun) appears as
`associate$0$', `associate' (verb) as `associate$1$'. `$0' always
follows a noun (but not every noun, by any means), `$1' always
follows a verb and so on. The tag also distinguishes senses or
other subdivisions for some words: thus `ball', noun, in the
sense of a spherical object, appears as "ball$0#1$", and `ball'
as a social gathering as "ball$0#2$".
 
It's important to realise that these tags are not intended to
provide a full blown linguistic analysis -- there are only nine
categories, of which the last two are "idiom/fossil/collocation"
and "infinitive particle or mixed categories". They are only
there to distinguish homographs. `Bath' (as a proper name) gets a
tag to distinguish it from `bath' as a common noun -- but no
other place names are tagged. So neither the TEI tags for
linguistic analysis nor the tags for place names seem
appropriate.
 
My question is: what shall I do with these tags? There seem to be
four possibilities:
 
1a. Throw them away
 
1b. Ignore them i.e. just leave them in the text as funny looking
tokens which the application will have to sort out as best it can
(They will of course be documented in the TEI.Header, so what
more could you ask).
 
2. Tag the word or phrase to which they belong as a distinct
segment (I suppose the S tag would do for this), including their
value on a suitable attribute. Something like this:
     <s category='0#1'>ball</s>
This would involve defining a new attribute of course, with a
default value of `unspecified'.
 
3. Represent the word plus its disambiguating tag as an entity.
Something like &ball01; perhaps, which could be defined simply as
"ball", if the distinction is not be kept, or some other string
if it is.
 
1a.  seems a shame: for some applications (such as making a word
index) the disambiguating tags are very useful.
 
1b. is the easiest course of action but feels unwholesome
 
2. looks like overkill and moreover invites the question as to
why only some words or phrases get segmented in this way
 
3. would be easily the most satisfactory if there weren't quite
so many entities to define -- about 500 in all
 
Any ideas or counter-suggestions gratefully received.
 
Lou Burnard
 
(wearing Oxford Text Archive hat, rather than TEI one)