Dear Sebastian,

At 07:04 AM 1/3/2007, Sebastian Rahtz wrote:
>>As for not delivering on the promise of "making their documents 
>>usable by others", I submit that it has not been demonstrated that 
>>ODD will ensure this any more than the older, less formal methods did.
>Indeed. I don't recall saying it would. I am not interested in defending ODD
>_per se_ here, I am interested in a much more formal definition of 
>to enable _some_ aspects of it to be checked by machine.

Fine, but that's exactly what I'm cautioning about -- where exactly 
do you draw the line around "some".

>>So, I think you're attacking a straw man with a broom. To my mind, 
>>the TEI is wonderful for achieving a very important kind of 
>>interchange that machines just aren't very good at
>the TEI gives us a vocabulary, yes. That's great.

It's more than great. And it's more than just the vocabulary -- it's 
also this list, plus many other invaluable resources. It's the 
important 95%. (And I also understand you're the guy most often 
tasked with the impossible 5%.)

>>... and the history of HTML shows us what the other kind of 
>>interchange is good for (not very much beyond display).
>and remember back before we had that. good gracious, would you
>like to go back to _pre_ HTML, when you consider the truly staggering advances
>of the last 15 years??? TEI is a speck of sand in the desert compared to what
>HTML has achieved.

That's a very good point: and one of the most important insights of 
these fifteen years has been that the markup language suitable for a 
display vocabulary is not (beyond details of syntax that are actually 
hugely important for tools development) actually much like the markup 
language suitable for other kinds of processing (esp. 
search/retrieval and analysis) promised by generic markup. These are 
on different layers and each layer benefits tremendously from being 
able to distinguish itself and direct its design to its own long- and 
short-term requirements. Indeed, without artificial intelligence (in 
this case specifically, an automated capability to detect and 
register higher-order semantics from presentational features in 
something like the way human beings do), discriminating the layers 
like this is the *only practical way* we have found, I submit, to get 
computers, on a broad scale, to do more than simply emulate and 
automate other media technologies -- the "glorified typewriter", if 
you like, or "home linotype". Not that glorified typewriters are a 
bad thing, mind you. But in combination with lower layers of truly 
descriptive markup, standardized display technologies are something 
else entirely and something far better and more powerful.

Yet what we then find is that at layers of greater semantic richness, 
"universal" languages take away as much as they offer. 
Standardization comes at the price of expressive power. This is an 
old lesson and can be studied in a wide range of different realms of 
culture and media studies.

I think where we differ is in what kind of expectations we have for 
standardization at the different layers. My focus is on a language, 
if you will, that allows great freedom, local expressiveness and 
variation. I take it for granted that individual projects and 
"subcultures" within TEI should have their own stylesheets in order 
to achieve their own goals and application requirements -- on the 
basis of strong standards in markup syntax, validation technologies 
and methods, and so forth (the critically important laws that enable 
freedom). Whereas you are in the position of arguing, it seems to me, 
that where the tradeoff is between local expressiveness and 
generalized processing, generalized processing should win -- to serve 
the abstract goal of "interchange".

Yet I also believe that my view of it is more in keeping with the 
Poughkeepsie goals of 1987 than yours. In fact, I feel that the 
battle the TEI formed itself to fight has been won -- with P3/P4, the 
wealth of TEI projects that have resulted from it, with XML and XSLT. 
Just look at what TEI set out to achieve back then, and this becomes 
clear: TEI is now *the* gold standard for scholarly text encoding. 
Even projects that don't use TEI tagging benefit tremendously from 
its methods and from the examples set by TEI projects. But beyond 
markup syntax, platform-independent character sets and standards for 
metadata (the important requirements of the 1980s), which enable me 
to do useful work with your data (a different thing from running your 
input through my stylesheets unaltered), it seems to me that further 
advances in the direction of unifying practice go quite beyond TEI's 
original mandate, and run the risk of compromising exactly what makes 
TEI so good -- the range of variation and promotion of *scholarly* 
(not technical) goals it allows. Aiming for completely transparent 
interchange across arbitrary and open-ended applications seems to me 
like setting out, once one has successfully colonized a new 
continent, to conscript its inhabitants and conquer the moon. Why not 
develop the continent we have found? Why not encourage local cultures 
to flourish on it -- even, where useful and meaningful, to draw 
boundaries among themselves and trade across them?

ODD, it seems to me, would be a fine basis for this, were it not 
being treated as the last word on the topic of modeling.

Which brings me back to another important, though subtle point. 
However much we enjoy dramatizing our differences here, we're not 
actually all that far apart. You concede at least, I take it, that 
bugs are bugs and should be fixed. I concede that ODD is a worthy 
touchstone even if not the last word. Where we differ is most of all 
in the rhetorical stress we attach to ODD. I would like to see it 
recommended as a tool, perhaps the best available at least for the 
neophyte -- but no more. Others want it to be more (I'll resist the 
impulse to caricature the position again).

>  In practice, TEI documents are not _even_ interchangeable
>for display. Think back on the last time you tried to combine TEI docs from
>several sources - aargh. I can't even straightforwardly combine documents
>for display written by myself, Lou and James from the same building, 
>because they each have their own dialect of TEI.

Which is a great and important lesson, it seems to me. Perhaps the 
three of you could standardize your own local dialect and then give a 
conference paper presenting your work and offering your 
quality-checking methods and stylesheets for others to assess and build on.

>>Indeed. But I've been arguing preemptively and hypothetically, 
>>since I think it would be a mistake for the Technical Council to 
>>rule, without offering any alternatives, "it's ODD or bust" -- 
>>which, I imagine, would probably lead to more occasions of "bust" 
>>than they'd like to see
>I think you're scare mongering, to be honest. My expectation of ever 
>more formal
>conformance rules is that an increasing number of people use the TEI.

Unfortunately there will be no way to tell. One hopes there will be 
an increasing number of TEI users, but why they do will always be 
debatable. Yet more importantly, I think there is a difference here 
between "formal" and "formal". I am in favor of clearly documented 
processes, procedures, checklists, explicit criteria for when 
variances are welcome and when they are gratuitous and unwelcome, 
indications of what users and projects can expect when making choices 
between tradeoffs in the design process. I am not, however, in favor 
of appointing a machine to be the final arbiter, however useful it is 
as a diagnostic device. You yourself have admitted that we do not 
have that machine.

>>-- or perhaps more to the point, of potential TEI users and allies 
>>just deciding to jump on other bandwagons.
>It's not an "either or" thing. Since you raise the analogy of taking 
>your football
>away, what I am interested in is not banning whatever sport you want to play
>in the local park, but in being clear what the games are. If I come upon your
>game with bat and ball, I want a clear sign saying "we are playing to the
>rules of american baseball", otherwise I'll think its proper rounders and may
>want to join in. If you want to invent a new game, to distinguish 
>yourself for a tenure
>committee, then that's great, but it's only fair to everyone if you 
>write out those
>rules in a simple, standard, way. Yes, you can define your game by 
>just drawing
>diagrams with stick people, but why raise the artificial barrier when everyone
>else uses words?

Indeed. It kind of lost me. Analogies aside, ODD (or its current 
implementation, as you prefer) breaks when an expert user tries to 
drop numbered divs from his model (a simple, straightforward and 
easily defensible requirement) and generate a DTD -- but this expert 
user is not allowed to say "okay, I'll write my rules without ODD" 
and remain in conformance. There's a problem there.

>>(It would be a different thing altogether for them to rule "we much 
>>prefer and recommend ODD, but if there are reasons you can't use 
>>it, technically or practically, we need to learn from these too".)
>it's a fine idea. but I haven't heard the technical or practical
>arguments which make use ODD to express your customization
>_impossible_, just sentiment.

I'm really saying nothing more than that I suspect this is a 
significant indicator that an escape clause allowing for conformance 
without an ODD needs to be entertained ... and frankly, at least 
until you fix the little bug, I see little reason to dismiss my own 
suspicions. (Little bugs are innumerable, but are only fixed one at a time.)

>>Many projects have picked up TEI only to let it drop again -- or 
>>have twisted it to the point where they don't want to show their 
>>work in what they think of as polite (or over-polite) society. TEI 
>>could learn a great deal if it could see into this blind spot. 
>>Indeed, TEI is good work precisely to the extent that its 
>>developers and proselytes have said "it's not finished yet; we have 
>>more to learn".
>No disagreement there, but what does this have to do with the 
>(separate!) arguments
>about conformance or ODD?

The burdens of "TEI conformance" in view of the perceived stresses 
(sometimes more perceived than genuine) between TEI practice and 
local processing requirements is the driving factor behind losing 
these potential constituents.

In fact, as I understood it, this was a problem that ODD was designed 
to address, not to exacerbate. (I guess this goes to show again that 
purpose can't be built into a tool.)

>>   While Ron doesn't have one of those, he has something even 
>> worse: one impossible to express in the TEI DTD architecture as it 
>> stands due to the expansion of its parameter entities.
>parameter entities in DTDs are a technical detail of the implementation, note.
>>This implies that in order to support conformant ODD fully, an ODD 
>>implementation would have to work around the XML modeling 
>>architecture it is based on.
>I think you're twisting things here. The TEI sources express themselves in
>RELAXNG notation and in the class system. Anything Ron does in his 
>will come out into a RELAXNG scheme and work (I hope). The small technical
>problem lies in converting RELAXNG schemas to DTD (and W3C schema) without
>making the result invalid. The formal amongst you (Syd, we need you!) can tell
>me whether it is technically impossible to convert any arbitrary 
>RELAXNG schema
>to DTD;

It depends on whether you are allowed to relax your RelaxNG 
constraints. If by "convert" you mean "express the same or somewhat 
looser models" the answer is, you always can. If you mean "express 
the same models", you can't.

Here's a model in RelaxNG (a useful one) impossible to express in DTD syntax:

element name { text?,
                element index { text },
                text? }

(meaning a "name" element is text mixed with one and exactly one 
"index" element)

The closest you can get to this in DTD is to give the "name" element 
mixed content and allow the "index" element to appear inside it zero 
or more times.

If a project had to use a DTD and needed this model, I'd advise them 
to validate their constraint on uses of "index" inside "name" (to 
single occurrences) in a separate layer (Schematron or other).

>if it is not, then we just have to program it; if it is, then we'll have to
>drop DTD. After all, we're not the only people in this bind.

By that, I take it you mean the bind of supporting both DTD and RNG. 
(If so, you're quite right.)

But the requirement to drop numbered divs from the model -- Ron's 
case -- doesn't fall into this category. DTDs can express that model 
just fine, and have. Indeed that's exactly what Ron has done outside ODD.

A further irony is that allowing div0, div1 and div2 etc. at all in 
P5, alongside unnumbered divs with exactly equivalent tagging 
semantics, while  simultaneously urging conformity of tagging 
practice in the name of standardizing an application profile, is in 
itself a concession to local idiosyncracy that boggles the mind.


Wendell Piez                            mailto:[log in to unmask]
Mulberry Technologies, Inc.      
17 West Jefferson Street                    Direct Phone: 301/315-9635
Suite 207                                          Phone: 301/315-9631
Rockville, MD  20850                                 Fax: 301/315-8285
   Mulberry Technologies: A Consultancy Specializing in SGML and XML