As a general remark, all this looks very much like tables laid on their
side. That is, the bridging of rows with brackets, iterated bracketing,
etc. looks just like what a sophisticated table model allows for columns
and column headers.
Which suggests that looking at table DTDs might provide a useful
framework for approaching the general question.