External entities fall into two categories: text entities and data entities. XML supports only text entities in content. SGML supports both text and data entities. Different processing is required for text and data entities.
An external text entity is an entity whose replacement text forms part of the document being parsed. You must resolve an external entity and feed its replacement text to the parser so that it is parsed as part of the current document.
To resolve an external text entity you can write an external-text-entity rule. This rule will be fired whenever the corresponding entity occurs in the document. When an external-text-entity rule is fired, a new output scope is created. The destination of this output scope is the parser. This means that you can send the replacement text of the entity to the parser simply by outputting it in the external-text-entity rule.
Here is the simplest case of an external-text-entity rule. This rule simply outputs a replacement string
for the entity company-name:
external-text-entity "company-name" output "Stilo International"
In most cases, however, you will want to use a system or public identifier to locate the replacement text. A system identifier is a reference to a data source in system-specific terms. In XML, system identifiers are URIs. The public identifier is an abstract string that can be mapped to a system identifier on a system by system basis.
You can retrieve the system identifier of an entity with the format string %eq and the public identifier with %pq. You can retrieve the name of the entity itself with %q. This example uses the system identifier to retrieve an entity definition:
external-text-entity #implied local stream replacement-text-file-name set replacement-text-file-name to "%eq" drop (ul "file://") do when file replacement-text-file-name exists output file replacement-text-file-name else output "<? the SYSTEM identifier for entity %q references a file that does not exist ?>" done catch #program-error output "<? the SYSTEM identifier for entity %q does not use a 'file://' URI ?>"
This rule is written to support only system identifiers that are files on the local system. To support a full range of URIs, you must write code to determine the type of URI used in a system identifier and retrieve the replacement text from that location.
Since the location of entity replacement files can vary from one system to another, you may want to use public identifiers to reference system identifiers appropriate to the local system. In the following code, the public identifier is used to index into a shelf of system identifiers to retrieve the replacement text:
global stream system-identifiers variable external-text-entity #implied local stream replacement-text-file-name set replacement-text-file-name to system-identifiers{"%pq"} drop (ul "file://") do when file replacement-text-file-name exists output file replacement-text-file-name else output "<? the SYSTEM identifier for entity %q references a file that does not exist ?>" done catch #program-error output "<? the SYSTEM identifier for entity %q does not use a 'file://' URI ?>"
This code assumes that you have initialized the shelf system-identifiers with system identifiers as items and public identifiers as keys. Since you control the system identifiers that are loaded onto the shelf, you can map public identifiers to any set of system identifiers that are appropriate for the system your program will run on.
Notice that the code above presumes that the system identifiers on the system-identifiers shelf are plain system file names, not URIs. If all the resources in your catalog are local files, there is no point in adding full URI syntax, just to strip it out again.
Instead of creating your own shelf to do the mapping, such as system-identifiers in the code above, you can use the built in OmniMark shelf #library. There are two advantages to using #library rather than declaring your own shelf:
library declaration.
#library shelf. In many cases you will be able to avoid writing an external-text-entity rule and rely on OmniMark's default behavior.
The following sample shows the code above rewritten to use #library and a library declaration:
library "-//ES//DTD for foo//EN" "foo.dtd" "-//ES//TEXT for bar//EN" "bar.txt" "-//ES//TEXT for baz//EN" "baz.txt" "-//ES//TEXT for bat//EN" "bat.txt" external-text-entity #implied do when file #library{"%pq"} exists output file #library{"%pq"} else output "<? the SYSTEM identifier for entity %q references a file that does not exist ?>" done
Usually, mappings of public identifiers to system identifiers are maintained in files called catalogs. The use of catalogs simplifies the maintenance of entity management.
The simplest way to create a catalog is to place an OmniMark library declaration in a separate file and then include it in any program that needs it. If you want to avoid hardcoding catalog references into your program, you can reference a catalog file using the -library command-line option, which is described in the OmniMark Engine documentation.
There are other catalog formats that you can use. The SOCAT catalog format is a standard for use with SGML, whereas OASIS catalogs are common with XML; the omoasiscatalogs can be used to process OASIS catalogs. You may need to use one of these catalog formats if you are using other XML or SGML tools that use a particular format. To use another catalog format, simply omit the library declaration and write your own function to load the #library shelf by scanning the catalog.
You can use plain file names rather than whole paths as system identifiers on #library. You can then specify the paths that OmniMark will search for these files by adding them to the #libpath shelf. You should end all such paths with a directory separator (usually a / or \) as the built-in entity manager will form paths by appending the system identifier to each entry on the #libpath shelf, without interpreting the strings in any way. The following code sample uses #libpath to help resolve public identifiers:
declare catch entity-replacement-found external-text-entity #implied do when entity is in-library do when file #library{"%pq"} exists output file #library{"%pq"} else repeat over #libpath do when file (#libpath || #library{"%pq"}) exists output file (#libpath || #library{"%pq"}) throw entity-replacement-found done again output "<? No definition for entity %q ?>" done catch entity-replacement-found ; EMPTY done
You can add items to the #libpath shelf with an ordinary set new statement or with the -libpath command-line option, which is described in the OmniMark Engine documentation.
If you have an XML document that references a DTD with a system or public identifier, you can write an external-text-entity rule to retrieve the DTD. The only difference from the examples above is the form of the external-text-entity rule itself. The external-text-entity rule header must refer to the DTD entity using the keyword #dtd:
external-text-entity #dtd
Note that an external-text-entity #implied will fire all normal entities but does not fire for an external DTD. You must write a separate external-text-entity #dtd rule to retrieve an external DTD.
If all your entity replacement files are local (rather then being retrieved from the web), you can take advantage of OmniMark's default entity manager. If you don't put any external-text-entity rules in your program, OmniMark will provide a default external-text-entity and compile it into your program for you. The source code for this rule is provided below.
The built-in entity manager assumes all the system identifiers it sees are local file names. It makes use of #library and #libpath, so you can load the appropriate values into those shelves to get the behavior you need.
In SGML, system identifiers are not required to be URIs. In fact there is no specific requirement about their contents, though it is common practice that they are file paths on the local system. This makes the basic code for retrieving an entity definition by its system identifier even simpler:
external-text-entity #implied do when file "%eq" exists output file "%eq" else output "<? the SYSTEM identifier for entity %q references a file that does not exist ?>" done
Apart from this difference in system identifiers, resolving entities in SGML works the same as resolving them in XML.
You can resolve external data entities by writing an external-data-entity rule. An external-data-entity rule behaves just like an external-text-entity rule except that no new output scope is created. Output in an external-data-entity rule goes to the existing output scope.
You can use exactly the same methods to resolve system and public identifiers as you do with an external-text-entity rule. However, the built-in entity manager is not active for external data entities.
How you process the data referenced by an external data entity is entirely up to you.
A special case of an external data entity is a SUBDOC entity. A SUBDOC entity references another SGML document that does not form part of the text of the current document, that may conform to a completely different DTD, but which shares the SGML declaration of the document that references it.
To process the document referenced by the SUBDOC entity, you use the do sgml-parse action. To cause this parse to inherit the SGML declaration of the outer parse, you specify subdocument instead of document in the do sgml-parse statement:
external-data-entity #implied when entity is subdoc-entity do sgml-parse subdocument scan file "%eq" output "%c" done catch #external-exception output "<? the SYSTEM identifier for entity %q references a file that does not exist ?>"
There are a number of standard entities that are referred to by public identifiers in most SGML declarations. OmniMark predefined these entities to avoid requiring all users to provide definitions for these entities. The definition for these entities is provided on the #libvalue shelf, which is used by the default entity manager. You can add or change items on the #libvalue shelf if you need to.
OmniMark's default entity manager is an external-text-entity rule that is compiled into any module
that does not specify any external-text-entity rules itself. Below is a sample
external-text-entity rule that behaves much like OmniMark's built-in rule. You can use this code to better understand the default entity manager, or use it as a base for
writing your own entity manager:
external-text-entity (#implied | #dtd | #charset | #capacity | #syntax) local stream file-name do when entity is (system | in-library) do when entity is system set file-name to "%eq" else when entity is in-library set file-name to #library{"%pq"} done do unless file file-name exists repeat over #libpath do when file ("%g(#libpath)" || file-name) exists set file-name to ("%g(#libpath)" || file-name) exit done again done do when file-name is attached output file file-name else put #error "File for " do when entity is (#dtd | #charset | #capacity | #syntax) put #error "%q (%g(#doctype))" when entity is #dtd else when entity is general put #error "entity &%q;" else put #error "entity %%%q;" done put #error " with public id public %"%pq%" " when entity is public put #error " does not exist!%n" halt done else when entity is public & #libvalue has key "%pq" do when #libvalue{"%pq"} is closed output #libvalue{"%pq"} else put #error "Stream #libvalue for key '%pq' must be closed before emitting contents.%n" done else when entity is (#charset | #capacity) ; EMPTY ... zero-length entity replacement text. else when entity is public put #error "Public identifier for " do when entity is (#dtd | #charset | #capacity | #syntax) put #error "%q" put #error " (%g(#doctype))" when entity is #dtd else when entity is general put #error "entity &%q;" else put #error "entity %%%q;" done put #error " public%"%pq%" is not in the library rules!%n" halt else do when entity is #dtd put #error "#dtd (%g(#doctype)) " else when entity is general put #error "Entity &%q;" else put #error "Entity %%%q;" done put #error " has neither a system nor a public identifier!%n" halt done