S H O E : Simple HTML Ontology Extensions

The SHOE FAQ

Parallel Understanding Systems Group
Department of Computer Science
University of Maryland at College Park

This is the FAQ for all things SHOE. If you have a question that is not covered here or would like to add a question to the list, please contact Jeff Heflin ([email protected]).

General
The Knowledge Annotator

1. General

1.1 What is SHOE?

What SHOE is...

SHOE is an HTML-based knowledge representation language. SHOE is a superset of HTML which adds the tags necessary to embed arbitrary semantic data into web pages.

SHOE tags are divided into two categories. First, there are tags for constructing ontologies. SHOE ontologies are sets of rules which define what kinds of assertions SHOE documents can make and what these assertions mean. For example, a SHOE ontology might say that a SHOE document can declare that some data entity is a "dog", and that if it does so, that this "dog" is permitted to have a "name". Secondly, there are tags for annotating web documents to subscribe to one or more ontologies, declare data entities, and make assertions about those entities under the rules proscribed by the ontologies. For example, a SHOE document subscribing to the SHOE ontology above might then declare that it's all about a dog named "Fido".

SHOE was designed with the needs of the web in mind. It has limited semantics to make it possible to handle large amounts of data. However, simple database semantics are not enough for web data; SHOE provides true knowledge-base semantics. It has a variety of mechanisms that try to deal with the fact that the data out there is distributed and under no one's total control.

SHOE can be used to embed data from a variety of sources and for a variety of purposes. It is not intended for any one particular function. However, SHOE is primarily meant to make it possible for web robots and intelligent agents to finally make a dent in making all our lives a little easier.

What SHOE Isn't...

SHOE is not just a meta-content language. SHOE provides a relatively rich level of semantics and abilities, which enable web designers to embed documents not only with information about the overall "content" of those documents but any arbitrary information at all. SHOE also allows agents to make automatic inferences about the data they learn, provides a hierarchical categorization scheme, and a sophisticated ontology mechanism designed specifically for the web needs. SHOE tags can be used for a wide range of agent-based functions.

SHOE is purposely not a verbose knowledge-representation system. The full semantic expressiveness found in languages such as KIF are inappropriate for SHOE because their computational complexity is too high. SHOE attempts to provide as rich expressivity as possible while keeping in mind that there's a tremendous amount of data out there.

SHOE does not have any pre-defined ontologies, categories, relationships, or inferences. SHOE is a language in which categories, relationships, attributes, inferences, etc. can be defined by ontologies, but SHOE itself does not define them. This is the job of ontology designers for specific tasks or domains. However, the SHOE project does offer some initial ontologies to start things rolling, though we hope these ontologies will ultimately be superseded by other more widely-accepted ontologies.

1.2 So what's so bad with the current search systems?

Okay. We'll start with the standard example. Suppose that you have available to you the following array of state-of-the-art web navigation systems:

Word-based (or phrase-based) search engines like Lycos or Excite.
Painstaking human-made web catalogs like Yahoo.
Resource-indexing mechanisms like Aliweb.

Now suppose that you are searching the web for the home pages of a Mr. and Mrs. Cook, whom you met at a computer conference last year. You don't remember their first names, but you do recall that both work for an employer associated with the massive ARPA funding initiative 123-4567 (this initiative doesn't really exist, but you get the idea). Now, if you had a database with all of the relevant facts stored in it, and a reasonably decent query language, it'd be pretty easy to construct a query that asks for exactly what you want. Here it is in a pseudo-logic form.

Find web pages for all  x, y, and z such that
    x is a person, y is a person, z is an organization where:
        lastName(x,"Cook") and lastName(y,"Cook") and
        employee(z,x) and employee(z,y) and
        marriedTo(x,y) and involvedIn(z,"ARPA 123-4567")

So you start the web search. Using an existing man-made web catalog (like Yahoo), you can find ARPA's home page but learn that hundreds of subcontractors and research groups are working on initiative 123-4567. Searching existing web indices (AltaVista, for example) for "Cook" yields thousands of pages about cooking (in fact, AltaVista returns over 200,000 responses--try it!). Searching for "ARPA" and "123-4567" provides you with hundreds and hundreds of hits about the popular initiative. Unfortunately, searching for "Cook" and the initiative yields nothing: apparently neither person lists the initiative on his or her web page. Wandering the web on your own is fruitless.

The problem with word indices is that they associate the syntax of a word with its meaning; there's no way in general for a word index to look at the word "Cook" on a web page and realize that it's about Cook County, or about cooking, or about a person named Cook.

The problem with hand-made web catalogs like Yahoo is that the web is growing so fast, and so much information is out there, that the humans at Yahoo can't possibly keep up.

The problem with resource-indexing mechanisms like Aliweb or MCF/HotSauce is that the kind of information they store is not general enough (so far). While it's certainly possible to use the languages developed for these systems to do similar things to SHOE, at this point in time these systems cannot describe the information necessary to solve the query above, namely categories, relations, and inferences from ontologies.

Here's some more fun (and impossible) queries:

Find me an internet provider who services my area and provides the most bandwidth I can use with my modem for the lowest flat fare.
Find a four-year undergraduate college or university in a state bordering Iowa which offers Japanese classes, a computational biology major, and an ROTC program.
I'm looking for a friend. Find someone who's a lot like myself, based on what I say about myself on my home page.

1.3 Why can't we just use XML?

The Extensible Markup Language (XML) is essentially a meta-language, a language for defining other tag-based languages. This allows individuals and organizations to create tag sets that describe more than just how to display their information. However, this flexibility leads to an interoperability problem: if a university and a furniture store both use the tag <Chair>, do they mean the same or different things? What if another furniture store uses the tag <Seat>? XML DTDs can be used to ensure that a set of documents use the same set of tags, but it would be impossible to create a single DTD that describes everything! As such XML will be very useful as a business-to-business data exchange language and has potential for E-commerce, but without something built on top of it, will be insufficient for search. This is where SHOE comes in!

1.4 Why not Natural Language Processing (NLP) systems?

There are some very impressive artificial intelligence (NLP) programs out there which attempt to read English sentences and figure out what they actually mean. Many people have suggested using these programs for figuring out web pages. In the ideal world, an intelligent agent robot could go out and assist you by reading HTML pages on its own, figuring out the information contained in them, and reporting back to you.

Unfortunately, the ideal world won't be here for some time. Natural language processing still has a very long way to go, with tremendous hurdles to overcome. Additionally, the web was not only written in a human-readable language (usually English) but in a human-vision-oriented layout (HTML with tables, frames, etc.) and with human-only-readable graphics. State-of-the-art natural language technology might help with the first problem, but nothing can currently handle the other two. Don't expect robots using this technology to be of real help any time soon.

1.5 What else could SHOE be used for?

SHOE isn't just for querying databases. When used directly by web browser software, it can help users navigate web pages more rapidly, understanding the "valuableness" of a web page or site as they consider browsing it. Or SHOE could be used to store information for an agent robot to "feed" off of as it wanders the net looking for information for its master. The robot could boil this information down to relay to its master later on, or it could use the information to help it navigate better. Imagine a robot on the prowl for fruit companies. The robot comes across Apple Computer. By reading SHOE tags, the robot learns that Apple Computer is a computer company and not an apple company, which saves it several days of fruitless searching about in Apple's huge web site.

1.6 But how are you going to get all those people to annotate their web pages?

Well, first off, this is exactly the question leveled at the original HTML designers: who would want to use a weird computer language to publish documents on the internet? But in the mean time, we're designing applications like the Knowledge Annotator to make it easier for people to do this without having to muck about with the nastiness of SHOE and HTML itself. It really is quite easy. Lastly, a large percentage of web pages out there are in fact generated not by humans but by computers. Getting them to embed SHOE data along with the human-readable information they're currently writing should be easy as cake.

1.7 How do I add SHOE to my own page?

First, you must locate an applicable ontology or create one of your own. You can then add SHOE instances to your page using one of two methods:

The Knowledge Annotator. See the web page for details on using this tool.
Any Text Editor. If you are feeling brave, you can open your html file in your favorite text editor and simply type in the appropriate SHOE tags. If you decide to do this, you should look at the SHOE Specification to learn the syntax.

However, if you know the syntax, it is easy to write tools that could say, convert data from your favorite database into SHOE web pages.

1.8 What are SHOE's semantics in general?

SHOE's semantics are intentionally very simple.

SHOE Ontologies declare:

Classifications (categories) for data entities. Classifications may inherit from other classifications ("Dogs are Animals").
Valid relationships between data entities and other data entities or simple data (strings, numbers, dates, booleans). Arguments for relationships are typed, either by the simple data that can fill the argument, or with the classification a data entity must fall under in order to fill an argument ("Dogs can chase cats").
Inferences in the form of horn clauses with no negation ("If a person works for an organization, that person automatically works for any organization the organization is a sub-part of").
Inheritance from other ontologies: ontologies may be derived from or extend zero or more outside ontologies ("The SPCA ontology extends the common Library of Congress ontology").
Versioning. Ontologies may extend previous ontology versions.

HTML pages with embedded SHOE data may:

Declare arbitrary data entities. Usually, one of these entities is the web page itself.
Declare the ontologies which they will use when making declarations about entities ("I'm using the 'Pets' ontology promulgated by the SPCA").
Categorize entities ("This entity is a dog").
Declare relationships between entities or between entities and data ("This entity likes to chase that entity", "This entity's name is 'Fido'").

SHOE allows n-ary relations, horn clause inference, simple inheritance in the form of classification, multi-valued relations, and a conjunctive knowledge base. It does not currently allow negation, disjunction, or arbitrary functions and predicates.

SHOE attempts to make it difficult for entities to pretend to be other entities by providing an easily verifiable key scheme based on URLs.

Agents using SHOE should assume that declarations made by entities are claims of those entities, not simple facts. If ten people are claiming to be Marilyn Monroe's lost daughter, a SHOE agent shouldn't be storing the "fact" that Marilyn has ten children.

1.9 Is SHOE SGML-compliant?

As of version 1.0, yes. Earlier versions were not. As a result, you'll notice some significant syntactic changes between versions 0.993 and 1.0, though there have been few semantic changes to the language. Here's the SHOE SGML DTD.

1.10 Is SHOE XML-compliant?

The short answers is yes, the XML version of the SHOE DTD can be found here. However, in order to keep XML simple, practices that were common in many existing SGML applications (including HTML) are not allowed in XML. To include XML-compliant SHOE in a web page, one must first make sure that the HTML is XML compliant. See the W3C's recommendation XHTML 1.0: The Extensible HyperText Markup Language for guidelines on this process. To include SHOE in XHTML markup, simply follow the guidelines set forth in the W3C's Namespaces in XML Recommendation. Essentially, this means insert the following at the desired location in the document:

<shoe xmlns="http://www.cs.umd.edu/projects/plus/SHOE/" version="1.0">

and include the desired ONTOLOGY or INSTANCE tags between this tag and a </shoe> tag.

Due to the differences between SGML and XML, SHOE XML must obey more restrictions than the original SHOE syntax. First, all empty elements, i.e., elements which have no content and no end tag, must end with a '/>' instead of a '>'. Specifically, this applies to the USE-ONTOLOGY, DEF-CATEGORY, DEF-ARG, DEF-RENAME, DEF-CONSTANT, DEF-TYPE, CATEGORY, and ARG elements. Second, no attribute minimization is allowed. Therefore, when specifying VAR or CONST within the categories or arguments in an inference rule, the attribute name USAGE must be explicitly provided, e.g. USAGE="VAR" instead of VAR. Third, since XML is case-sensitive, all element and attribute names must be in lower case. Finally, all attribute values must always be quoted, including those which are numeric as well as the "FROM" and "TO" keywords. Future versions of SHOE are likely to recommend that all SHOE documents follow these rules.

SHOE can also be used independently of XHTML. A non-embedded SHOE XML document must begin with the following lines:

<?xml version="1.0"?>
<!DOCTYPE shoe SYSTEM
   "http://www.cs.umd.edu/projects/plus/SHOE/shoe_xml.dtd">

1.11 Where can I find a list of SHOE ontologies?

We maintain a list of SHOE ontologies. If you develop your own ontologies, we encourage you to share them with the Internet community by notifying us and allowing us to include your ontology in the list.

Where can I find other example ontologies?

There are a number of different formal ontologies written in languages other than SHOE. We provide pointers to some of those that are available on the Web.

Ontolingua Server: This service provided by the Knowledge Systems Laboratory at Stanford provides a number of ontologies contributed by various users.
Ontosaurus: This ontology browser from USC's Information Sciences Institute provides a number of ontologies written in the Loom knowledge representation language.
WebOnto: A Java-based ontology browser from the Knowledge Media Institute that contains many ontologies.
Cyc Upper Ontology: Cyc is the most famous (or infamous) ontology project, with the amibitious goal of encoding the entirety of common sense. Although Cycorp charges for full access to their knowledge base, they provide the upper-level for free. This upper-level is very comprehensive, but can be confusing even to so-called "ontologists."
Generalized Upper Model: A general task and domain independent "linguistically motivated ontology."
IEEE Standard Upper Ontology (SUO) Working Group: An attempt by a large number of ontologists to create a standard top-level ontology. This project is in-progress, but already has interim results.

2. The Knowledge Annotator

2.1 What is the difference between the application and applet versions of the Knowledge Annotator?

The applet version can be run by accessing the Knowledge Annotator home page with a Java-enabled browser. Since it is a Java applet, it is subject to the usual security restrictions of applets. As such, it can only open web pages that are on the server on which the applet resides (in this case, cs.umd.edu) and it cannot read or write to the client machine (your computer). The application version does not have these restrictions, but in order to use it, you must first download the code. Also, Java code is not compiled into native machine code, so you need a Java interpreter such as java or jre to run it.

2.2 Why won't the Knowledge Annotator applet work on my machine?

The applet should run on any browser with a correct implementation of Java 1.0. Unfortunately, most browsers have their own unique bugs in their implementations. Troubleshooting information is available here.

2.3 Why do I get the error message "Unable to start a java applet. Can't find 'java40.jar' in your CLASSPATH." when I open the Knowledge Annotator web page in my browser?

If you get this message, then you are probably running Netscape 4.0. Essentially, when the CLASSPATH environment variable (which is used by Java) is set, Netscape will only look for the core Java classes in directories specified by the CLASSPATH. If it is not set, then it looks in the default directories specified during installation. An easy workaround is to unset your CLASSPATH variable before you run Netscape. This can be done conveniently with a short script that wraps the command to execute the Netscape.

2.4 Why can't I open my web page using the Knowledge Annotator applet?

As mentioned in the question 2.1 Java applets cannot open URLs that are on machines other than the server which provides the applet. We are currently working on a proxy server mechanism, where the applet will request that the server retrieve a web page for it. This will be slow though, since all of the applet's web page requests will have to be routed through cs.umd.edu, even if they are located on the same machine with which you are browsing the Web. In the meantime, we recommend that you download the application version of the Annotator and work with it.

2.5 How can I use the applet to add tags to my own web page?

As mentioned earlier, due to Java security restrictions the applet cannot read files from any server other than ours and cannot be used to save files. Therefore, we recommend that you download the application version of the Knowledge Annotator if you wish to add SHOE to your pages. However, as long as you only use ontologies that are available on www.cs.umd.edu, there is a way to use the applet version of the annotator:

Start the applet.

Chose View | Source HTML from the menu.

Open the HTML of your web page in a local text editor.

Copy the entire contents of your text editor.

Paste into the Knowledge Annotator window.

Chose one of the other views in the Knowledge Annotator.

Use any functions of the Annotator that you wish.