SUO: RE: Some thoughts on SUO conformance
Dear Frank,
First, let me say I think this is a good exposition of
what we need to think about.
Now let me answer you question - what does conformance
mean for some eventual SUO.
The SUO essentially assigns specific
meanings to strings of characters, as defined by the
axioms in the eventual SUO.
Therefore one form of conformance means using
these strings with these meanings with nothing either
added or taken away.
There will be other forms of conformance (as in compiler
vs program in a language).
Matthew West
Principal Consultant
Shell Information Technology International Limited
Shell Centre, London SE1 7NA, United Kingdom
Tel: +44 20 7934 4490 Other Tel: +44 7796 336538
Email: matthew.r.west@is.shell.com
Internet: http://www.shell.com
> -----Original Message-----
> From: Frank Farance [mailto:frank@farance.com]
> Sent: 03 October 2001 15:49
> To: Standard Upper Ontology
> Subject: SUO: Some thoughts on SUO conformance
>
>
>
> I'm certain this will be a contentous topic. I don't have
> particularly strong opinions towards any particular
> conformance paradigm (several are possible), but I figured I
> would get the discussion started on this topic.
>
> Some background information:
>
> - Generically, a standard is a specification (a
> standard is developed by an accredited standards development
> organization (SDO), while a specification is developed
> outside accredited SDOs).
> - Most standards specify requirements.
> - The requirements are described in standards
> wording, a particular style of writing (can look like legal writings).
> - There are two types of wording in a standard:
> - Normative wording (describes requirements)
> - Informative wording (supportive text that
> helps the reader understand, e.g., notes, footnotes, etc.).
> - Typically, normative wording is one of the following:
> - Scope: describes the applicability of the standard
> - Normative references, which "incorporate by
> reference" other standards and specifications (think of it as
> an "#include" statement in a standard)
> - Definitions: terminology that is necesary
> for describing the requirements of the standard
> - Assertions: statements that describe requirements
> - Inquires/Ranges: a special kind of
> requirement that permits a range of implementations, or
> permits an application (use) of the standard to adapt to an
> implementation
> - Negotiations: a special kind of requirement
> that permits implementations to adapt to other
> implementations (e.g., modem negotiation)
> - Assertions (and inquires, ranges, negotations,
> etc.) may vary from strong to weak:
> - The verb "shall" is used to indicate a
> strong (mandatory) assertion
> - The verb "should" is a weaker assertion,
> indicating a preference or a recommendation (recommended practice)
> - The verb "may" is the weakest assertion
> (makes no requirements)
> - Assertions that indicate prohibitions (another kind
> of requirement) use "shall not" and "should not". Standards
> writers need to be careful when using "may not" because it
> might be misinterpreted.
> - The standard is partitioned into clauses. Usually,
> the first four clauses are:
> - Clause 1: Scope and Purpose
> - Clause 2: Normative References
> - Clause 3: Definitions and Abbreviations
> - Clause 4: Conformance
> - Standards writers must be careful not to include
> assertions in the Definitions Clause.
> - The remaining Clauses are organized in a way that
> (1) best expresses the requirements of the standard, and (2)
> assures consistent and full intereptation of the standard.
> - Annexes may be included. Annexes may be normative
> or informative. The Clauses define the "main part" of the standard.
> - The success of a standard is largely dependent on
> how well the "market" adopts the standard ("market" might
> mean users, vendors, institutions, governments, etc., and
> varies from standard to standard).
> - Market adoption is dependent on several factors
> (not a complete list):
> - Functionality: Is the standard useful? An
> incomplete or poorly specified standard is less likely to be adopted.
> - Errors: If there are many "bugs" in the
> standard, then there may be significant mistakes in
> implementation and use (e.g., compatibility and
> interoperability problems).
> - Interpretation: A standard is read and used
> by users, vendors, institutions, etc.. If it is difficult to
> interpret the standard or interpretations are inconsistent
> (e.g., due to poor wording or staff changes in the
> committee), then the standard is weakened because there is
> less agreement on its requirements.
> - Conformance: A standard is less useful if
> we can't determine which "implementations" meet the
> requirements of the standard.
> - In the maintenance phase of standards (we are in
> the development phase), we will be required to handle defect
> reports (fixing errors) and formal interpretations of our
> standard (questions about ambiguities).
> - Usually, standards committees have a difficult time
> determining how well the market will adopt their standard.
> Many times, the success/failure of a standard is understood
> 4-7 years after its development. Using "best practices"
> (tried and true experience from standards committees/history)
> is one way to help reduce the "risk" associated with market
> adoption. Best practices are techniques that reduce risk,
> but they cannot assure the success of a standard.
> - Tips for improving market adoption:
> - Functionality: scoping, coherence, and
> completeness (in the lay sense of "complete") are three main
> features. Scoping: getting the boundaries right: What's in
> scope, out of scope. Coherence: Simply, the document needs
> to make sense as a whole. Completeness (lay sense of
> "complete"): Within the scope, the requirements should fully
> describe the scope, but should be "just enough" ==> don't
> overspecify, don't tell people *how* to implement their system.
> - Errors: Like software, it is much cheaper
> to fix the problem earlier than later. For standards, the
> "cost" differential is significantly higher ==> it is much
> much much more expensive to fix problems later, e.g., after
> publishing the standard. Like software, too, perfection is
> not required (it will take forever), so the committee needs
> to have a sense of "what is good enough".
> - Interpretation: Many standards add (brief)
> clauses on architecture, conceptual model, etc., because they
> help with the interpretations later on. Standards wording is
> hard stuff ... having a (say) conceptual model can be a great
> help during interpretations. An annex (or seperate document)
> on Rationale (the thinking behind the committee's decisions)
> can really help with interpretations ... and can help with
> newcomers (gets them up to speed) and attrition (fewer
> dependencies on participants that leave).
> - Conformance: Need to develop a conformance
> clause early. A reasonably accurate statement is: you don't
> understand a standard until you understand its conformance.
> - Conformance: Typically, an implementation conforms
> to a standard when it satisfies all of the requirements
> (assertions, etc.) of a standard.
> - An "implementation" of a standard is an entity
> (e.g., system) that claims conformance to the standard.
> Typically, "implemenations" state these conformance claims
> with an "implementation conformance statement" (ICS).
> - Usually, the "owner" of "implemenation" makes the
> conformance claim.
> - Conformity assessment is the measurment of how well
> an "implementation" meets the requirements a standard.
> - Conformity assessment may be used to validate an
> ICS. Conformance tests and other methods may be used to
> perform conformity assessment.
>
> -------------------------------------------
>
> OK, everything above is pretty generic. Above, I've included
> related issues (e.g., interpretations, standard wording)
> because I'm sure they will come up in discussions of conformance.
>
> Now to the specifics ...
>
> In many standards, committees develop a "conformance
> paradigm": for example, there are various "roles" of
> implementations. Here's some samples:
>
> - Power source vs. electrical appliance: The
> conformity assessment (conformance measurement) of a
> 110-volt, 15-amp power source is different than the
> conformity assessment of a 110-volt, 15-amp appliance.
> - Client vs. server: In network and communication
> protocols, clients and servers have different
> "understandings" of conformance.
> - Programming language vs. program: One can define a
> programming language standard, but a conforming compiler is
> different than a conforming program.
>
> [Note: Conformance paradigms aren't aslways 2 parties ... the
> above are familiar paradigms.]
>
> A main purpose of understanding and developing a conformance
> paradigm is to support many "implementations" while balancing
> the needs of interoperability and compatibility. Although it
> might be possible to develop a separate standard for each of
> the "roles", it is impractical for many reasons: why develop
> a standard for compilers for the language X and another
> standard for programs of the language X when >97% of the
> wording is the same?
>
> So what is the conformance paradigm for SUO? I'm not sure,
> but I have some ideas ... I really would like to hear what
> the group thinks about this.
>
> I think the PAR is a good starting point:
>
> SCOPE
>
> This standard will specify an upper ontology that
> will enable computers to utilize it for applications such as
> data interoperability, information search and retrieval,
> automated inferencing, and natural language processing. An
> ontology is similar to a dictionary or glossary, but with
> greater detail and structure that enables computers to
> process its content. An ontology consists of a set of
> concepts, axioms, and relationships that describe a domain of
> interest. An upper ontology is limited to concepts that are
> meta, generic, abstract and philosophical, and therefore are
> general enough to address (at a high level) a broad range of
> domain areas. Concepts specific to given domains will not be
> included; however, this standard will provide a structure and
> a set of general concepts upon which domain ontologies (e.g.
> medical, financial, engineering, etc.) could be constructed
>
> PURPOSE
>
> A. AUTOMATED REASONING: The standard will be suitable
> for automated logical inference to support knowledge-based
> reasoning applications.
>
> B. INTER-OPERABILITY: The standard will provide a
> basis for achieving Inter-Operability among various software
> and database applications.
> 1) Application developers can define new data
> elements in terms of a common ontology, and thereby gain some
> degree of interoperability with other conformant systems.
> 2) Applications based on domain-specific
> ontologies that are compliant with this standard will be able
> to interoperate (to some degree) by virtue of the shared
> common terms and definitions.
> 3) The SUO will play the role of a neutral
> interchange format whereby owners of existing applications
> will be able to map existing data elements just once to a
> common ontology. This provides a degree of interoperability
> with other applications whose representations conform to SUO.
> This entails the SUO being able to be mapped to more
> restricted forms such as XML, database schema, or object
> oriented schema.
>
> C: APPLICATION AREAS
> 1) E-commerce applications from different
> domains that need to interoperate at both the data and
> semantic levels.
> 2) Educational applications in which students
> learn concepts and relationships directly from, or expressed
> in terms of, a common ontology. This will also enable a
> standard record of learning to be kept.
> 3) Natural language understanding tasks in
> which a knowledge-based reasoning system uses the ontology to
> disambiguate among likely interpretations of natural language
> statements.
>
>
> Here's what I extract (one person's opinion) from the PAR
> regarding conformance.
>
> "This standard will specify an upper ontology that
> will enable computers to utilize it for applications such ..."
>
> It's clear that there will be applications of the standard
> (not necessarily applications in a programming sense) and
> they will be dependent upon the SUO [for something]. My
> guess is that these applications will want to "use" SUO is
> some conforming way (i.e., claim conformance to the "use" of the SUO).
>
> "An upper ontology is limited to concepts that are
> meta, generic, abstract and philosophical, and therefore are
> general enough to address (at a high level) a broad range of
> domain areas."
>
> [For the purposes of discussion, I'll assume that the upper
> ontology will be defined in KIF. Similar questions would be
> asked if we used something else.]
>
> If we are defining the upper ontology, then:
>
> - Will a conforming system be a system that includes
> all the KIF statements, i.e., a conforming system must be a
> superset of the SUO KIF?
> - Will a conforming system be a system that is
> "equivalent"? For example, if X is a conforming implemention
> of SUO, must it be the SUO KIF, or can it be "equivalent" to SUO KIF?
> - If we use "equivalence", what does that mean?
> - If we have equivalent systems, we would
> probably need to verify they are equivalent in some automated
> way. If not (e.g., a manual proof of equivalence), what
> happens if these proofs are significantly difficult (e.g.,
> takes a year or two to prove)? If the proof of equivalence
> is difficult, then conformity assessment is difficult ==>
> conformance is weak ==> compatibility and interoperability
> may be in question ==> high risk for market adoption.
> - Or (differently), will the SUO (KIF) somehow define
> all the questions we can ask about a conforming SUO system?
>
> In many conformance paradigms, there is the notion of
> "strictly conforming" (an implementation only meets the
> requirements of a standard) and "conforming" (an
> implementation meets the requirements of a standard, but may
> have "extensions). We might need to distinquish between
> "strictly conforming" and "conforming" implementations of
> SUO. For example, an implementation that contains only SUO
> is "strictly conforming", but an implementation that includes
> SUO and domain-specific ontologies is "conforming". <--
> Stuff to think about.
>
> "B. INTER-OPERABILITY: The standard will provide a
> basis for achieving Inter-Operability among various software
> and database applications."
>
> How is interoperability achieved/measured? Later on the
> Purpose states that interoperability will be achieved "by
> virtue of the shared common terms and definitions" ... so
> we'll need to "measure" the amount of commonality of terms
> and definitions (is this a simple count of axioms, or is it
> an "equivalence" mapping/technique?)
>
> How are "extensions" handled? (FYI, some "extensions" may be
> considered for later editions of this standard, so handling
> extensions is an important versioning/configuration management issue.)
>
> "3) The SUO will play the role of a neutral
> interchange format whereby owners of existing applications
> will be able to map existing data elements just once to a
> common ontology. ..."
>
> It appears that the conformance paradigm might include some
> "neutral interchange format".
>
> "... This provides a degree of interoperability with
> other applications whose representations conform to SUO.
> This entails the SUO being able to be mapped to more
> restricted forms such as XML, database schema, or object
> oriented schema."
>
> I'm not yet sure what "representations conform to SUO" means,
> but we should give this some thought in the context of conformance.
>
> It is important to note that the Scope describes the standard
> that we are writing and the Purpose describes the reason for
> the standard and its potential applications. We mostly need
> to concern ourselves with the Scope and we don't need to be
> too worried about the Purpose. Side thought: Think of Scope
> as "normative" (what we are required to do in our standards
> development) and Purpose as "informative" (helpful information).
>
> Comments?
>
> -FF
> --------------------------------------------------------------
> ---------
> Frank Farance, Farance Inc. T: +1 212 486 4700 F: +1
> 212 759 1605
> mailto:frank@farance.com http://farance.com
> Standards, products, services for the Global Information
> Infrastructure
>