1 Introduction

This document identifies some of the most common issues found in OASIS specifications. It is designed as a checklist, both for specification editors and for reviewers of these specifications.

Explanation of the table entries:

Item ID: a unique ID for the verification item to be checked.
Definition: the definition for the verification item to be checked, in form of a question. A “yes” answer means that the check item is satisfied. A “No” means that the specification fails to satisfy this check item.
Severity: Blocker/Critical/Major/Minor

Minor means that the item failure should be corrected even if not a serious impediment to implementers. Not doing so will indicate a low concern for quality to potential adopters.
Major means that failing the item is likely to confuse standard users and must be corrected to avoid differing implementations. This will be an issue for some reviewers, when membership approval is needed.
Critical means a serious flaw that makes it very difficult if not impossible for implementers to understand the specification, develop an implementation or claim conformance, even if not violating the TC process rules. The failure gives the OASIS TC admin valid reasons to not process further such a specification and must be corrected.
Blocker means that the item is required by the OASIS TC Process and failing the item imperatively requires to be addressed by editors. The OASIS TC admin would be at fault if allowing the specification to process further.

Some values may be predefined for the item, meaning the reviewer can only pick one of them. If none are specified, the reviewer can label a failing item according to his/her judgment.

Check: left for the reviewer to fill-in: a Yes/No response to the question in “Definition”. (“No” means that the specification failed the item).

2 Specification Review Checklist

2.1 Conventions

In the checklist table below, for each checklist item the severity field will indicates the possible severity levels.

M indicates either Minor or Major levels (in the checklist below, no item allows one without allowing the other)
C indicates Critical level.
B indicates Blocker level.

For example, severity content “M,C” indicates that depending on the particular issue at hand that matches that checklist item, the reviewer may qualify it as Minor, Major or Critical but never as Blocker.

2.2 Issue Categories

Issues are classified into seven categories defined as follows:

Reference: issues with references in the specification or document (incorrect, missing, not well-formed, normative vs. non-normative, etc.).
Style: issues with the editorial aspect of the document (formatting, errors, narrative style, etc.)
Technical: issues with the technical content (including terminology, completeness, error handling, etc.)
Normative: issues with the normative quality of the content (including separation normative vs. non-normative, use of formal keywords)
Structure: issues with the general layout and structure of the specification (including sections layout, use of several parts and artifacts, general packaging).
Conformance: issues with the conformance clauses in the specification (including referencing of normative content, relation to implementations, etc.)
Process: issues related to consistency with rules in the OASIS TC process and other requirements, including positioning on standard track vs. informative track.

2.3 Issues of type Reference

Item ID	Definition	Severity	Check
R1	Unused references: Are all references in the reference section actually used in the narrative of the specification?	M
R2	Unlisted references: Are all references in the specification also present and expanded in the reference section?	M,C
R3	Completeness of references: Are all expected references present in the reference section?( I.e all relevant external work has not been ignored.) [1]	M,C
R4	Proper use of normative references: Are all listed normative references used in a normative way? (i.e. they are not appropriate as non-normative references, and are used in a normative way in the specification narrative) [2]	M,C
R5	Proper use of non-normative references: Are all the references that are listed as non-normative only used in a non-normative way in the specification narrative? ( i.e. they are not appropriate as normative references)	M,C
R6	Reference URIs: Do all Internet-based references contain full, visible, and correct URIs? Do these links resolve properly?	M,C,B
R7	OASIS reference: Do all references to OASIS documents use the OASIS format style, as provided in the “Citation format” section of every recent OASIS document, when the citation format is available?	M
R8	Referenced versions: In case the specification appears to be tightly dependent on other specifications/standards, are the versions of these external works clearly indentified?	M,C

2.4 Issues of type Style

Item ID	Definition	Severity	Check
E1	Clarity: Is the wording clear, unambiguous – no confused language?	M,C
E2	DRY: (Don't Repeat Yourself) Is the narrative not redundant?	M
E3	Examples: Are they numbered?	M
E4	Tables: Are they titled, and referenced in the narrative?	M
E5	Hanging paragraphs: Are all paragraphs numbered at the right level, and not hanging under a high-level section title?	M
E6	Figures: Are all figures explicitly referenced and used in the narrative of the specification?	M
E7	cross-references: Are all internal cross-references valid, consistent and appropriate (i.e. references to other sections and to appendices are as intended, e.g. not accidentally misaligned.) and use live links? Is the table of Content correct in referencing sections?	M
E8	Conciseness: Is the specification narrative concise, relevant, and not redundant? (i.e. no text can be removed without hurting understanding, no text is unnecessarily detailed, verbose or peripheral)	M
E9	Typos: Is there no misspelling, missing or extra characters?	M
E10	Formatting and Indenting: Is it correct and consistent?	M

2.5 Issues of type Technical

Item ID	Definition	Severity	Check
T1	Terminology: Are the key terms and acronyms defined clearly and unambiguously?	M
T2	Alignment: Is the specification using terms normally used in the subject area, and using them with the expected conventional meaning?	M
T3	Glossary: Is there a glossary section in case several new terms have been defined or redefined, and are used all over the specification?	M
T4	Trademarks: Are all trademarks & servicemarks recognized as such?	M,C
T5	Completeness: Is technical content complete enough for implementers? Is the specification understandable by someone with average level of expertise in the domain?	M,C,B
T6	Technical quality: Is technical content correct, clear, well explained and unambiguous?	M,C,B
T7	Error handling: Are error cases sufficiently described and handled?	M,C
T8	Implementations: Is the notion of “implementation(s)” of that specification clearly defined? The different kinds of these well identified?	M,C

2.6 Issues of type Normative

Item ID	Definition	Severity	Check
N1	Normative completeness: Are normative statements precise enough and unambiguous for a specification user / developer? Are examples used only to illustrate the normative content, and not as substitute for normative statements?	M,C
N2	Keywords set: Is the set of normative keywords the same as introduced in the specification? (RFC2119 vs ISO, use of upper case)	M,C,B
N3	Keywords use: Is there no Improper use of keywords ? (missing in normative text, mis-used in non normative text)	M,C
N4	Normative vs. non-Normative: Is the distinction between normative and non-normative content clear throughout the document? For any content is it clear whether it is Normative or non-Normative?	M,C
N5	Attribution: Is it clear what kind of implementation or which part of it is subject to normative content ?	M,C
N6	Normative interpretation: For content not explicitly using normative keywords, (e.g. tables, technical descriptions, diagrams) is it clear how to interpret it in a normative way?	M,C

2.7 Issues of type Structure

Item ID	Definition	Severity	Check
S1	Abstract: Is there an abstract? Is the abstract representative of the specification narrative?	M
S2	Work product architecture: Is the overall architecture of the work appropriate, i.e. the choice of either (1) multiple related specifications (or work products), vs. (2) a multi-part specification or work product vs. (3) a single-part work product with possible adjuncts, is appropriate.	M
S3	Sectioning of the specification: Does the sections layout make sense? Does the content of sections match their title?	M
S4	Accessory content: Is accessory content (schemas, use cases, samples, examples) present where needed and used appropriately at the right place (e.g. Appendix, adjunct document), with sufficient introduction & explanation, and with the right status (normative/non)?	M,C
S5	Partitioning across documents: Does the division of content across documents make sense? Does each document of the work product represent a self-sufficient piece of work for the user understanding with reasonable dependencies to others?	M

2.8 Issues of type Conformance

Item ID	Definition	Severity	Check
C1	Presence of Clauses: Are Conformance Clauses present and uniquely identifiable?	B
C2	Conformance targets: Are the different types of conformance targets – i.e. possible implementations - of the specification well identified, and covered by related Conformance Clauses?	M,C
C3	Conformance Clause focus: Is each Conformance Clause clearly identifying its conformance target(s), i.e. narrowing on a type of implementation, and not addressing a broad and disparate set of implementations?	M,C
C4	Precise normative basis: Are the Conformance Clause referring precisely enough to normative statements and content in the specification body ? Is it clear what is expected from an implementation to conform?	M,C,B
C5	Normative overload: Is the Conformance Clause making NO additional normative statements that should have been instead in the specification body for the Clause to reference?	M
C6	Misuse of non-normative content: Does each Conformance Clause NOT use inappropriately non-normative content in the specification body by making it appear as normative?[3]	M,C
C7	Mismatch of prescription levels: In case there is discrepancy between the level of prescription in the referenced normative content and the level of prescription stated in the Clause (e.g. MAY or SHOULD in the specification body becoming MUST in the Conformance Clause) is it always in the direction of the Clause strengthening the prescription, and not weakening it (e.g. NOT making a MUST into a SHOULD)?	M,C
C8	Handling of optionality: According to the Conformance Clauses, is it clear how each conformance target has to deal with normative statements in the specification narrative that are optional?	M,C
C9	Backward compatibility: According to the Conformance Clauses, is it clear whether each conformance target is backward compatible with a previous version of the specification, and under which conditions?	M,C
C10	Relevance of normative content: Is every mandatory normative statement traceable to at least one Conformance Clause? And if not, is it explained why and if future Clauses may address it?	M

2.9 Issues of type Process

Item ID	Definition	Severity	Check
P1	Scope: Does the specification content appear to be within the scope of the TC charter?	M,C,B
P2	Conformance section: For a standard track document, are Conformance Clauses present in a separate conformance section and uniquely numbered?	B
P3	Inappropriate normative keywords: For a non-standard track document, is it free of any normative keyword (RFC2119, ISO)?	B
P4	Inappropriate conformance clauses: For a non-standard track document, is it free of any Conformance Clause?	C,B
P5	Valid computer language: For a standard track document, are normative computer language definitions (includes schemas and Java(TM) code, including fragments of such) well formed and valid?	C,B
P6	Separate language definition files: are computer language definitions other than examples (includes schemas and Java(TM) code, including fragments of such) provided in separate plain text files?	C,B
P7	Referencing of separate files: For a standard track document, is each adjunct text file referenced in the Work Product main documents?	C,B
P8	Handling of inconsistencies: For a standard track document with separate text files: is there no statement against the TC process rule that definitions in separate files prevail over conflicting definitions found in the specification body?	B
P9	Standard track: Is the work in the appropriate track (CN vs. CS) relative the nature of its content?	C,B

[1] NOTE: this verification implies a fair amount of subject matter expertise on the part of the reviewer, although references to relevant external work should be found in the charter or TC creation request.

[2] NOTE: this verification may require some subject matter expertise on the part of the reviewer, although the context of these references and the language used should make their normative use clear.

[3] NOTE: a case of negative answer to this question is of a conformance clause that requires a conformance target to be able to process a (non-normative) example. Although it may be that such an example MUST be processable, the Clause should NOT refer to it directly: this should be a consequence of implementing correctly other normative content.