WORLD METEOROLOGICAL ORGANIZATION

COMMISSION FOR BASIC SYSTEMS

INTER-PROGRAMME EXPERT TEAM ON METADATA AND DATA INTEROPERABILITY

GENEVA, 27-29 April 2010

DISCLAIMER

Regulation 42

Recommendations of working groups shall have no status within the Organization until they have been approved by the responsible constituent body.  In the case of joint working groups the recommendations must be concurred with by the presidents of the constituent bodies concerned before being submitted to the designated constituent body.

Regulation 43

In the case of a recommendation made by a working group between sessions of the responsible constituent body, either in a session of a working group or by correspondence, the president of the body may, as an exceptional measure, approve the recommendation on behalf of the constituent body when the matter is, in his opinion, urgent, and does not appear to imply new obligations for Members. He may then submit this recommendation for adoption by the Executive Council or to the President of the Organization for action in accordance with Regulation 9(5).

© World Meteorological Organization, 2010

The right of publication in print, electronic and any other form and in any language is reserved by WMO. Short extracts from WMO publications may be reproduced without authorization provided that the complete source is clearly indicated. Editorial correspondence and requests to publish, reproduce or translate this publication (articles) in part or in whole should be addressed to:

Chairperson, Publications Board

World Meteorological Organization (WMO)

7 bis, avenue de la Paix Tel.: +41 (0)22 730 84 03

P.O. Box No. 2300 Fax: +41 (0)22 730 80 40

CH-1211 Geneva 2, Switzerland E-mail: Publications@wmo.int

NOTE

The designations employed in WMO publications and the presentation of material in this publication do not imply the expression of any opinion whatsoever on the part of the Secretariat of WMO concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries.

Opinions expressed in WMO publications are those of the authors and do not necessarily reflect those of WMO. The mention of specific companies or products does not imply that they are endorsed or recommended by WMO in preference to others of a similar nature which are not mentioned or advertised.

This document (or report) is not an official publication of WMO and has not been subjected to its standard editorial procedures. The views expressed herein do not necessarily have the endorsement of the Organization.

CONTENTS

Agenda

Executive summary

General summary of the work of the session

Annexes

AGENDA

1. ORGANIZATION OF THE SESSION

1.1 Opening of the meeting

1.2 Adoption of the agenda

1.3 Working arrangements

2. WMO CORE PROFILE OF THE ISO METADATA STANDARD

2.1 Review of the version 1.1 of the WMO core profile

2.1.1 Editorial corrections

2.1.2 Clarification required for the implementation of the WMO core profile by WIS centres

- File naming conventions

- Metadata identifiers

- Relating data-files to metadata-files

- Metadata versioning

- Dataset hierarchies

- Geographic metadata conventions & machine-readable gazetteer for Volume A

- Automated validation

- Human readable metadata & SRU query response

- Controlled vocabularies and ontologies

- Definition of access control policy controlled vocabulary & implementation in metadata record

- Miscellaneous metadata conventions

- Metadata management policies

- Metadata standards maintenance

3. AWARENESS OF THE DEVELOPMENT OF A WMO CORE PROFILE OF THE ISO 19100 SERIES OF STANDARDS FOR DATA AND METADATA WITHIN THE WMO COMMUNITY

4. WORK PLAN OF THE IPET-MDI

4.1 Actions arising from this meeting

4.2 Tooling to support collaborative development of WMO Core Profile standard

5.    CLOSURE OF THE MEETING

___________

Executive Summary

The first meeting of the CBS Inter-Programme Expert Team on Metadata and Data Interoperability (IPET-MDI) was held in Geneva, Switzerland, from 27 to 29 April 2010 under the chairmanship of Mr J. Tandy (UK).

The meeting focused its activities on the WMO core profile of the ISO metadata standard, in particular on the clarification required for the implementation of the WMO core profile by WIS centres and the tool to support the development of WMO Core Profile standard. The meeting developed 62 recommendations in this respect related to the following items:

- File naming conventions

- Metadata identifiers

- Relating data-files to metadata-files

- Metadata versioning

- Dataset hierarchies

- Geographic metadata conventions & machine-readable gazetteer for Volume A

- Automated validation

- Human readable metadata & SRU query response

- Controlled vocabularies and ontologies

- Definition of access control policy controlled vocabulary & implementation in metadata record

- Miscellaneous metadata conventions

- Metadata management policies

- Tooling to support collaborative development of WMO Core Profile standard

The recommendations of the meeting will be submitted to the meeting of the OPAG-ISS Implementation Coordination Meeting on ISS (ICT-ISS) scheduled in Geneva from 27 to 30 September 2010.

The documents prepared for the meeting are available from: http://www.wmo.int/pages/prog/www/WDM/IPET-MDI-I/IPET-MDI-I_list_documents.html

1. ORGANIZATION OF THE SESSION

1.1 Opening of the meeting

1.1.1 The first meeting of the CBS Inter-Programme Expert Team on Metadata and Data Interoperability (IPET-MDI) was held in Geneva, Switzerland, from 27 to 29 April 2010 under the chairmanship of Mr J. Tandy (UK). The list of participants is given in Annex to this paragraph.

1.1.2 On behalf of the Secretary-General, Mr P. Shi, Director of the WIS Branch, welcomed the participants.  He recalled that the fourteenth session of the Commission for Basic Systems endorsed the version 1.1 of the WMO core profile of the ISO Metadata standard. Version 1.1 is being implemented by the centres that are candidates to become Global Information System Centres (GISC) or Data Collection or Production Centres (DCPC). Those centres raised issues concerning the implementation of the version 1.1. The sixth session of the Inter-Commission Coordination Group on WIS, which was held in Seoul in February 2010, invited the IPET-MDI to provide guidance on metadata for authors, including samples and templates, and to define clear specification of minimum metadata requirements. Mr P. Shi stressed the importance of considering the inclusion of this set of guidance in a guide for the implementation of the WMO core profile.

1.1.3 The Commission agreed that the application of the ISO 19100 series of geographic information standards to the development of a WMO conceptual model of data representation should be considered as a fundamental element of a CBS policy on data representation systems. Applying a standard approach for data representation, this should lead to the development of a WMO core profile of the ISO 19100 series for metadata and data, encompassing the WMO core profile of the ISO metadata standard. CBS tasked the IPET-MDI to develop and maintain a WMO conceptual data model and a WMO core profile of the ISO 19100 series of standards for metadata and data.

1.1.4 CBS agreed that WMO and CBS would benefit from closer cooperation with the Open Geospatial Consortium (OGC), which sets standards for web access to geospatial information. A Memorandum of Understanding between WMO and OGC was signed in November 2009.The WMO/OGC Memorandum of Understanding is instrumental in providing the mechanism for the co-ordination between the activities carried out by OGC and WMO with a view to developing the use of ISO/OGC standards for the WIS. Mr P. Shi invited the meeting to consider how to benefit from this MoU to facilitate the development of a WMO conceptual data model and a WMO core profile of the ISO 19100 series for metadata and data.

1.2 Adoption of the agenda

The participants of the meeting agreed that development of the guidance for the WMO Core Profile metadata standard. Due to time constraints the item "DEVELOPMENT OF A WMO CONCEPTUAL DATA MODEL" was removed from the agenda. The modified agenda was adopted by the participants.  It is reproduced at the beginning of this report.

1.3 Working arrangements

The meeting agreed upon its working hours. The documents prepared for the meeting are available from http://www.wmo.int/pages/prog/www/WDM/IPET-MDI-I/IPET-MDI-I_list_documents.html.

2. WMO CORE PROFILE OF THE ISO METADATA STANDARD

2.1 Review of the version 1.1 of the WMO core profile

2.1.1 Editorial corrections

2.1.1.1 The meeting agreed on the editorial corrections to the UML representation of the version 1.1 of the WMO core profile of the ISO metadata standard as given in IPET-MDI-I/Doc. 2.1 (1) with a view to aligning it to the Annex A to ISO 19115 Cor 1, and to publish the corrected version. The version number for this amended release is discussed in item "2.1.2.58 Versions and future update cycles". As these are ONLY correcting typos in WMO Core Profile documentation, there is no need to reflect any changes in schema etc. Only the UML descriptive document needs updating. Noting that these editorial corrections had no impacts on the implementation of the WMO core profile, the meeting recommended inviting the chair of the OPA-ISS and the President of CBS to request the Secretariat to post this corrected version in http://wis.wmo.int/2010/metadata/version_1-1-1 together with the current extensions to ISO code lists.

2.1.1.2 Additional constraints identified during this meeting will be expressed within the UML, therefore the standard profile documentation must be updated. A schematron rule set will be derived from these UML constraints. Guidance notes will be provided explaining more about the impact of the constraints. To ensure completeness, conditions and notations on usage will be added to the data-dictionary where appropriate to supplement the UML model.

2.1.2 Clarification required for the implementation of the WMO core profile by WIS centres

2.1.2.0 This section has notes regarding the usage of ISO MI_Metadata and MD_Metadata root elements and the use or non-use of namespace prefixes in examples in this document.  The elements mentioned above are both valid root elements in ISO19115-2.  MD_Metadata is the current WMO practice, and as such, sections that address elements in current practice use the MD_Metadata as the root element. A transition to using the MI_Metadata element as the recommended root element is envisioned, so in this document examples that illustrate future practice that will require MI_Metadata as the root element will use that element.  In future practice MI_Metadata will replace all usage of MD_Metadata.  Section 2.1.2.39 has more details about this transition.

Similarly, examples in this document sometimes use the gmd prefix for elements that are shown with default namespace in other examples.   Users of this document should understand that current WMO practice uses default namespaces, but in the future it will be recommended that WMO XML use an explicitly named prefix (gmd).  For a given XML document one method or the other must be chosen as is documented in section 2.1.2.43.

File naming conventions

2.1.2.1 The meeting noted the following concerns relating to the ‘file naming convention, identifier & uniqueness’ issue. This issue has been obscured by continually entangling several problems:

· The granularity level described by each metadata record;

· The need to accommodate a simple solution for GTS datasets intended for global exchange; and

· The association of metadata files and data-files (or groups of data-files)

2.1.2.2 To illustrate the problem, the team discussed two cases:

A. Metadata record A describes a dataset of bulletins which are stored in the 24-hour cache of the GISC. The metadata record is equivalent (although more informative) to a record in WMO vol C1 & describes the normal contents of this type of bulletin; for example SYNOPS from several observation stations; including MLO (Mauna Loa, Hawaii).

B. Metadata record B describes a long-term climate record from station MLO which is comprised of a collection of SYNOPTIC observations from, say, 1954.

2.1.2.3 Whilst both datasets are continually changing, both metadata records are ‘quasi-static’; only needing to be changed when the observation regime changes (i.e. a new instrument is deployed or the exact observation location changes) The critical differences between records A & B in this example are:

· Temporal extent: A has a relative temporal extent in any 24-hour period, whilst B has a temporal extent from 1954 to (almost) present day;

· Citation authority: authority for A is int.wmo.wis, whilst B is gov.noaa

· Quality control: the dataset described by B may have undergone additional quality control to validate the observation record for inclusion in a long-term archive

2.1.2.4 Whilst the meeting noted that there may be significant overlap between A and B, one cannot assume that overlap exists. The meeting concluded that metadata records A and B describe entirely different products!

2.1.2.5 The meeting noted that for efficiency, some Regional Transportation Hubs (RTH) ‘batch’ bulletins into groups (a.k.a. messages) for efficient transfer. This is not evident to the end users, hence the content of a transport-level GTS message is not required to be described by a metadata record. The meeting assumed that the user/consumer is interested in the discrete bulletins which are currently catalogued in Volume C1 and will be exposed via the WIS Discovery Access and Retrieval (DAR)-metadata catalogue.

2.1.2.6 Significant discussion about the file-naming-convention (see Attachment II-15 to the Manual on the GTS) lead to the following statements:

· All files (not bulletins) exchanged via the GTS must conform to this convention.

o There are two types of data transfer in the GTS: file transfer and legacy bulletin.  Legacy bulletins are solely identified by their abbreviated heading line (AHL), not by the filename of their transfer container (a.k.a. message) file.

· Whilst OAI-PMH has been agreed as the protocol for GISC-GISC metadata harvesting, the WIS functional architecture indicates that the default mechanism for National Centres (NC) and DCPCs to pass their metadata records to their affiliated GISC is via sending of files. The meeting noted that the Internet connectivity is not always available and transfer via GTS is necessary in such case.

o Therefore, filenames of all metadata files transferred via the GTS must conform to the endorsed file-naming-convention.

· The file-naming-convention is summarized as:

«productPart»_«originatorPart»_«YYYYMMDDhhmmss»[_«additionalPart»].«type».[«compression»]

o where «productPart» is «pflag»_«productidentifier», and «pflag» describes the format of «productidentifier»,

o «originatorPart» is C_«CCCC», where «CCCC» is 4-letter code for the originating telecommunications centre, and

o «type» is general format type from fixed acceptable list.

· There are four types of «productPart» (hence values of «pflag») defined:

o T_«TTAAii» and A_«TTAAii»«CCCC»«YYGGgg»[«BBB»] are provided for mapping bulletins into file transfer.  The latter (A_ form) gives all AHL information, while the former (T_ form) is sufficient for bulletins not using «BBB» for amendment/correction (ex. NWP output).  «YYGGgg» is day-in-month, hour, and minute in UTC.

o Routing of bulletins between RTHs is specified using «TTAAii» and «CCCC».

o The file-naming-convention is designed in order to resolve the limitations of too narrow namespace «TTAAii», and following two «pflag»s are prepared for that purpose.

o W_«WMO Product Identifier» is globally unique identifier of the product, where «WMO Product Identifier» is a comma-separated list of data properties, including originating centre, data category, and other information, optionally followed by varying date and «BBB».  Invariant string is intended to be used in routing control.

Note 1: W_«WMO Product Identifier» scheme employs commas (,) to separate elements – this may create problems if these file-names are ever integrated into other lists or URLs as the comma is commonly used as a list delimiter in software systems. ET-CTS should be informed of this concern.

o Z_«local product identifier» is a effectively a free-form identifier type, which is unique only within the originating centre.

· Eventually bulletins and T_ and A_ product-identifiers will be phased out in favour of file transfer with W_ and Z_ identifiers.  However, currently Volume C1 and the Routing Catalogue only covers bulletins, which have to be mapped to either T_ or A_ types of filename.

· Implementation note: within their GISC implementation, CMA have only accounted for T_ and A_ product identifier types.

2.1.2.7 As a DAR metadata record is considered to be quasi-static (i.e. it is unlikely to change over the period of days or months) certain instance-specific elements of the data-filename must be excluded from the name of any associated metadata-identifier. For example, an ‘A_’ type product-identifier ‘«TTAAii»«CCCC»«YYGGgg»[«BBB»]’ would be truncated to ‘«TTAAii»«CCCC»’ to provide a unique identifier that does not vary over time.

Separate metadata records for amendments / corrections are considered unnecessary: the «BBB» element should never be present in a metadata filename.

When amended to remove the elements that may vary between bulletin instances, A_ product identifiers differ in content from T_ product identifiers only by including «CCCC». Noting that «CCCC» is also incorporated in the top-level file-naming convention, there appears to be no value in using A_ product identifier types for metadata; T_ product identifier types should suffice.

W_ and Z_ product identifiers should retain their arbitrary complexity, albeit amended to remove elements that vary between bulletin/product instances (if necessary).

Given the nature of W_ and Z_ product-identifier schemes, the team were unable to see a mechanism for establishing an implicit machine-interpretable linkage between a metadata and its associated data files.

The meeting agreed to use suffix ".xml" instead of ".met" for compatibility with common implementations (cf. 2.1.2.12).

Recommendation 1: when metadata file is transferred between WIS centres, the metadata filenames:

- must comply with the WMO file-naming-convention;

- should use T_ product identifiers in place of A_ product identifiers to aid simplicity

- should truncate W_ product identifiers to form an invariant string; i.e. the date and amendment / correction code will be excluded

- must ensure that Z_ product identifier are globally unique within the WIS

- must use the «YYYYMMDDhhmmss» element from the top-level file-naming convention to express the metadata publication / update date-time (this must represent the same date-time as the MD_Metadata/dateStamp element);

- Use an ‘.xml’ extension:

- Examples:

- T_FCUK31_C_EGRR_20100310180000.xml

- T_HHXA05_C_BABJ_20100427083000.xml

- W_FR-meteofrance-toulouse,GRIB,ARPEGE-75N10N-60W65E_C_LFPW_20100428115800.xml

2.1.2.8 Where metadata records are harvested from contributors outside the WIS community using OAI-PMH,　there is no way for the harvester to know file naming conventions used by the metadata provider. Therefore, in this situation, there is no requirement to constrain the filename.  Only where metadata records are transferred via the GTS or other GTS-like file-transfer mechanisms do the filenames need to conform with the convention above.

XML formats are not currently used to transfer data via the GTS.  However, once this changes, it is conceivable to have collisions between data-files and metadata files.  To remedy this, there is a need to be able to distinguish between the files with additional information in the filename.

Note 2: The ‘.xml’ file extension does not currently appear in the permitted list in the file-naming convention. A proposal to modify the file-naming convention is offered to ET_CTS & ET_OI:   Allocate a new extension=”.iso19139.xml”

Metadata identifiers

2.1.2.9 Metadata identifiers do not need to explicitly match (a truncated version of) the data-filenames they describe. The team identified that the metadata naming conventions from IPET-MI-II (Moscow, 2006) could be applied [in principle].

Recommendation 2: gmd:MD_Metadata/gmd:fileIdentifier is a URI (Universal Resource Identifier) structured as follows:

- fixed string "urn:x-wmo:md:"

- citation authority

- Examples:

§  "int.wmo.wis"

§  "gov.noaa"

§  "edu.ucar.ncar"

§  "uk.gov.metoffice"

- Citation authority "int.wmo.wis" is used for all datasets intended for global exchange

- Citation authority for other datasets remains with the dataset provider, expressed using a reverse-DNS name

- Register of citation authorities should be maintained by WMO secretariat

- separator colon ‘:’

- empty string, reserved for future extension

- separator colon ‘:’

- unique identifier:

- if a GTS «TTAAii» and «CCCC» is allocated for the product use «TTAAii»«CCCC»;

- else if a WMO Product Identifier is allocated for the product use a truncated WMO product identifier field of the associated data-files, excluding the date-stamp and any other varying elements as necessary; or

- else use a locally-unique identifier for the citation authority (not recommended for data and products intended for global exchange)

Examples:

- urn:x-wmo:md:int.wmo.wis::FCUK31EGRR

- urn:x-wmo:md:cn.gov.cma::NMC.NWP.HCXA05BABJ

- urn:x-wmo:md:int.wmo.wis::FR-meteofrance-toulouse,GRIB,ARPEGE-75N10N-60W65E_C_LFPW

[Action MDI-1-i : Ted - propose mechanism for unique identification of place names - e.g. urn:x-wmo:place:int.wmo.wis::AtlanticOcean etc.]

The meeting noted that some additional guidance may be necessary to facilitate the construction of WMO product identifiers.

The meeting noted with appreciation the WMO Product Identifier syntax adopted by the Satellite community for RARS data and encouraged similar harmonization efforts for other communities and data types.

The GTS Manual recommends using BUFR/CREX Common Code Table C13 categories as data designators in WMO Product Identifiers, although exactly how these categories are integrated deserves further specification. The meeting noted the abbreviated categories proposed to this end in the joint ICM-MTN & ET-OI report for every data categories and sub-categories in BUFR/CREX common code table C13, but saw the need for further refinement and a formal approval as an extension to table C13

Recommendation 3: gmd:MD_Metadata/gmd:fileIdentifier elements are treated as CASE-INSENSITIVE when assessing metadata records for duplication.

2.1.2.10 The meeting noted that:

· OAI-PMH requires the identifier must comply with generic URI syntax (RFC2396, now obsoleted by RFC3986).  The naming convention defined in Recommendation 2 above is compliant to URN scheme of URI.  The namespace identifier of URN (i.e. "x-wmo") begins with "x-", which indicates an experimental namespace not registered to IANA (cf. RFC3406).  In the long term, the WIS community may wish to apply to IANA for registration of formal namespace "wmo" under URN, or namespace under "info:" URI scheme if the review process for URN does not work (http://info-uri.info/registry/docs/misc/faq.html#use_urn describes such high burden).

· OGC & others are beginning to favour persistent URLs over URNs as they are resolvable; an example would be: "cma.gov.cn/md/HCXA20BABJ.NMC.NWP"

2.1.2.11 Modifying the MD_Metadata/fileIdentifier may cause issues with harvesting processes used by systems such as geonetwork. The gmd:MD_Metadata/gmd:fileIdentifier is used as a primary key to identify records for harvesting. Changing the primary key may result in the record appearing twice. To avoid needing to change the identifier dynamically during harvest, it is envisaged that the addition of a WMO Core Profile compliant identifier will be added by bi-lateral arrangement a priori. Also, data custodians may choose to create new metadata records specifically for the WIS community. However, OAI servers do not distinguish between metadata catalogues during harvesting; the entire catalogue is open to all. Existing OAI servers are not currently set up to expose only a subset of their catalogue. To resolve this concern, if the data-provider is able to guarantee global uniqueness of their original MD_Metadata/fileIdentifier this will remain acceptable as a primary key within WIS and there is no requirement to add a WMO Core Profile compliant identifier.

Recommendation 4: WIS Centers should not modify MD_Metadata/fileIdentifer (as well as any other content) when receiving metadata from external data provider, as far as they guarantee global uniqueness of the fileIdentifier.  UUID (Universal Unique Identifier) is considered globally unique in this respect.  However, a WIS Center may agree with the data provider to give a fileIdentifier compliant to the WMO Core Profile.  In that case previously given fileIdentifier is preserved in:

"MD_Metadata/dataQualityInfo/DQ_DataQuality/lineage/LI_Lineage/source/LI_Source/sourceCitation/CI_Citation/identifier/MD_Identifier/code".

It is possible that future versions of ISO 19115 will allow multiple identifications in a single metadata record. If this becomes part of the standard an alternative convention may be developed.  Note that renaming of fileIdentifier shall not apply when GISC is relaying GISC-to-GISC synchronisations, in order to avoid looping.

The minimum required information for a sourceIdentifier is:

<gmd:dataQualityInfo>
<gmd:DQ_DataQuality>
<gmd:scope>
<gmd:DQ_Scope>
<gmd:level>
<gmd:MD_ScopeCode codeList="http://www.isotc211.org/2005/resources/Codelist/gmxCodelists.xml#MD_ScopeCode" codeListValue="dataset">dataset</gmd:MD_ScopeCode>
</gmd:level>
</gmd:DQ_Scope>
</gmd:scope>
<gmd:lineage>
<gmd:LI_Lineage>
<gmd:source>
<gmd:LI_Source>
<gmd:sourceCitation>
<gmd:CI_Citation>
<gmd:title>
<gco:CharacterString>This is the title of the original dataset</gco:CharacterString>
</gmd:title>
<gmd:date>
<gmd:CI_Date>
<gmd:date>
<gco:DateTime>2010-05-08T07:38:00</gco:DateTime>
</gmd:date>
<gmd:dateType>
<gmd:CI_DateTypeCode codeList="http://www.isotc211.org/2005/resources/Codelist/gmxCodelists.xml#CI_DateTypeCode" codeListValue="creation">creation</gmd:CI_DateTypeCode>
</gmd:dateType>
</gmd:CI_Date>
</gmd:date>
<gmd:identifier>
<gmd:MD_Identifier>
<gmd:code>
<gco:CharacterString>This is the identifier</gco:CharacterString>
</gmd:code>
</gmd:MD_Identifier>
</gmd:identifier>
</gmd:CI_Citation>
</gmd:sourceCitation>
</gmd:LI_Source>
</gmd:source>
</gmd:LI_Lineage>
</gmd:lineage>
</gmd:DQ_DataQuality>
</gmd:dataQualityInfo>

The meeting noted that ISO 19115 is under revision process, by which MD_Metadata/gmd:fileIdentifier will be obsoleted by alternative notation that allows multiple identification information.  After that takes place, it might be less confusing than "changing" fileIdentifier that a WIS Centre "adds" second file identifier compliant to WMO Core Profile.  But careful consideration on synchronization and metadata ownership is required.

Note 3: ISO19115 Revision team should be informed of our need to use multiple MD_Metadata/fileIdentifier elements

2.1.2.12 The meeting noted that contrary to previous understanding, the common OAI implementation do permit the file-name and identifier of a metadata record to be different. However, it appears that both GEONETWORK and jOAI both require the filename to have a ‘.xml’ extension.

Relating data-files to metadata-files

2.1.2.13 The relationship between a given metadata identifier and the names of the group of data-files it describes cannot be routinely inferred from the MD_Metadata/fileIdentifier element or metadata filename.

Recommendation 5: where possible, the metadata identifier should map onto the invariant elements of the data-filename to help a human reader infer the connection. An explicit machine-interpretable linkage to the associated data-files will be expressed within the metadata record itself using the

MD_Metadata/gmd:describes/gmx:MX_DataSet/gmx:dataFile/gmx:MX_DataFile/gmx:fileName

element with regular expression of filename as content. Additional mandatory elements are: fileDescription and fileType

2.1.2.14 The meeting agreed that there appears to be no requirement for a machine-interpretable linkage at the file-name level; the metadata record will not be employed to resolve routing information for the GTS, so speed & robustness of GTS message-switching will be unaffected. However, for efficiency, a GISC node may choose to maintain an external mapping of metadata identifiers to associated filename expressions enabling quick look-up on ingest of new data-files for indexing to the associated metadata file. In order to ensure consistency across all GISC nodes, all datasets intended for global exchange must employ the same mechanism to define the explicit machine-interpretable linkage from metadata record to data-files. Other datasets are free to choose their own schemes or employ local agreements as these links need only be resolved at the NC or DCPC.

Recommendation 6: where the citation authority is int.wmo.wis (signifying a dataset intended for global exchange), a standardized regular expression syntax will be used to express the linkage between the metadata record and the associated data-files via the

MD_Metadata/gmd:describes/gmx:MX_DataSet/gmx:dataFile/gmx:MX_DataFile/gmx:fileName

element. Multiple linkages may be expressed using the OR syntax in regular expression ‘|’.

Examples:  

- Data-filename instance: “A_SMCI01BABJ281200_C_BABJ_20100428120000.txt”

- MD_Metadata/fileIdentifier=”urn:x-wmo:md:int.wmo.wis::SMCI01BABJ”

- MD_Metadata/gmd:describes/gmx:MX_DataSet/gmx:dataFile/gmx:MX_DataFile/gmx:fileName=”/^A_

SMIC01BABJ[0-9]+_C_BABJ_[0-9]+\.txt$/”

- MD_Metadata/gmd:describes/gmx:MX_DataSet/gmx:dataFile/gmx:MX_DataFile/gmx:fileDescription=”General

description of the datafile”

- MD_Metadata/gmd:describes/gmx:MX_DataSet/gmx:dataFile/gmx:MX_DataFile/gmx:fileType=”text/plain”

- Data-filename instance: “W_FR-meteofrance-toulouse,GRIB,ARPEGE-75N10N-60W65E_C_LFPW_

20061001000000.bin”

- MD_Metadata/fileIdentifier=”urn:x-wmo:md:int.wmo.wis::FR-meteofrance-toulouse,GRIB,ARPEGE-75N10N-

60W65E_C_LFPW”

- MD_Metadata/gmd:describes/gmx:MX_DataSet/gmx:dataFile/gmx:MX_DataFile/gmx:fileName=”/^W_FR-

meteofrance-toulouse,GRIB,ARPEGE-75N10N-60W65E_C_LFPW_[0-9]{14}\.bin$/”

- MD_Metadata/gmd:describes/gmx:MX_DataSet/gmx:dataFile/gmx:MX_DataFile/gmx:fileDescription=”General

description of the datafile”

- MD_Metadata/gmd:describes/gmx:MX_DataSet/gmx:dataFile/gmx:MX_DataFile/gmx:fileType=”application/

octet-stream”

No serious issues are likely to arise from use of regular expressions on GTS filenames as the character set is limited to International Alphabet 5.  Further information on Regular Expressions can be found

at http://docs.google.com/View?id=df4kmgqs_50ffmzmqgj.

The summary is:

· Metadata creator or manager should provide regular expression that is understandable for all GISCs.  Recommended syntax is chosen from common subset of various implementations (cf. above reference for detail; syntax letters are ? * + {,} (|) ^ $ [] [^] [a-z] \d \w . \. \+).

· GISC Cache should support at least recommended syntax.  It may also support other metacharacter for further development, but it must not alter recommended syntax and its semantics.  It is advised to check regular expression provided in metadata before matching.

· Although WMO Filename Convention is case-insensitive, metadata creators should give regular expression in correct case of letters.  For example, location identifier should be described as [A-Z]{4}, not [a-z]{4}.

· GISC Cache should use "ignore-case" option of regular expression library in matching metadata and dataset fragment.

Above recommendations should be reviewed when use of the regular expression is extended beyond current GTS files and bulletins, or WIS data or products intended for global exchange is extended beyond current GTS.

Metadata versioning

2.1.2.15 The team was asked to verify the agreement made at the ICG-WIS 6 indicating that only the fileIdentifier element and metadata creation date-time stamps will be used to assess uniqueness / duplication of a given metadata record compared to other records.The sequence (time-order) of metadata files with the same fileIdentifier will be assessed according to the date-time stamp.

Recommendation 7: WIS implementers will assess the uniqueness of a metadata record using only the fileIdentifier element and metadata creation date-time stamps:

gmd:MD_Metadata/gmd:fileIdentifier

gmd:MD_Metadata/gmd:dateStamp

Version numbering or date-stamps are explicitly forbidden from the fileIdentifier element. The MD_Metadata/dateStamp is the correct place for this information. It may also optionally appear in the metadata filename (see below). Changing the fileIdentifier element in any way will appear to be a new product!

Note to implementers: The gmd:dateStamp within the metadata record should be distinguished from the datestamp in OAI-PMH wrapper XML, which appears as oai:header/oai:datestamp.  Note that the OAI-PMH Implementation Guideline mandates that the OAI-PMH Aggregators (i.e. a WIS Center that provides metadata harvested from somewhere else) must use local harvesting time as oai:datestamp.   In other words, OAI-PMH providers of GISCs and RTH-type of DCPCs must not use gmd:dateStamp as oai:datestamp, because connected GISC cannot use incremental harvesting from such provider.  In some implementations the timestamp of metadata files are given as oai:datestamp, and the operator can cause re-synchronization by changing metadata file time stamps (such as touch(1) command).  This is intended behavior, and developers should follow this methodology so as not to disable this functionality.

Metadata hierarchies

2.1.2.16 Fundamental to ISO19115 are the concepts of dataset (DS_DataSet) and aggregations (DS_Aggregate). Aggregations, or collections, of datasets can be of many types including series, platforms & sensors etc. Every type of aggregation can include subsets and/or discrete datasets. The relationships permitted within the ISO19115 conceptual model allow one to create a hierarchy of related aggregations and datasets. This enables data providers to structure their metadata for improved clarity & browsing. Both datasets and aggregations have associated metadata (MI_Metadata). It is implied that metadata records describing a given aggregation or dataset within the hierarchy inherit content from their parent metadata record, only amending as necessary. Additionally, the MD_AggregateInformation element of a specific metadata record provides a mechanism to associate the subject dataset with another related dataset. In this case (crossReference or largerWorkCitation instead of direct hierarchy), there is no implied inheritance. These mechanisms allow one to create a hierarchy that enables a data-consumer to browse between related dataset descriptions.

2.1.2.17 NCAR’s Community Data Portal, the THREDDS (Thematic Realtime Environmental Distributed Data Services Project and NOAA’s National Geophysical Data Centre [http://www.ngdc.noaa.gov/] have gained experience from structuring datasets within hierarchies. These dataset hierarchies can be used to structure a browsable network of related datasets. Such browsable networks can be used to supplement other browse-hierarchies inferred from the categorization of metadata records using structured taxonomies etc. (see below). Reference information can be found at http://www.unidata.ucar.edu/projects/THREDDS

2.1.2.18 NOAA-NGDC are also developing experience in decomposing their catalogue of metadata records into re-usable elements explicitly identified using UUIDs and resolvable at a specified URL. Each metadata record uses xlink to reference the metadata component for inclusion. This normalization of metadata records aids maintenance by minimizing redundant information. NGDC respond to consumer queries with either fully-resolved & complete metadata records (from a web-accessible folder) or partially complete metadata records containing the xlinks which reference the missing elements. Both are valid depending on the needs of the consumer.

2.1.2.19 Referring to the needs of WIS, the main focus is to enable the discovery of datasets. Building a browse-hierarchy from a network of related metadata records will add value in the long-term. However, this does not offset the additional complexity & risk of implementation during the development-phase of the WIS.

Recommendation 8: Following creation and subsequent validation of the DRAFT metadata records to populate the initial WIS DAR catalogue, IPET-MDI will identify candidate static metadata components (i.e. CI_ResponsibleParty object describing WMO) that could be referenced via xLink from the discovery metadata records.

[Action MDI-1-ii: Jean Pierre - Develop candidate static metadata components]

Such static metadata components will be identified by UUID and resolvable by URL. These metadata components will be hosted at wis.wmo.int/.

Recommendation 9: in order to maintain simple GISC implementation, discrete metadata records will be used to describe GTS products intended for global exchange. Multi-level dataset-hierarchies may be used to describe datasets resolved at the DCPC and NC levels.

Recommendation 10: metadata records describing GTS products intended for global exchange will continue the practices established to populate Vol C1 where each metadata record describes a set of data-products that vary only with time. Once a particular dataset has been discovered, GISC nodes are only required to offer a query service that allows sub-querying based on time. This will require significant numbers of fine-grained metadata to be created and published in order to retain simplicity of operation at the GISC-nodes.

Recommendation 11: DCPCs and NCs may choose to describe their dataset at whichever level in the ‘hierarchy’ is deemed appropriate. For example, a single TIGGE metadata record may describe 10000 data-files or more, thus avoiding the need to create many hundreds or thousands of metadata records. In this case, the onlineResource is likely to refer to a more sophisticated service that facilities query by multiple parameters in order to minimize the number of positive results returned for a given query.

[Action MDI-1-iii: Manuel – build dataset examples for TIGGE]

Recommendation 12: IPET-MDI develops best practice for creating and maintaining hierarchies of related datasets.

[Action MDI-1-iv: Ted & Michael develop best practices for building hierarchies of related datasets using the ISO19115 conceptual model subject to the constraints of implementation!]

2.1.2.20 As implementation experience with WIS increases, IPET-MDI will continue to review the value of structured dataset-hierarchies against the potential cost & risk of implementation. This may lead to updated recommendations that positively endorse the use of dataset-hierarchies for DCPCs or NCs dataset aggregations and perhaps even products intended for global exchange served by the GISCs. These recommendations imply that NMHSs such as JMA will be required to create and maintain many thousands of metadata records to describe their GRIB bulletins (approx. 21000 products). It is recognized that the query service at the GISC is likely to return large numbers of positive hits to any query for JMA GRIB data, which is not particularly helpful for the user. However, there is nothing to stop JMA from exposing a richer, more sophisticated service interface(s) as a DCPC that provides higher levels of control to the user. These interfaces would likely be described by fewer, more coarse-grained metadata records, thus returning fewer positive hits to an initial query in the DAR catalogue. In this case, the user experience is likely to be significantly improved.

Geographic metadata conventions & machine-readable gazetteer for Volume A

2.1.2.21 The implementation experiences of DWD, CMA and JMA had noted a divergence of approaches in the use of geographic metadata. The WIS implementation community sought explicit guidance on this topic.

Recommendation 13: boundingBox

MD_Metadata/identificationInfo/MD_DataIdentification/extent/EX_Extent/geographicElement/EX_GeographicBoundingBox

will be mandatory to ensure compliance with INSPIRE regulations. Where the sensor platform is mobile (e.g. AMDAR, ships or bouys) the bounding box will describe the maximum reasonable extent within which the platform may operate, potentially a global extent. However, it should be noted that GTS practices (tables C1, C2 & C3 of the Manual on GTS) keep the extent of the bulletin fixed and require that the data from mobile platforms is allocated to the appropriate bulletin based on the location of the platform.

The team noted that specifying datasets that include the north or south poles may be difficult. Also, best practices on acceptable units for lat/lon must provided (e.g. -180 to +180, -90 to +90; 0 to 360; or East/West notation.

[Action MDI-1-v: Jean Pierre & Simon - develop a first-cut set of metadata with appropriate bounding boxes, including SHIP & AMDAR products]

Recommendation 14: following best-practice developed by NGDC, boundingExtent, boundingBoxExtent, boundingTemporalExtent, and boundingVerticalExtent elements will be tagged for identification and easy re-use within the metadata record:

<gmd:extent>
<gmd:EX_Extent id="boundingExtent">
   ...
<gmd:EX_GeographicBoundingBox id="boundingGeographicBoundingBox">
   ...
<gmd:EX_TemporalExtent id="boundingTemporalExtent">

   ...

<gmd:EX_VerticalExtent id="boundingVerticalExtent">

Recommendation 15: place-names (such as observation stations and other static sensor platforms) will be expressed as both keyword (type=place) and geographicIdentifiers:

· MD_Metadata/identificationInfo/MD_DataIndentification/descriptiveKeywords/MD_Keywords/keyword

· MD_Metadata/identificationInfo/MD_DataIndentification/descriptiveKeywords/MD_Keywords/type = "place"

· MD_Metadata/identificationInfo/MD_DataIdentification/extent/EX_Extent/geographicElement/EX_GeographicDescription/geographicIdentifier/MD_Identifier

Recommendation 16: place-names used as keywords and geographicIdentifiers will be defined in a register (or controlled vocabulary) based on the content of Vol A. Each entry must have a UUID (as primary key) in addition to the (mostly) unique WMO station-code identifier. The station-gazetteer register & each entry will be resolvable by URL (incorporating the UUID) and, as a minimum, validate the authenticity of a given station.

The Volume A gazeteer implementation will be available in the form of a CT_CodelistCatalogue (e.g http://asdd.ga.gov.au/asdd/profileinfo/anzlic-allgens.xml) that can be referenced as: a keyword thesaurus

MD_Metadata/identificationInfo/MD_DataIdentification/descriptiveKeywords/MD_Keywords/thesaurusName/CI_Citation

or as an authority in the geographicIdentifier case:

MD_Metadata/identificationInfo/MD_DataIdentification/gmd:extent/EX_Extent/geographicElement/EX_GeographicDescription/gmd:geographicIdentifier/MD_Identifier/authority/CI_Citation

In the geographicIdentifier case, the code:

MD_Metadata/identificationInfo/MD_DataIdentification/extent/gmd:EX_Extent/geographicElement/EX_GeographicDescription/geographicIdentifier/MD_Identifier/authority/CI_Citation/identifier/MD_Identifier/code/CharacterString

shall have the form:

http://wis.wmo.int/WMOVolumeAGazeteer.xml#WMOStationNumber.

The CT_CodelistCatalogue includes a codeEntry element for each item in the catalog. The codelistItem/CodeListDictionary/codeEntry/description shall include the bounding box or location and  elevation for the item in well-known text (http://en.wikipedia.org/wiki/Well-known_text).

Note: If the WMO Station Index refers to separate locations for surface and upper air observations, the location shall be described as MULTIPOINTZ((LON1 LAT1 ELE1),(LON2 LAT2 ELE2)).

[Action MDI-1-vi: Manuel & Jeremy (& Alexander Besprozvannykh?) - develop simple Vol A gazetteer]

2.1.2.22 More sophisticated gazetteers are desirable for future implementations, perhaps providing a Web Feature Service interface. UK NERC has some experience of establishing sophisticated gazetteer services. Also, Australian NODC have funding request in place (pending decision in June 2010) to set up a gazetteer service via 12-month project.

[Action MDI-1-vii: Greg - liaise with groups building gazetteer services and provide implementation experience and project updates to IPET-MDI]

Automated validation

2.1.2.23 A set of conformance classes will be establish for validating compliance of metadata records against the WMO Core Profile. Each conformance class will vary in its scope of validation and will have an associated conformance test that can be executed automatically.

Recommendation 17: ISO19115 / ISO19139 compliance will be assessed for all metadata records

Candidate conformance tests for ‘vanilla’ ISO19115 include:

· Validate against ISO19139 schema

· Validate ISO19115 condition statements (schematron)

· Validate ISO19115 code-lists (schematron)

Recommendation 18: metadata describing GTS products intended for global exchange must conform to the WMO Core Profile metadata standard.

Recommendation 19: metadata records from DCPCs and NCs should employ the WMO Core Profile metadata standard. However, in order to be inclusive of other data-provider communities, this is not mandatory. Compliance with ISO19115 remains a mandatory requirement.

2.1.2.24 Candidate conformance tests for WMO Core-Profile compliance include:

· Validate WMO Core Profile extensions: condition statements (schematron)

· Validate WMO Core Profile extensions: code-lists - extending to include thesauri, thematic hierarchies, keyword lists, station-gazetteer (schematron)

· Validate Extension metadata for WMO Core Profile

[Action MDI-1-viii: Ted - define the WMO Core Profile extension object]

[Action MDI-1-ix: Jeremy - define the full set of WMO Core Profile code-lists]

Recommendation 20: WMO Core Profile compliance conformance tests will create warnings if a deprecated practice is identified.

Recommendation 21: a central repository of code-lists (etc.) must be established in order that the schematron rule-sets can validate against an official source.

Recommendation 22: unknown extensions to ISO19115/ISO19139 will be ignored unless (a) they conflict with WMO Core Profile, or the extensions are sufficiently documented in order to validate against (see below)

Recommendation 23: Where WMO Core Profile metadata records include multiple language definitions, validation will ensure that English definitions are present at a minimum.

2.1.2.25 Candidate conformance tests for validating other ISO19115 profiles include:

· Validate extension metadata

· Validate extensions as directed by extension metadata; i.e. supplementary schema, constraints and codelists

[Action MDI-1-x: Jean Pierre & Jeremy - review candidate conformance tests & rank in order of validation]

[Action MDI-1-xi: Jean Pierre with guidance from Greg & Ted - develop schematron rule-sets to implement the conformance tests, using the geonetwork & ANZLIC schematron rules as a baseline. The focus will be on validation against the WMO profile. Developing validation tests against other profiles by interpretting the ISO19115 extension record is a significantly lower priority]

Recommendation 24: conformance tests (including schematron rule-sets) will be published via a central repository

2.1.2.26 Schema validation tools commonly check for the existence of elements rather than whether or not they have content. It was noted that, in order to unambiguously identify empty elements, geonetwork automatically adds a “nilReason” into a gco:characterString if it is empty.

Recommendation 25: validation of metadata must occur when metadata is harvested from DCPC or NC into the GISC catalogue. It is recommended (although not mandatory) that metadata is validated when synchronized between GISC-nodes to protect against inadvertent corruption during the harvesting process.

Human readable metadata & SRU query response

2.1.2.27 The response to a Z39.50 SRU query may contain multiple ‘results’. Whilst a particular result may be described using the full WMO Core Profile or ISO19139, efficiency suggests you only return a small subset of metadata for each.

Recommendation 26: use Timo’s existing SRU response schema.

[Action MDI-1-xii: Eiji - review Timo’s existing SRU response schema & develop to suit community needs]

2.1.2.28 Both SRU response and full metadata records must be human readable. In particular, to support Meteo France in the development of baseline metadata records for the GISC catalogue, a stylesheet must be provided that provides insightful guidance to the reviewers of the draft records (they will not be metadata experts!) and allows them to modify any mistakes in-situ. It was noted that ‘tool-tips’ (annotation from the schema documents) may be incorporated in this guidance. It was also noted that geonetwork’s “nested box” presentation format was simple to understand, suggesting that the geonetwork stylesheet may be a good starting place. NGDC have developed some sophisticated style-sheets that offer in-line editing and FAQ-style interfaces. Also NCAR have developed a 'nested box' UI to display metadata and NetCDF header information.

[Action MDI-1-xiii: Eiji + Michael + Ashok -  develop stylesheet to render human readable response to SRU query]

[Action MDI-1-xiv: Eiji + Michael + Ashok + Ted - develop stylesheet to render human readable WMO Core Profile records, presented in response to browsing the GISC catalogue]

[Action MDI-1-xv: Eiji + Michael + Ashok + Ted - develop a sophisticated stylesheet that provides guidance & in-line editing capabilities to support validation of baseline metadata developed by Meteo France (would this also provide a basic template for the minimum metadata footprint – i.e. be able to create a new metadata record from scratch?)]

Controlled vocabularies and ontologies

2.1.2.29 The user’s browse experience of the DAR catalogue can be greatly improved by providing one or more hierarchies of search terms. A user is able to browse these search terms in the knowledge that they do not have to guess how datasets have been categorized. Furthermore, these hierarchies can be used to assist metadata authors categorize their datasets so that they are readily discoverable. Particularly important is the thematic keyword taxonomy that is used to categorize the dataset. The MD_TopicCategory object enumerates a fixed set of categories and cannot be extended. Only a few values from MD_TopicCategoryCode are likely of interest within the WMO Core Profile:

· climatologyMeteorologyAtmosphere [TopicCatCd 004];

· environment [TopicCatCd 007];

· inlandWaters [TopicCatCd 012]; and

· oceans [TopicCatCd 014]

The WMO Core Profile requires further domain-specific categorization of data.

Recommendation 27: domain-specific categorization of data will be achieved using the

MD_Metadata/identificationInfo/MD_DataIdentification/descriptiveKeywords/MD_Keywords/keyword

element(s) (type = theme).

2.1.2.30 The WMO Common Codes table C13 provides the mandated categorization structures for WMO content. Table C13 is in the process of having a set of short-names added to supplement to numeric classification. This is currently incomplete, but it represents a creditable effort to establish a structured taxonomy of keywords consistent with GTS practices.

[Action MDI-1-xvi: Manuel + Simon - further develop the short-names for table C13]

Recommendation 28: for consistency, each WMO Core Profile compliant metadata record will contain at least one domain-specific categorization keyword from one of the following taxonomies:

· Nasa’s Global Change Master Directory (GCMD) [http://gcmd.nasa.gov/]

· CF convention standard names [http://cf-pcmdi.llnl.gov/documents/cf-standard-names/]

· WMO Common Codes table C13

· WMO #182 International Meteorological Vocabulary *

[*enhanced by Jean Pierre to a structured taxonomy: meeting document “IR4.JPA.v0.1.pdf” refers]

2.1.2.31 However, it was noted that the WMO International Meteorological Vocabulary does not appear to be maintained or governed.

Note: maintenance & governance of controlled vocabularies established by other commissions must be agreed (issue to be passed to ICT-ISS).

2.1.2.32 Other candidate taxa may be sourced from:

· Earth System Curator [http://www.earthsystemcurator.org/]

· ESMF (Earth System Model Framework) [http://www.cisl.ucar.edu/research/2005/esmf.jsp]

· Metafor [http://metaforclimate.eu/]

[Action MDI-1-xvii: Manuel + Jeremy + Ted - define (best) practices for referencing keywords from fixed vocabularies]

[Action MDI-1-xviii: Manuel + Michael + Jeremy - investigate the use of Protegé tools, RDF/OWL & SKOS Simple Knowledge Organization System [http://www.w3.org/2004/02/skos/] to build hierarchical taxonomies / simple ontologies from GCMD, CF standard names & WMO Common Codes table C13. Note that NCAR's portal codebase now includes these technologies]

[Action MDI-1-xix: Jeremy - Assess current best-practices for providing simple tools to help metadata authors browse ontologies or taxonomies or skos]

2.1.2.33 Other controlled vocabularies include:

· GTS product priority type

· Access control category code

· MD_MaintenanceFrequencyCode *

· ...

Recommendation 29: *use of bespoke extensions to the MD_MaintenanceFrequencyCode code-list will be prohibited. As an alternative the following must be used:

- MD_Metadata/metadataMaintenance/MD_MaintenanceInformation/maintenanceAndUpdateFrequency = “continual”

- MD_Metadata/metadataMaintenance/MD_MaintenanceInformation/userDefinedMaintenanceFrequency = « ISO8801-compliant duration »

This applies to both data and metadata maintenance. The example above is for metadata. For the reference, the XPaths are:

· MD_Metadata/identificationInfo/MD_DataIdentification/resourceMaintenance/MD_MaintenanceInformation/maintenanceAndUpdateFrequency

and

· MD_Metadata/identificationInfo/MD_DataIdentification/resourceMaintenance/MD_MaintenanceInformation/userDefinedMaintenanceFrequency

2.1.2.34 Other Technical Commissions should provide their keyword proposals based on their vocabularies and provide templates appropriate to commission’s work.

Other Technical Commissions should support such an activity by providing the crucial domain expertise and may already have ontology activities.

2.1.2.35 Controlled vocabularies from other commissions include:

· Agriculture (c. Byong-Lyol Lee [KMA - blleesnu@snu.ac.kr])

· Atmospheric Science (c. Joerg Klausen [chair of ET-WDC & GAWSIS] & Geir Braathen [WMO Secretariat - gbraathen@wmo.int])

o naming chemical compounds

o analytical methods used in atmospheric composition monitoring

o physical principles used in atmospheric composition monitoring

· Aeronautical

· Hydrology

· JCOMM (c. Greg Reed)

· CF-NetCDF

Recommendation 30: multi-language implementations of code-lists may be used.

[Action MDI-1-xx: Jeremy - add example of both specification & use of multi-language code-list]

Definition of access control policy controlled vocabulary & implementation in metadata record

2.1.2.36 The subject here is the access constraints for the target dataset.

Recommendation 31: candidate vocabulary terms for WIS access constraints include:

· Unrestricted

· Resolution 40 and Resolution 25 Essential Data

· Resolution 40 and Resolution 25 Additional Data

· Custom

2.1.2.37 Unrestricted is semantically equivalent to Resolution 40 and Resolution 25 Essential Data.

Continuing current GTS practices where RTHs are responsible for enforcing access control policies on behalf of data-providers, GISCs will be responsible for enforcing the access control policies cited in the metadata record for each product.

Recommendation 32: best practice suggests that the absence the useConstraints attribute:

MD_Metadata/identificationInfo/MD_Identification/resourceConstraints/MD_LegalConstraints/useConstraints

implies Unrestricted. However, the Resolution 40 and Resolution 25 Essential Data access control category should be used explicitly to indicate that data-providers are sharing this dataset under WMO Resolution 40 or Resolution 25.

Recommendation 33: the Custom access control category is used to indicate access control that is defined by the data-provider. The details of the policy must be referenced by URL.

Recommendation 34: the access control policy will be defined in the following elements:

· MD_Metadata/identificationInfo/MD_Identification/resourceConstraints/MD_LegalConstraints/useLimitation/gco:CharacterString = ”«category-code»”

· MD_Metadata/identificationInfo/MD_Identification/resourceConstraints/MD_LegalConstraints/useConstraints/MD_RestrictionCode = ”otherRestrictions”

· MD_Metadata/identificationInfo/MD_Identification/resourceConstraints/MD_LegalConstraints/otherConstraints/gco:CharacterString = ”«category-code»”

· «category-code»  will be a semi-controlled vocabulary for the WMO Core Profile  and can be either:

o “WMO Essential”

o “WMO Additional”

o «free-text»

“otherRestrictions” is part of the codelist MD_RestrictionCode

«free-text» is used to describe the custom access control policy and is recommended  to include the URL of a resource describing the access control policy in detail.

If the access is NOT controlled (i.e. unrestricted) the resourceConstraints element is omitted.

2.1.2.38 Example from NOAA

Metadata constraints (WMO probably has their own language...)
<gmd:metadataConstraints>
<gmd:MD_LegalConstraints>
<gmd:useLimitation>
<gco:CharacterString>While every effort has been made to ensure that these data are accurate and reliable within the limits of the current state of the art, NOAA cannot assume liability for any damages caused by any errors or omissions in the data, nor as a result of the failure of the data to function on a particular system. NOAA makes no warranty, expressed or implied, nor does the fact of distribution constitute such a warranty.</gco:CharacterString>
</gmd:useLimitation>
</gmd:MD_LegalConstraints>
</gmd:metadataConstraints>

Maintenance note
<gmd:metadataMaintenance>
<gmd:MD_MaintenanceInformation>
<gmd:maintenanceAndUpdateFrequency gco:nilReason="Unknown"/>
<gmd:maintenanceNote>
<gco:CharacterString>This metadata was automatically generated from the FGDC Conten
Standards for Digital Geospatial Metadatastandard version FGDC-STD-001-1998.</gco:CharacterString>
</gmd:maintenanceNote>
</gmd:MD_MaintenanceInformation>
</gmd:metadataMaintenance>

Note: ISO19115:2012 revision is likely to add a citation attribute that will enable the custom access control policy to reference the details. Resolution 40 and Resolution 25 may also be cited using this mechanism.

Miscellaneous metadata conventions

2.1.2.39 Root elements for metadata files:

Recommendation 35: the root element for each metadata record shall be MI_Metadata from ISO19115-2:2009. The selection of MI_Metadata as the root elements permits inclusion of content from part 2 of the ISO geographic metadata standard; namely the remote-sensing extensions.

The ISO19115-2 XML schema can be found at

http://www.ngdc.noaa.gov/metadata/published/xsd/schema/gmi/

[Action MDI-1-xxi: Ted - recommend best practice to enable transition from MD_Metadata to MI_Metadata - including reference 'official' schema location]

It is noted that the use of MI_Metadata as a root element will have a detrimental effect on the ability of geonetwork to validate and consume metadata records.

Recommendation 36: the continued use of  MD_Metadata as root element will be permitted. However, validation rules in conformance tests will be configured to flag this as a DEPRECATED practice. The validation warning will recommend use of MI_Metadata.

Recommendation 37: a single metadata file may contain a collection of metadata records. In this case, the root element must be a concrete sub-class of DS_Aggregate.

2.1.2.40 Metadata standard name:

Recommendation 38: MD_Metadata/metadataStandardName must always use the following text:

<gmd:metadataStandardName>
       <gco:CharacterString>ISO 19115-2 Geographic information — Metadata — Part 2: Extensions for imagery and gridded data</gco:CharacterString>
</gmd:metadataStandardName>

This text is the full title of the ISO19115-2 geographic information metadata standard.

2.1.2.41 Metadata standard version:

Recommendation 39: MD_Metadata/metadataStadardVersion must always use the following text:

<gmd:metadataStandardVersion>
       <gco:CharacterString>ISO 19115-2:2009-02-15</gco:CharacterString>
 </gmd:metadataStandardVersion>

This text is the standard short-name used to identify the ISO19115-2 geographic information metadata standard. Adherence to the WMO Core Profile is cited by the inclusion of an appropriate

MD_Metadata/metadataExtensionInfo/MD_MetadataExtensionInformation object:

/gmi:MI_Metadata/gmd:metadataExtensionInfo/gmd:MD_MetadataExtensionInformation/gmd:extensionOnLineResource/gmd:CI_OnlineResource/gmd:name/gco:CharacterString

contains human-readable concise name and version of the Profile, while

/gmi:MI_Metadata/gmd:metadataExtensionInfo/gmd:MD_MetadataExtensionInformation/gmd:extensionOnLineResource/gmd:CI_OnlineResource/gmd:linkage/gco:URL

contains machine-readable URL at which metadata release package for that version is located.

2.1.2.42 File conventions:

Recommendation 40: each metadata file must contain one, and only one, record. Valid root elements are:

· DS_Aggregate (implying all concrete sub-classes)

· MI_Metadata

· MD_Metadata [DEPRECATED]

Recommendation 41: each metadata file must be comprised of well-formed XML. [http://www.w3.org/TR/REC-xml/#sec-well-formed]

Recommendation 42: metadata files must be encoded in UTF-8 [RFC3626, http://en.wikipedia.org/wiki/UTF-8]

2.1.2.43 Use of Explicit Namespaces

Recommendation 43: The use of a default namespace is currently customary in XML records and is an acceptable construction in metadata XML for WIS. 

Including explicit namespace prefixes for all elements in an XML record is the recommended and preferred practice in metadata XML for WIS - but not mandatory.  In this practice all namespaces should be declared with a prefix and each element in the XML record should explicitly include its namespace prefix. 

Current Practice:

namespace declaration (in the root element):

xmlns=http://www.isotc211.org/2005/gmd <!--Default namespace declaration -->

example element:

<identificationInfo> <!-- default namespace assumed for element -->

Recommended Practice:

namespace declaration (in the root element):

xmlns:gmd=http://www.isotc211.org/2005/gmd <!--Explicit namespace declaration -->

example element:

<gmd:identificationInfo> <!--namespace prefix included in element -->

[Action MDI-1-xxii: Michael - add examples with agreed upon reference URLs for schemas]

2.1.2.44 Conventions for specifying distribution information

A consistent approach for specifying the onlineResource for GTS datasets is encouraged.

Recommendation 44: for GTS products intended for global exchange, the onlineResource attribute of the MD_Distributor object should conform to the following syntax:

· http://«hostname»/«path»/«MD_Metadata/fileIdentifier»

where:

· «hostname»/«path»/ refer to the location of the web-service that serves the GTS product on behalf of the data-owner. This will normally be the NC or DCPC who is responsible for publishing the product, but may be a GISC node who has agreed to serve the ‘source’ data-files on their behalf.

· «MD_Metadata/fileIdentifier» is the globally unique identifier of the metadata record that describes the product being served.

Example implementation of such web service is available at

http://www.gisc.kishou.go.jp/cgi-bin/gisccache; in this case

http://www.gisc.kishou.go.jp/cgi-bin/gisccache/urn:x-wmo:md:int.wmo.wis::SMJP02RJTD should be onlineResource for SMJP02 RJTD.

GISCs will be able to identify GTS products (for global exchange) using the citation authority in the mandatory «MD_Metadata/fileIdentifier», being "int.wmo.wis", irrespective of the approach used to specify the online resource. Once identified as GTS products, the GISCs will amend the metadata served in response to a query to offer access to the local GISC cache in addition to the original source.

Recommendation 45: when serving querying responses from the GISC search portal, GISCs will amend the information served to the user by inserting an additional MD_Distributor object that describes their local GISC-cache. This gives the recipient a choice of destinations: the original source or the local cache. Such amended metadata must not be re-distributed for other GISCs]. There is no restriction on the format or structure of onlineResource attributes describing the local GISC-cache. Also note that there is no restriction on the format or structure of onlineResource attributes describing products that are not globally exchanged: i.e. those served only from an NC or DCPC and do not form part of the GISC cache.

This recommendation should ensure compliance with INSPIRE which mandates a gmd:transferOptions element for every "URL available to obtain more information on the resources and / or access related services". Each additional MD_Distributor object will contain an additional transferOptions element.

An 'empty' gmd:distributionInfo object is included by way of example:

<gmd:distributionInfo>
    <gmd:MD_Distribution>
        <gmd:distributor>
            <gmd:MD_Distributor>
                <gmd:distributorContact>

                    ...

                </gmd:distributorContact>
                <gmd:distributorTransferOptions>                               

                    <gmd:MD_DigitalTransferOptions>

                        <gmd:onLine>

                            <gmd:CI_OnlineResource>

                                <gmd:linkage>

                                    <gmd:URL>XXX</gmd:URL>

                                </gmd:linkage>
                                <gmd:protocol>

                                    <gco:CharacterString>WWW:LINK-1.0-http--link</gco:CharacterString>

                                </gmd:protocol>
                                <gmd:name>

                                    <gco:CharacterString>XXX</gco:CharacterString>

                                </gmd:name>
                                <gmd:description>

                                    <gco:CharacterString>XXX</gco:CharacterString>

                                </gmd:description>

                            </gmd:CI_OnlineResource>

                    </gmd:onLine>
                      </gmd:MD_DigitalTransferOptions>

                   </gmd:transferOptions>

               </gmd:distributorTransferOptions>

            </gmd:MD_Distributor>
        </gmd:distributor>
    </gmd:MD_Distribution>

</gmd:distributionInfo>

The following schematics provide more information:

Example:

· Original onlineResource citation:

<gmd:URL>

http://gisc.dwd.de/GISC_DWD/Level/2/urn:x-wmo:md:int.wmo.wis::SMAA01EDZW

</gmd:URL>

· Additional onlineResource citation for DWD GISC-node:

<gmd:URL>

http://gisc.dwd.de/GISC_DWD/retrieve.do?pidpat=urn:x-wmo:md:int.wmo.wis::SMAA01EDZW&TOPLEVEL=False

</gmd:URL>           

Readers should note that the «hostname» & path «path» are defined by the data-provider to suit their local policies, or those of their delegated agents where a GISC serves the source data-files on their behalf.

Metadata management policies

2.1.2.45 Metadata ownership & publication

Recommendation 46: Continuing current GTS practices and procedure, metadata ownership resides with the data-provider organization. Only the data-provided organization has the authority to create or update metadata for their products they provide.

2.1.2.46 Each data-provider is responsible for the accuracy of their metadata. Note that data that does not have a corresponding metadata record will not be visible in the catalogue: the implication is that it will not be discoverable!

Recommendation 47: the maintenance schedule for metadata will be identical to those in the Manual on GTS for maintaining Vol. C1

2.1.2.47 Each NC or DCPC will be affiliated with a specific GISC. NC or DCPCs must agree a mechanism to publish (& delete) metadata records with their affiliated GISC.

Recommendation 48: bi-lateral agreement between NC / DCPC and their affiliated GISC must include information on metadata exchange and deletion mechanisms.

2.1.2.48 GISCs do not create any original metadata – they only publish content from NCs and DCPCs. However, it is the GISC’s responsibility to ensure that their affiliated NCs or DCPCs are complying with the metadata management policies. The WMO Secretariat currently maintains a list of additional data separately from Volume C1 on the basis on the information provided by the Permanent Representatives.

2.1.2.49 As noted in the previous discussion on dataset hierarchies, some elements of metadata records can be normalized into static metadata components and referenced via xlink from within the metadata records. Use of this mechanism is considered to simplify metadata management as all metadata records who reference a given component will automatically be updated if the source metadata is updated. Depending on consumer-preference, these xlink references may be resolved or not: retaining the xlink reduces the metadata record size, resolving the xlink (i.e. inserting the content from the metadata component) simplifies parsing.

[Action MDI-1-xxiii: Ted, Michael & Jean Pierre - define best practices relating to resolving metadata components – especially with reference to the implications of updating metadata components & needing to propagate changes to impacted metadata records.]

2.1.2.50 Creation of initial 'baseline' metadata records from WMO No. 9

Météo-France had developed a software package for the conversion of WMO No. 9 Volume C1 into ISO 19115 compliant metadata records. The procedure creates a DAR metadata record compliant with version 1.1 of the WMO Core Profile for every bulletin found in Volume C1. Information is extracted from the bulletin definition and enriched with additional sources of information, including WMO documents and references – WMO No. 9 Volume A, WMO No. 306 Manual on Codes, WMO No. 386 Manual on the GTS -.Rules are created to interpret and mine information from Volume C1 free-format fields. They reflect recurring syntaxes found in Volume C1. These rules are provided in annex 1 to IPET-MDI-I/Doc. 2.1.2(3). The meeting stressed the need for Members to follow these rules to ensure the best metadata records are created. The recommendations of the IPET-MDI-I as regards the implementation of the WMO core profile will be introduced by Météo-France in a new version of the software package provided they do not necessitate long developments.

2.1.2.51 Procedure for generation and validation of baseline metadata records

Météo-France proposed the following procedure for the generation and validation of the catalogue of metadata  records for the bulletins of Volume C1:

· With the assistance of the Secretariat, Météo-France will implement the pre-validation procedure defined in paragraph 4 of IPET-MDI-I/Doc. 2.1.2(3);

· Météo-France will generate the metadata records for every WMO Member, and make them available to Members. The WMO Secretariat will inform Members of the availability of GTS metadata records and validation procedure;

· Members will review the records and edit all necessary changes. If necessary, Météo-France will possibly provide guidance during this process,

· Once editing is complete, Members will provide their final set of GTS metadata records to Météo-France, for insertion in the reference set,

· Météo-France will provide the complete final set of GTS metadata to WMO.

2.1.2.52 Support for non-expert metadata authors / validators

The meeting stressed the need for a sophisticated stylesheet that provides guidance & in-line editing capabilities to support validation of baseline metadata developed by Meteo France (see above paragraph 2.1.2.28).

The maintenance of the DAR catalogues will require that the WMO Members provide the information on additional data in the DAR catalogue and this will duplicate the information provided to the Secretariat, with possible differences in the information.  Therefore, the creation of a DAR metadata catalogue including information on additional data would lead to the parallel maintenance of the two lists, each with the potential to appear as redundant and sources of confusion.

Recommendation 49: Arrangements between GISCs, MTN centres and the Secretariat, including a time schedule for action, should be developed:

(a) To implement the pre-validation procedure proposed by Météo-France and leading to a complete final set of GTS metadata:

(b) To ensure that the updates to Volume C1 issued after the production of the complete final catalogue of GTS metadata records by Météo-France be included in the catalogue of metadata records, until Volume C1 be replaced by the DAR catalogue.

The meeting recommended that procedures for maintaining Volume C1 the declaration of additional data be reconsidered and agreed to submit this issue to the next ICT-ISS. ET-OI should be involved in the development of these arrangements.

[Action MDI-1-xxiv: Guofu + Eiji, Ashok, Siegfried / Jürgen & Jean Pierre - to establish & lead a sub-group to assist the WMO Secretariat [Pierre] specify how the transition from Volume C1 to the WIS DAR catalogue will be achieved. Of particular interest will be the recommendation of operational practices whilst Volume C1 is being maintained in parallel with the new DAR catalogue while the WIS infrastructure matures.]

2.1.2.53 Marking metadata as draft

To support the initial implementation of WIS, Metéo France will create a complete set of metadata records for the GTS products intended for global exchange. These metadata records will form a baseline for the DAR catalogue until the data-owners (i.e. WMO member-states & organizations) are ready to begin maintenance of their metadata records. As these records are created by Metéo France rather than the official data-owner, these records will be marked as draft.

Once the metadata has been validated by the appropriate member organization, ownership will be transferred and the 'DRAFT' status will be removed. In order to ensure that DRAFT status is amended during validation, the team recommend a scheme whereby the member organization will need to overwrite an element with their own information.

The following schemes were reviewed but discarded:

· MD_Metadata/metadataMaintenance/MD_MaintenanceInformation/maintenanceNote = “DRAFT”

· MD_Metadata/metadataConstraints/MD_Constraints/useLimitation = “DRAFT”

Recommendation 50: Météo France ‘baseline’ metadata will be marked as DRAFT using the following scheme:

· gmd:MD_Metadata/gmd:metadataMaintenance/gmd:MD_MaintenanceInformation/gmd:contact/gmd:CI_ResponsibleParty/gmd:organisationName = "Meteo France (on behalf of «NC or DCPC» by interim agreement with ICG-WIS)"

· gmd:MD_Metadata/gmd:metadataMaintenance/gmd:MD_MaintenanceInformation/gmd:contact/gmd:CI_ResponsibleParty/gmd:contactInfo/gmd:CI_Contact/gmd:address/gmd:CI_Address/gmd:electronicMailAddress = "quelqu-un@meteo.fr"

2.1.2.54 Multi-language metadata

Recommendation 51: the following policies will apply to multi-language metadata records:

- At a minimum, metadata will be published in English

- An abstract must primarily be in English; alternative language versions may be provided

- GISC systems will always operate on the primary English metadata

- Where a 3rd party wishes add translation to a metadata record, they must propose changes to the metadata custodian (data-owner) for the record. Only the metadata custodian (or their delegated agent) has the authority to update the metadata. Updates will be propagated via the WIS via the normal mechanism.

The default language and character set of the metadata record is explicited in the MD_Metadata/language and MD_Metadata/characterSet elements:

<gmd:language>
   <gmd:LanguageCode codeList="http://wis.wmo.int/2006/catalogues/gmxCodelists.xml#LanguageCode" codeListValue="eng"/>
</gmd:language>
<gmd:characterSet>
    <gmd:MD_CharacterSetCode codeList="http://wis.wmo.int/2006/catalogues/gmxCodelists.xml#MD_CharacterSetCode" codeListValue="utf8"/>
</gmd:characterSet>

Each alternate language for the metadata is defined via a MD_Metadata/locale element:

<gmd:locale>
   <gmd:PT_Locale id="locale-fr">
      <gmd:languageCode>
         <gmd:LanguageCode codeList="http://wis.wmo.int/2006/catalogues/gmxCodelists.xml#LanguageCode" codeListValue="fra"/>
         </gmd:languageCode>
      <gmd:characterEncoding>
          <gmd:MD_CharacterSetCode codeList="http://wis.wmo.int/2006/catalogues/gmxCodelists.xml#MD_CharacterSetCode" codeListValue="utf8"/>
      </gmd:characterEncoding>
  </gmd:PT_Locale>

</gmd:locale>

Each metadata element with a CharacterString type and free text domain can be instantiated with a gmd:PT_FreeText_PropertyType type:

<gmd:abstract xsi:type="gmd:PT_FreeText_PropertyType">
<gco:CharacterString>Abstract in english</gco:CharacterString>
    <gmd:PT_FreeText>
        <gmd:textGroup>
            <gmd:LocalisedCharacterString locale="#locale-fr">Résumé en français</gmd:LocalisedCharacterString>
        </gmd:textGroup>
        <gmd:textGroup>
            <gmd:LocalisedCharacterString locale="#locale-sp">Resumen en espanol</gmd:LocalisedCharacterString>
        </gmd:textGroup>
    </gmd:PT_FreeText>
</gmd:abstract>

In the example above, two alternate languages are assumed to be defined. The abstract element is provided in the default language with a common gco:CharacterString element. An additional gmd:PT_FreeText element is present, containing possibly a gmd:textGroup element for every alternate translation.
Translations may be defined in translation files instead of being embedded in the metadata record.

In this case, it is simpler to group in a single file the definition of one alternate language and the associated translations, each translated string identified by a unique id attribute. Here is an example translation file:

<gmd:PT_LocaleContainer
xmlns:gco="http://www.isotc211.org/2005/gco"
xmlns:gmd="http://www.isotc211.org/2005/gmd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.isotc211.org/2005/gmd http://www.isotc211.org/2005/gmd/gmd.xsd">
<!-- gmd:description element not shown -->
<gmd:locale>
    <gmd:PT_Locale id="locale-fr">
        <gmd:languageCode>
            <gmd:LanguageCode codeList="http://wis.wmo.int/2006/catalogues/gmxCodelists.xml#LanguageCode" codeListValue="fra"/>
        </gmd:languageCode>
        <gmd:characterEncoding>
            <gmd:MD_CharacterSetCode codeList="http://wis.wmo.int/2006/catalogues/gmxCodelists.xml#MD_CharacterSetCode" codeListValue="utf8"/>
        </gmd:characterEncoding>
    </gmd:PT_Locale>
</gmd:locale>
<!-- gmd:date element not shown -->
<!-- gmd:responsibleParty element not shown -->
<gmd:localisedString>
    <gmd:LocalisedCharacterString locale="#locale-fr" id="abstractfr">Résumé en français</gmd:LocalisedCharacterString>
</gmd:localisedString>
<!-- other gmd:localisedString elements not shown -->
</gmd:PT_LocaleContainer>

The translated values of textual metadata elements are then referenced in the multi-lingual metadata as in:

<gmd:abstract xsi:type="gmd:PT_FreeText_PropertyType">
<gco:CharacterString>Abstract in english</gco:CharacterString>
    <gmd:PT_FreeText>
        <gmd:textGroup xlink:href="fr-fr.xml#abstractfr"/>
    </gmd:PT_FreeText>
</gmd:abstract>

Note: the object type MemberName is used within software systems like a foreign key. As such it is never translated in multi-lingual metadata.

2.1.2.55 Metadata deletion

Recommendation 52: Only the data-owner has the authority to delete metadata records. The circumstances allowing deletion of metadata records include:

-  When incorrect metadata is mistakenly or maliciously published

-  When product or information is no longer available the metadata record must be removed from the DAR catalogue

Recommendation 53: When GISC is requested to delete a metadata record it must remove the record from the DAR catalogue (so that it can no longer be discovered), but stores the record temporarily for a period of time prior to purging. Recommended ‘grace period’ is 7-days.

2.1.2.56 An NC or DCPC may choose to maintain a historical archive of metadata records, even if the dataset is no longer available. OAI-PMH is the mandatory mechanism for GISC-to-GISC synchronization of DAR catalogues OAI-PMH can (optionally) support propagation of deletion.

Recommendation 54: the OAI-PMH protocol will be used to achieve GISC-to-GISC synchronization of deleted records.

2.1.2.57 Notifying an affiliated GISC regarding the deletion of metadata records is a subject to local agreement.

Recommendation 55: bi-lateral agreement between NC / DCPC and their affiliated GISC must include information on metadata exchange and deletion mechanisms.

Potential mechanisms may include:

· If OAI-PMH harvesting used to publish metadata to the GISC, use the OAI-PMH protocol; or

· Publication of a service message (format to be agreed bi-laterally) to GISC requesting deletion of record (using MD_Metadata/fileIdentifier as primary key); or

· GISCs may provide metadata editor to allow data originator to delete the record themselves.

2.1.2.58 Implementation note from DWD:

jOAI monitors each directory of files that is configured in the system and automatically adds, updates or deletes items from the OAI repository as files are added, updated or deleted from the directories. After the initial configuration, the synchronization between the files and the OAI repository occurs automatically every 8 hours or may be synchronized manually at any time.

2.1.2.59 Publication of Metadata of ISO copyright material

Agreement between WMO & ISO TC/211 secretariats allows the use of final draft docs for development of WMO Core Profile. Publications from WMO must include note referencing copyright remains with ISO. WMO are requested to re-publish only content that is deemed of interest for our community. The implication is that IPET-MDI can publish a document (or documentation package) that includes all necessary ISO copyright material thus removing the need for WMO documentation to refer to external ISO standards documents.

It should be noted that the agreement between WMO and ISO does not extend to IOC, thus the inclusion of JCOMM is to be confirmed.

2.1.2.60 Guidance notes for metadata authors

Ted Habermann presented the guidance material developed for NOAA. This included:

Hyper-linked documentation

· ‘Rubric’ / training aids – stylesheets that provide metadata quality indexes & guidance on how to fix issues. A number of different rubrics were displayed, each responding to a specific training requirement.

· FAQ stylesheets

· Stylesheets that provide editing interfaces for metadata with built-in guidance.

More information can be found here: https://www.nosc.noaa.gov/dmc/swg/wiki/index.php?title=Creating_Good_Documentation

Recommendation 56: IPET-MDI will adopt a similar package of guidance material as NOAA.

Metadata standards maintenance

2.1.2.61 Versions and future update cycles

As one of the recommendations made by IPET-MDI is the use of MI_Metadata as the recommended root element. This implies inclusion of ISO19115-2:2009.

Recommendation 57: The next release of WMO Core Profile metadata standard will by 1.2

Recommendation 58: Subsequent releases of the WMO Core Profile metadata standard will employ tertiary / minor updates; 1.2.1, 1.2.2, 1.2.3 etc.

2.1.2.62 During the time until Congress 2011, the WMO Core Profile metadata standard is expected to have a number of updates. A release process for this period is defined below. After Congress 2011 a formal process similar to that used for the Manual on Codes may be defined using an annual update and approval cycle.Milestone dates  are:

· v1.2: 4^th June 2010 – publication of v1.2 including recommendations from IPET-MDI-1 & support of WMO ET-GDDP assessment phase

· v1.2.1: 27^st August 2010 – start of preparations for CBS demo

· v1.2.2: 1^st October 2010 – CBS demo

· v1.2.3: 11^th April 2011 – next ICG-WIS meeting & preparations for Congress demo in mid-May

2.1.2.63 v2.0 is expected to incorporate a move to ISO19115:2012 and harmonization with JCOMM & Hydrology metadata profiles. The timescales for ISO19115 revision are:

· Committee Draft (Nov 2010),

· Draft International Standard [DIS] (2011),

· Final publication ISO19115:2012

Revision timelines for ISO19115 will have significant impact on the timescales for moving WMO Core Profile metadata standard to v2.0.

2.1.2.64 Candidate release process (until congress 2011):

· OPAG-ISS / WISPO will establish a set of agreed release milestones for metadata guidelines. These will be organized to support the WIS project plan; e.g. guidelines released with sufficient lead-time ahead of major deliverables. The release schedule is critically important to maintain as this is the 'tempo' of change; implementers can plan to adapt at known dates and be assured that there will be no change until the next release milestone. Each release milestone must clearly define the reasons why it exists - what future project deliverable / outcome does it support?

· JIRA release management software [web-based service, provided by NCAR (Don Middleton)] used to capture issues.

· IPET-MDI chair will distribute 'problem tickets' to the team, whereupon they will be assessed each in turn against the growing historical knowledge base and 'inside' knowledge.

· Where IPET-MDI deem it necessary to propose new metadata usage guidelines to clarify or resolve issues, such problem tickets will be allocated to the AGREED release milestones in consultation with ET-WISC.

· Where metadata usage guidelines are considered to impact the GISC implementation community, then a future release milestone will be identified where the guideline becomes BINDING. Until that time, the guideline is not enforceable but will be marked as DEPRECATED in the automated validation. Any instances of deprecated practices will identify when they are due to become binding.

· IPET-MDI will develop & publish the guidelines and associated collateral material (including updates to the automated validation scripts) in time to meet the release milestone.

· Release package ‘manifest’ – host @ wis.wmo.int/2010/...

o Schema(s) iso19139

o Schematron rules

o Controlled vocabularies, code-list extensions (& gazetteers)

o Guidance notes – wiki (& later PDF document compiled by WISPO-contractor)

o Sample metadata

o Static metadata components & component management tools

o Conformance tests

o Stylesheets + ‘rubric tests’ [prioritize ‘minimum metadata’ requirements]

o Metadata editor stylesheets; template metadata

o UML

Recommendation 59: the candidate release process was approved.

Recommendation 60: all material pertaining to a release of the WMO Core Profile standard will be published to:

- http://wis.wmo.int/«YYYY»

Note that GISCs are likely to keep a local copy of these entities in order to speed validation etc.

The WMO Secretariat will support the release process.

The IPET-MDI team will work closely with the WISPO and their nominated expert contractor to further develop the metadata guidelines into the Manual on WIS.

2.1.2.65 ISO19115:2012 candidate revisions

Outputs from ISO19115 revision team that are being considered:

· Newconcept: deprecation ... things in 19115 will be deprecated to ensure backward compatibility whilst allowing strong guidance on preferred methods ... as much as one would want to make the new stuff mandatory, you cannot without impacting backward compatibility!

· MD_MetadataHierarchyLevel

· hierarchyLevel

· hierarchyLevelName

· ... to ensure that name & level are connected somehow

· MD_Reference

· identifier

· onlineResource

· ... characterstring vs identifier [MD_Identifier] ... people are including the namespace in the identifier which  is really a citation to the authority ... MD_Identifier includes a unique identifier (i.e. 32-byte UUID) + citation authority (namespace: uk.gov.metoffice)

· ... onlineResource allows explicit reference to the URL where the object can be found

· ... *may* be replaced with just a Citation object – which would force the addition of title and date

3. AWARENESS OF THE DEVELOPMENT OF A WMO CORE PROFILE OF THE ISO 19100 SERIES OF STANDARDS FOR DATA AND METADATA WITHIN THE WMO COMMUNITY

3.1 There is a special relationship between WIGOS (WMO Integrated Observing System http://www.wmo.int/pages/prog/www/wigos/index_en.html) & WIS (WMO Information System http://www.wmo.int/pages/prog/www/WIS/). It should be noted that WIGOS depends on the successful implementation of WIS but not vice-versa. Fundamental to the success of WIGOS will be the ability of the WIS infrastructure to cope with the diverse information requirements from the integrated sensor networks. This will require further evolution of the metadata and data standards managed by IPET-MDI.

http://www.gisc.kishou.go.jp/news/2010/0309workshop_in_Tokyo/WIS%20Workshop%20in%20TOKYO%20March%202010/WS_2010_0309/1.JMA%20WorkShop%202010-WIGOS%20Overview.pdf

3.2         The meeting noted the information on the development of standards for WMO data and associated metadata relevant to WIGOS given in the document “WIGOS standardization framework for data and associated metadata” (see IPET-MDI-I/Doc. 4.2(1). It invited the Secretariat to continue updating the document “WIGOS standardization framework for data and associated metadata” with a view to fostering the awareness of the development of the standards for data and metadata for the development of the WIGOS.

4. WORK PLAN OF THE IPET-MDI

4.1 Actions arising from this meeting

The meeting agreed on the action sheet included in Annex to this paragraph. The meeting noted that the chair of the IPET-MDI will submit the recommendations of the IPET-MDI to the meeting of the OPAG-ISS Implementation Coordination Team on ISS (ICT-ISS) scheduled in Geneva from 27 to 30 September 2010. The outcomes of the meeting of the ICT-ISS will be submitted to the Extra-ordinary session of CBS (Windhoek, Namibia, 17-24 November 2010).

4.2 Tooling to support collaborative development of WMO Core Profile standard

Wikis

4.2.1 The WIKI requirements for online tools are:

- Hyper-text documentation

- Version control updates

- The ability to rapidly ‘dump’ content for group assessment or ad-hoc discussion

- The ability to organize content for improved clarity

Recommendation 61: IPET-MDI adopt the WIS-WIKI [http://www.wmo.int/pages/prog/www/WIS/wiswiki/tiki-index.php]

Recommendation 62: use of IPET-MDI Huddle site will discontinued

[Action MDI-1-xxv: Jeremy (Timo to assist?) - migrate useful content from IPET-MDI Huddle site to WIS WIKI & retire]

4.2.2    Working practices on WIS WIKI

- Do not delete content from wiki-pages; strikethrough text then add your name and editorial comment. The ‘document owner’ will resolve the amendments

- There is no ability to check out attached documents for update. Please be aware of other users and avoid over-writing their changes. It is your responsibility to merge content from updated source & your amendments.

4.2.3 The use of Tiki-wiki software to underpin the WIS WIKI caused concern amongst the team. Recommended alternatives included Confluence

[http://www.atlassian.com/software/confluence/] and MediaWiki

[http://www.mediawiki.org/wiki/MediaWiki]. Both of these tools have the capacity

to publish PDF document output from WIKI-pages (e.g.

http://www.mediawiki.org/wiki/Extension:EPubExport).

[Action MDI-1-xxvi: Ted, Dave & Timo - discuss choice of wiki software for future robustness & capability]

4.2.4 The version control capabilities of the wiki may be sufficient for our needs. Alternatively, we may use subversion [svn - http://subversion.tigris.org/]

[Action MDI-1-xxvii: Michael - assess whether NCAR could host a WIS subversion repository]

4.2.5 The OCG MetOcean Domain Working Group wiki is the location where details of the conceptual modelling activity are posted:

http://external.opengeospatial.org/twiki_public/bin/view/MetOceanDWG/WebHome

Release management tools

4.2.6 JIRA is being used, courtesy of NCAR, to manage problem tickets and releases for WIS.  The JIRA online tool can be found here:

https://vets.development.ucar.edu/jira/secure/IssueNavigator.jspa?reset=true&mode=hide&jqlQuery=component+%3D+IPET-MDI+AND+project+%3D+WIS.  

All IPET-MDI members have been added to this JIRA WIS user list. Members are directed to use IPET-MDI component of the WIS project to filter for IPET-MDI issues.

5. CLOSURE OF THE MEETING

The meeting was closed on 29 April 2010 at 18:20 p.m.

Annex to paragraph 1.1.1

List of participants

CHAIRPERSON:

Mr Jeremy TANDY UK Met Office

Fitzroy Road

EX1 3PB Exeter

UNITED KINGDOM

Tel.: +44 1392 886584

Fax: +44 1392 885681

E-mail: jeremy.tandy@metoffice.gov.uk ]

E-mail: jeremy.tandy@gmail.com

VICE-CHAIR:

Mr Guofu WANG China Meteorological Administration

National Meteorological Information Centre

46, Zhongguancun Nandajie

100081 Beijing

CHINA

Tel.: +86 10 6840 7274

Fax: +86 10 6217 3225

E-mail: wanggf@cma.gov.cn

MEMBERS:

Mr Jean-Pierre AUBAGNAC Météo-France

42, Avenue Gustave Coriolis

31057 Toulouse

FRANCE

Tel.: +33 561078245

Fax: +33 561078109

E-mail: jean-pierre.aubagnac@meteo.fr

Mr Michael BUREK National Center for Atmospheric Research

1850 Table Mesa Drive

P. O. Box 3000

80305 Boulder, Colorado

USA

Tel.: +1 303 497 1202

Fax: +1 303 497 1286

E-mail: mburek@ucar.edu

Dr Simon ELLIOTT EUMETSAT

EUMETSAT-ALLEE 1

D-64295 Darmstadt

GERMANY

Tel.: +49 6151 807385

Fax: +49 6151 807304

E-mail: simon.elliott@eumetsat.int

Mr Siegfried FECHNER Deutscher Wetterdienst (DWD)

Frankfurter Str. 135

D-63067 Offenbach

GERMANY

Tel.: +49 69 8062 2865

Fax: +49 69 8062 3566

E-mail: siegfried.fechner@dwd.de

Mr Manuel FUENTES ECMWF

Shinfield Park

Reading RG2 9AX

UNITED KINGDOM

Tel.: +44 1189 499387

Fax: +44 1189 869450

E-mail: manuel.fuentes@ecmwf.int

Mr Ted HABERMAN NOAA/NGDC E/GCI

325 Broadway

Boulder Colorado 80305

USA

Tel.: +1 303 497 6472

Fax: +1 303 497 6513

E-mail: ted.habermann@noaa.gov

Mr Ashok K. JASWAL India Meteorological Department

National Data Centre

Shivajinagar

Pune 411005

INDIA

Tel.: +20 2 5535211

Fax: +20 2 5535435

E-mail: akjaswal@imdpune.gov.in

Mr Eiji TOYODA Japan Meteorological Agency (JMA)

1-3-4 Otemachi, Chiyoda-Ku

Tokyo 100-8122

JAPAN

Tel.: +81 80 51806841

Fax: +81 3 32118404

E-mail: toyoda.eizi@gmail.com

TECHNICAL COMMISSIONS REPRESENTATIVES:

JCOMM

Mr G. Reed Australian Ocean Data Centre Joint Facility

Fleet Headquarters

Wylde Street

Potts Point NSW 2011

AUSTRALIA

Tel.: +61 2 9359 3141

Fax: +61 2 9359 3120

E-mail: greg@metoc.gov.au

WMO SECRETARIAT:

C/DRMM Mr Pierre KERHERVÉ

WIS/PM Mr David THOMAS

SO/DRMM Mr Atsushi SHIMAZAKI