WORLD METEOROLOGICAL ORGANIZATION

COMMISSION FOR BASIC SYSTEMS

 

MEETING OF THE

EXPERT TEAM ON INTEGRATED DATA MANAGEMENT

FINAL REPORT

 

GENEVA, 5 - 8 NOVEMBER 2001


DISCLAIMER

Regulation 42

Recommendations of working groups shall have no status within the Organization until they have been approved by the responsible constituent body. In the case of joint working groups the recommendations must be concurred with by the presidents of the constituent bodies concerned before being submitted to the designated constituent body.

Regulation 43

In the case of a recommendation made by a working group between sessions of the responsible constituent body, either in a session of a working group or by correspondence, the president of the body may, as an exceptional measure, approve the recommendation on behalf of the constituent body when the matter is, in his opinion, urgent, and does not appear to imply new obligations for Members. He may then submit this recommendation for adoption by the Executive Council or to the President of the Organization for action in accordance with Regulation 9(5).


CONTENTS

Pages

Agenda

ii

Executive summary

iii

General summary of the work of the session

1

List of participants

7

Glossary

11

References

12


AGENDA

1. ORGANIZATION OF THE MEETING

1.1 Opening remarks
1.2 Adoption of the agenda
1.3 Working arrangements

2. REVIEW OF WMO REQUIREMENTS FOR A METADATA STANDARD

3. REVIEW OF EXISTING RELEVANT METADATA STANDARDS

4. RECOMMENDATIONS ON COMPREHENSIVE WMO METADATA STANDARDS

5. XML STRUCTURES FOR REPRESENTING METADATA

6. FURTHER DEVELOPMENT

7. CLOSURE OF THE MEETING


Executive Summary

A meeting of the CBS Expert Team on Integrated Data Management was held 5 to 8 November 2001 in Geneva. The experts, understanding that metadata means different things to different people, the team worked to develop a metadata framework, which at the highest level is applicable to all WMO tasks, but which can be extended to specific areas and to new ventures in an acceptable and standardized form.

The experts, recognizing that there were existing metadata standards that could be applicable to WMO requirements, reviewed a number of relevant standards. After their review they examined the Dublin Core and draft ISO standard Geographic Metadata (19115) most closely. They determined that the elements needed to meet WMO requirements could be considered to conform to Dublin Core and that the draft ISO standard could be applied to WMO requirements.

The ISO 19115 specifies a process (in ISO 19115 Annex C) where a community can adopt parts of the standard that it feels are relevant (including the "Core Elements") and also extend the elements, keywords and code table instances to suite that community. The team noted that the WMO might need to accept more than one such "Community Profile" (ISO 19115 Annex C.5) for the different WMO Programmes, but that there should be a Community Core Profile which could be adopted by all of WMO, with the potential for further extensions under ISO 19115 Annex C where necessary. With this process in mind the team developed a "WMO Community Core Metadata" profile. An overview of the ISO and WMO core elements and their corresponding names within the draft ISO standard is provided in the report of the meeting.

The team agreed that the various Programmes of WMO have a wide range of requirements for documenting their datasets. Thus, it would be difficult to develop a standard for comprehensive metadata that would meet the needs of all Programmes and development of a comprehensive WMO metadata standard would be a difficult, lengthy and expensive undertaking. The team will investigate this issue further over the next several months.

The team will also review the proposed core metadata standard with respect to existing datasets, develop a draft list of keywords to describe WMO datasets, develop list of proposed extensions needed for ISO code lists, and propose an XML schema and examples.


1. ORGANIZATION OF THE MEETING (agenda item 1)

1.1 Opening remarks

1.1.1 A meeting of the CBS Expert Team on Integrated Data Management opened at 0900 on Monday, 5 November 2001 in the headquarters of the WMO in Geneva. Mr S. Foreman (UK) chaired the meeting. Mr D. Schiessl, Director of the WWW Basic Systems Department, opened the meeting on behalf of the Secretary-General and welcomed the experts to Geneva. He outlined the objectives of the meeting and noted the primary purpose was to develop a metadata standard to be used in future WMO information systems as well as a standard Internet technology representation of the metadata for use on the World Wide Web for consideration by CBS-Ext.(2002).

1.2 Adoption of the agenda

1.2.1 The experts adopted the agenda as reproduced at the beginning of this report.

2. REVIEW OF WMO REQUIREMENTS FOR A METADATA STANDARD (agenda item 2)

2.1 The experts, understanding that metadata means different things to different people first considered a working definition of metadata for its deliberations. It agreed that at the most basic level metadata is data about data. More specifically, it is the descriptive data necessary to allow us to find, process and use data, information and products. While metadata generally can describe products, services and software as well as data at different stages of manipulation, it can also be a specification. Metadata can be extensive and all-inclusive, or it can be specific to a more limited function.

2.2 WMO Programmes and Members currently maintain a tremendous volume and variety of metadata. However little of this is in an electronic, standardized form which can be used to find data ("Navigational" metadata), and it is seldom distributed with the data or in a manner to allow the data to be recognized and understood by computer applications or even human users ("Associative" metadata).

2.3 The Expert Team has been tasked to provide a metadata framework which at the highest level is applicable to all WMO tasks, but which can be extended to specific areas and to new ventures in an acceptable and standardized form. This will not be a metadata repository of the all-inclusive form, but the basis of a description which can be extended by users to cover their own unique applications.

2.4 The group discussed the various requirements for metadata and noted that it is needed for many purposes. A brief summary of the most common applications of metadata is given below.

  1. Description and documentation of data and products

  2. Exchange of data and products

  • Between applications within an organization
  • Between organizations within a community
  • Between communities
  1. Conserving/husbanding data

  • Describe what it is and how it was collected and produced
  • Define conditions for use
  • Define retention period and validity period
  1. Location/Discovery of data and products

  • Describe what is available and how to find it
  • Allow users to find data and products of interest without their knowing where to look
  • Define conditions for use
  1. Make data/products available

  • Support Web sites
  • Allow cross-discipline use
  1. Processing data

  2. Support use of available tools

  • Off the shelf software
  • Geographic Information Systems
  • Interface with external standards
  1. Transform data and products

  • Conversion
  • Apply data and products to different purposes

3. REVIEW OF EXISTING RELEVANT METADATA STANDARDS (agenda item 3)

3.1 The experts, recognizing that there were existing metadata standards that could be applicable to WMO requirements, reviewed a number of relevant standards. They examined the Federal Geographic Data Committee (FGDC) standard, the Global Change Master Directory Directory Interchange Format (DIF), the Australia, New Zealand Spatial Information Council (ANZLIC) standard, the "Dublin Core" standard and the International Standards Organization (ISO) Draft International Standard 19115 for Geographic Metadata.

3.2 Representatives of the Synthetic Environment Data Representation and Interchange Specification (SEDRIS) made a presentation to the meeting on their organization and activities. They informed the group that SEDRIS is fundamentally about two key issues: representation of environmental data, and the interchange of these environmental data sets. To achieve the first goal, SEDRIS offers a data representation model, augmented with an environmental data coding specification and spatial reference model. Therefore, the data representation aspect of SEDRIS is about capturing and communicating meaning and semantics. For the second part, experience indicates that it is not enough to be able to clearly represent or describe the data, we must also be able to share such data with others in an efficient manner. So the second aspect of SEDRIS concerns interchange of data that can be described using the SEDRIS data representation model. For interchange, the SEDRIS Application Programming Interface (API), its format, and all the associated tools and utilities play the primary role, while being semantically coupled to the data representation model. In other words, SEDRIS provides a language for describing the environment, a mechanism to share this description and an infrastructure technology to assist implementation of this exchange mechanism.

3.3 The SEDRIS experts also provided advice based on their extensive experience in development of a comprehensive environmental data and application standard. They noted that:

  1. One should separate the context of how data is used from the implementation and representation

  2. It is more important to be systematic and consistent than perfect and complete

  3. The standard should be entirely public and controlled with no user-definable fields

3.4 The expert team found the presentation on SEDRIS to be very interesting and informative. The team felt that it was clear that a tremendous amount of work had been committed to SEDRIS and that it could provide many capabilities that might be applied to WMO requirements. At the same time the team was concerned that SEDRIS had developed and used its own API rather than relying on an external standard such as Extensible Mark-up Language (XML). While SEDRIS had created representations of their data models in XML, the extensible nature of XML seems to be in conflict with one of their dictums (expressed in 3.3 c above). The expert team recognized that there were advantages to the SEDRIS approach but felt that when applied to the issue of a WMO metadata standard the disadvantages outweighed the advantages. It agreed that XML would be the most appropriate representation form to use for WMO metadata since it could take best advantage of ongoing activities in the wider Internet community.

3.5 The United States Department of Defence (DoD) Meteorological and Oceanographic Community (METOC) have jointly collaborated on logical data standards since 1995, and the SEDRIS community have adopted their conceptual data model JMDCM (Joint METOC Conceptual Data Model). Within their data dictionary, they have included terms to represent all elements of existing WMO codes. This considerable body of work will repay further investigation, particularly as the Expert Team considers controlled vocabularies and keywords in extending the proposed WMO metadata model.

3.6 A fundamental assumption underlying all of the metadata standards that were discussed was the definition of the dataset, that is, the collection of information that a single metadata entry describes. The team recognized that different groups have different requirements and perspectives. Consequently, different groups organize their data differently and have very different definitions of what constitutes a dataset. The expert team, recognizing that a restrictive definition of a dataset could not be applied to the widely varied requirements of all WMO Programmes, agreed on a minimal and flexible working definition of a dataset as follows:

A dataset is a collection of information (data, products, etc.) that the owner considers as a unit.

Each dataset would have one and only one metadata description. However there may exist several versions of a description for several languages.

3.7 After considering the several metadata standards noted above the expert team decided to examine the Dublin Core and ISO standards most closely. The draft ISO standard 19115 is based on a number of subsidiary standards defining Geographic terms and processes starting at ISO 19000. This set of standards provides an enormous range of definitions and specifications of metadata elements, provides a schema and establishes a common set of metadata terminology, definitions, and extension procedures. In the words of the standard it is meant to:

  1. Provide data producers with appropriate information to characterise their geographic data properly.

  2. Facilitate the organization and management of metadata for geographic data.

  3. Enable users to apply geographic data in the most efficient way by knowing its basic characteristics.

  4. Facilitate data discovery, retrieval and reuse. Users will be better able to locate, access, evaluate, purchase and utilize geographic data.

  5. Enable users to determine whether geographic data in a holding will be of use to them.

3.8 The team also considered whether the elements needed to meet WMO requirements would be considered to conform to Dublin Core. While there were some ambiguities in assigning terms to Dublin Core elements (recognized by the Dublin Core Metadata Initiative), the team concluded that it was possible to do so.

3.9 The experts felt that, while comprehensive in its coverage, the draft ISO standard is very complex. They therefore concentrated their discussion on the high-level or core elements that were most important for meeting WMO requirements. The team agreed that, although some optional items were not relevant to WMO requirements, there was no fundamental shortcoming that would preclude its application for WMO purposes. However, the group felt that two mandatory items should be optional: dataset language and metadata point of contact. The dataset language should not be mandatory unless the language code allows for cases where a language is not applicable, which is the case for most WMO datasets. The metadata point of contact should be optional because there are some cases where an agreed point of contact does not exist and designation of this person could be difficult.

3.10 The ISO 19115 specifies a process (in ISO 19115 Annex C) where a community can adopt parts of the standard which it feels relevant (including the "Core Elements") and also extend the elements, keywords and code table instances to suite that community. The team noted that the WMO might need to accept more than one such "Community Profile" (ISO 19115 Annex C.5) for the different WMO Programmes, but that there should be a Community Core Profile which could be adopted by all of WMO, with the potential for further extensions under ISO 19115 Annex C where necessary. The team recommended that the following additional information should also be included as elements within the "WMO Core Metadata" profile.

  1. Dataset source or originator

  2. Keywords to identify the "Theme" or overall subject area of the dataset

  3. Unique dataset identifier

  4. Expanded or more specific date and time items

  5. Expanded description of geographic coverage

  6. Security or access rights/restrictions

  7. Processing level (observations versus analyses)

  8. Distribution mechanisms

3.11 The group, recognizing that the draft ISO metadata standard might be able to address most WMO requirements, closely examined the ISO standard to determine if it could accommodate these additional core items. It decided that all of the WMO core items could be accommodated although some WMO extensions to the ISO code lists would be required. The table below provides an overview of the ISO and WMO core elements and their corresponding names within the draft ISO standard. It should be noted that some of the items listed in the table describe individual elements while others describe "classes", which contain multiple elements.

WMO Community Core Metadata Profile (WMO Core Metadata)

Required Elements

Generic Name

ISO Field/Class Name and Reference Lines

Occurrence
1 - unique,
N - many

Definition
Title CI_Citation.title (360)

1

Name of the dataset
Identifier CI_Citation.identifier (365)
and identifierType (366)

1

Unique identifier for the dataset
Reference Date CI_Date.date (394) and dateType (395)

N

Reference date for the dataset (could be creation, publication or revision date)
Responsible party CI_ResponsibleParty (374-379)

N

Identification of and means of communication with person(s) and organizations associated with the dataset. As a minimum, role code of "originator" should be defined.
Abstract MD_Identification.Abstract (25)

1

Brief narrative summary of the contents of the dataset
Keywords MD_Keywords (53-55)

N

Keywords, their type and reference source (a defined WMO thesaurus)
Geographic Extent EX_GeographicExtent (339-349)

N

Geographic area covered by the dataset (can be bounding box, polygon or geographic identifiers (names of areas, ISO 19112)
Topic Category MD_DataIdentification.topicCategory (41)

N

Numeric code for the discipline covered by this dataset. Acceptable code values already defined within the ISO standard as code table B.5.27.
Temporal Extent Ex_TemporalExtent (350-351 and reference to ISO 19108)

N

Time Period covered by the dataset
Metadata Language MD_Medadata.language (3)

1

Language of this metadata item
Metadata character set MD_Medadata.characterSet (4)

1

Character set of this metadata item (Default of ISO 10646-1)
Metadata Point of Contact MD_Metadata.contact (8) > CI_ResponsibleParty (374-379)

N

Party responsible for this metadata item
Metadata date stamp MD_Metadata.dateStamp (9)

1

Date that this metadata item was created
Access rights or restrictions MD_Constraints (67-77)

N

Restrictions on the access and use of the resource or metadata (Could specify WMO Additional Data as free text or could expand code list B.5.24 to include this item)
Distribution mechanisms MD_Distribution (271-273)

N

Information about the methods and formats to be used in the distribution of this dataset
Processing level LI_Lineage (82-85)

N

Information about the level of processing applied to the dataset
Elements that should be defined whenever applicable
Dataset Language MD_DataIdentification.language (39)

N

Language(s) used in the dataset, if applicable.
Character Set MD_DataIdentification.characterSet (40)

1

Character set used in the dataset, if applicable
Additional Optional Elements

Spatial Resolution

MD_DataIdentification.spatialResolution (38)

N

Factor that provides a general understanding of the spatial density of the data in the dataset (e.g. grid spacing)

Spatial representation type

MD_DataIdentification.spatialRepresentationType (37)

N

Method used to spatially represent data in the dataset

Reference System

MD_ReferenceSystem (187-188)

N

Information about the reference systems used (temporal, coordinate and geographic)

Vertical Extent

Ex_VerticalExtent (354-358)

N

Vertical domain of the dataset

On-line resource

CI_OnlineResource (396-402)

N

Location (address) for on- line access using a Uniform Resource Locator address or similar addressing scheme such as http:// www. statkart. no/ isotc211
Metadata file identifier MD_Metadata.fileIdentifier (2)

1

Unique identifier for this metadata item
Metadata standard name MD_Metadata.metadataStandardName (10)

1

Name of the metadata standard (including profile name) used
Metadata standard version MD_Metadata.metadataStandardVersion (11)

1

Version (profile) of the metadata standard used

Validity Time

MD_Usage (62-66)

N

Specific date or time ranges that the dataset is valid

Notes:

1. The proposed WMO standard would provide a definition for directory searches and exchange and would not specify how the information should be archived or presented to users.

2. The core elements listed above define a minimum set of information required to exchange data for WMO purposes and are not exhaustive. To fully meet the requirements of all WMO Programmes for metadata, application of far more comprehensive standards would be required. Although the draft ISO standard might include all of the elements essential for WMO this can not be determined unless a detailed analysis of WMO metadata requirements is undertaken (see section 4 below).

4. RECOMMENDATIONS ON COMPREHENSIVE WMO METADATA STANDARDS (agenda item 4)

4.1 The various Programmes of WMO have a wide range of requirements for documentation of their datasets. Thus, it would be difficult to develop a standard for comprehensive metadata that would meet the needs of all Programmes and development of a comprehensive WMO metadata standard would be a difficult, lengthy and expensive undertaking. One possible approach would be for each WMO Programme to conduct analysis of its own metadata requirements and then have an inter-programme team develop a comparative analysis. However, the team felt that a careful and well thought out examination of the potential benefits should be carried out before committing substantial resources to such an endeavour. The team decided to add further investigation of this issue to its near-term work programme (see section 6 below).

5. XML STRUCTURES FOR REPRESENTING METADATA (agenda item 5)

5.1 XML is rapidly becoming a standard for exchanging information between applications, as well as for providing information on which the formatting of data for display in a browser may be defined. Industry standards (e.g. Simple Object Access Protocol (SOAP)) are being defined to allow the exchange of information between applications using the XML standard, with the expectation that many business transactions will use XML as their standard means of data exchange.

5.2 There are many possible ways of representing WMO metadata in XML. To ensure interoperability the experts agreed that a framework for mapping the proposed metadata standard into XML should be developed and added this item to its near-term work programme.

5.3 Annex D of the draft ISO 19115 standard provides an "Abstract Test Suite", which should be used to determine if the proposed WMO Core Metadata standard conforms to the ISO standard. The team has adopted an item in its short-term work programme to check this conformity.

6. FURTHER DEVELOPMENT OF METADATA STANDARDS (agenda item 6)

In light of the experience gained during its deliberations, the team agreed to carry out the following tasks over the next 6 months.

Task

Responsible

Deadline

Review proposed standard with respect to existing datasets

All team members

1 February 2002

Develop draft list of keywords to describe WMO datasets and circulate for review

Secretariat

1 February 2002

Develop list of proposed extensions needed for ISO code lists (e.g. list of standard parameter codes)

All team members

1 February 2002

Develop recommendation on required modifications to the Guide on WWW Data Management

G. Ross

1 February 2002

Propose XML schema, representation and examples

G. Ross

1 March 2002

Investigate software to be used in implementation

All team members

1 April 2002

Develop recommendation on pursuit of a comprehensive WMO metadata standard

S. Foreman

1 May 2002

Evaluate WMO Core Metadata conformity to ISO 19115

S. Foreman

1 June 2002

 

7. CLOSURE OF THE MEETING

The meeting closed on Thursday 8 November 2001.


LIST OF PARTICIPANTS

Steve Foreman, Chair Met Office
London Road
Bracknell, Berkshire RG12 2SZ
United Kingdom

Tel: (+44 1344) 854680
Fax: (+44 1344) 856099
E-mail: sjforeman@meto.gov.uk
Alexander Besprozvannykh RIHMI-WDC
6, Korolev St.
Obninsk, Kaluga reg, 249020
Russian Federation

Tel: (08439) 74959
Fax: (095) 255 22 25
E-Mail: alexander@meteo.ru
Wang Guofu China Meteorological Administration
46 Baishiqiaolu Road
Western Suburb
Beijing 100081
China

Tel: (+86 10) 684 0874 85
Fax: (+86 10) 621 759 30
E-mail: Wanggf@cma.gov.cn
Henry Hayhoe
representing CAgM
Agriculture and Agri-Food Canada
Research Branch
Eastern Cereal and Oilseed Research Centre
Ottawa, Ontario K1A 0C6
Canada

Tel: (+1 613) 7591524
Fax: (+1 613) 7591924
Email: hayhoeh@em.agr.ca
Gil Ross Met Office
London Road
Bracknell, Berkshire
RG12 2SZ
United Kingdom

Tel: (+44)(0) 1344 856 973
Fax: (+44)(0) 1344 856 119
E-mail: gil.ross@metoffice.com
Jürgen Seib Deutscher Wetterdienst
Kaiserleistr. 42
63067 Offenbach
Germany

Tel: (+49 69) 80622243
Fax: (+49 69) 80622801
E-mail: juergen.seib@dwd.de
WMO Secretariat World Meteorological Organization
7 bis Avenue de la Paix
Case postale No. 2300
CH-1211 Geneva 2
Switzerland
David McGuirk WWW-B Dept.
Tel: (+41 22) 730 8241
Fax: (+41 22) 730 8021
E-mail: mcguirk_d@gateway.wmo.ch
Valerie Gerard WCP Dept.
Tel: (+41 22) 730 8278
Fax: (+41 22) 730 8042
E-mail: gerard_v@gateway.wmo.ch
Hama Kontongomde WCP Dept.
Tel: (+41 22) 730 8251
Fax: (+41 22) 730 8042
E-mail: Kontonpomde_h@gateway.wmo.ch
Stephen Lellyett WCP Dept.
Tel: (+41 22) 730 8151
Fax: (+41 22) 730 8042
E-mail: lellyett_s@gateway.wmo.ch
Additional experts
Jack Cogman
(ISO SC24/WG8 Convenor)
Thales Training and Simulation
Gatwick Road
Crawley, West Sussex, RH109RL
United Kingdom

Tel: (+44) (0) 1293 563 343
Fax: (+44) (0) 1293 563 800
E-mail: jack.cogman@thales-tts.com
Louis A. Hembree Naval Research Laboratory
Marine Meteorology Division
Monterey, CA 93943-5502
USA

Tel: (+1 831)-656-4787
Fax: (+1 831)-656-4769
E-mail: hembree@nrlmry.navy.mil
Farid Mamaghani SEDRIS
19223 SE 45th Ct.
Issaquah, WA 98027
USA

Tel: (+1 425) 641-6192
E-mail: farid@sedris.org
Parsh Mehta Accuweather
385 Science Park Road
State College, PA 16803-2215
USA

Tel: (+1 215) 868-0248
Fax: (+1 814) 235 8699
E-mail:pmehta@accuwx.com

 


Glossary

ANZLIC

Australia, New Zealand Spatial Information Council

CAgM

Commission for Agricultural Meteorology

CBS

Commission for Basic Systems

DIF

Directory Interchange Format

Dublin Core

Metadata standard developed by the library community

ET

Expert team

FGDC

Federal Geographic Data Committee (USA)

HTML

Hypertext markup language

ISO

International Standards Organization

METOC

US Department of Defense Meteorological and Oceanographic Community

NMHS

National meteorological and hydrological services

NMS

National meteorological service

SEDRIS

Synthetic Environment Data Representation and Interchange Specification

SOAP

Simple Object Access Protocol

WMO

World Meteorological Organization

WWW

World Weather Watch

XML

Extensible mark-up language

 


References

  1. Geographic Information Draft European Prestandards (http://forum.afnor.fr/afnor/WORK/AFNOR/GPN2/Z13C/index.htm)

  2. The Spatial Data Infrastructure Cookbook (http://www.gsdi.org/pubs/cookbook/cookbook0515.pdf)

  3. Problems in Ocean Data Management: Seeking Solutions (http://www.bom.gov.au/OOPC/NVODS_WS/)

  4. Weather Observation Definition Format (OMF) (http://zowie.metnet.navy.mil/~spawar/JMV-TNG/XML/OMF.html)

  5. Dublin Core Metadata Initiative: (http://dublincore.org/about/ and http://www.ukoln.ac.uk/metadata/dcdot/)

  6. US Department of Defense Meteorology and Oceanography Data Administration (https://www.cnmoc.navy.mil/da/jmcdm.htm)

  7. ISO 639, Code for the representation of names of languages

  8. ISO 3166, Codes for the representation of names of countries and their subdivisions

  9. ISO 8879, Information processing Text and office systems Standard Generalized Markup Language (SGML)

  10. ISO/IEC 10646-1, Information technology Universal Multiple-Octet Coded Character Set (UCS) Part 1: Architecture and Basic Multilingual Plane

  11. ISO/IEC 11179 (all parts), Information technology ¯ Specification and standardization of data elements

  12. ISO/TS 19103:1) , Geographic information Conceptual schema language

  13. ISO 19106:1), Geographic information Profiles

  14. ISO 19107:1), Geographic information Spatial schema

  15. ISO 19108:1), Geographic information Temporal schema

  16. ISO 19109:1), Geographic information Rules for application schema

  17. ISO 19111:1), Geographic information Spatial referencing by coordinates

  18. ISO 19112:1), Geographic information Spatial referencing by geographic identifiers

  19. ISO 19115:1), Geographic information Metadata

1. To be published.