WORLD METEOROLOGICAL ORGANIZATION
COMMISSION FOR BASIC SYSTEMS
MEETING OF THE
EXPERT TEAM ON INTEGRATED DATA MANAGEMENT
FINAL REPORT
GENEVA, 5 - 8 NOVEMBER 2001
DISCLAIMER
Regulation 42
Recommendations of working groups shall have no status within the Organization until they have been approved by the responsible constituent body. In the case of joint working groups the recommendations must be concurred with by the presidents of the constituent bodies concerned before being submitted to the designated constituent body.
Regulation 43
In the case of a recommendation made by a working group between sessions of the responsible constituent body, either in a session of a working group or by correspondence, the president of the body may, as an exceptional measure, approve the recommendation on behalf of the constituent body when the matter is, in his opinion, urgent, and does not appear to imply new obligations for Members. He may then submit this recommendation for adoption by the Executive Council or to the President of the Organization for action in accordance with Regulation 9(5).
CONTENTS
Pages |
|
ii |
|
iii |
|
1 |
|
7 |
|
11 |
|
12 |
1. ORGANIZATION OF THE MEETING
1.1 Opening remarks
1.2 Adoption of the agenda
1.3 Working arrangements2. REVIEW OF WMO REQUIREMENTS FOR A METADATA STANDARD
3. REVIEW OF EXISTING RELEVANT METADATA STANDARDS
4. RECOMMENDATIONS ON COMPREHENSIVE WMO METADATA STANDARDS
5. XML STRUCTURES FOR REPRESENTING METADATA
7. CLOSURE OF THE MEETING
Executive Summary
A meeting of the CBS Expert Team on Integrated Data Management was held 5 to 8 November 2001 in Geneva. The experts, understanding that metadata means different things to different people, the team worked to develop a metadata framework, which at the highest level is applicable to all WMO tasks, but which can be extended to specific areas and to new ventures in an acceptable and standardized form.
The experts, recognizing that there were existing metadata standards that could be applicable to WMO requirements, reviewed a number of relevant standards. After their review they examined the Dublin Core and draft ISO standard Geographic Metadata (19115) most closely. They determined that the elements needed to meet WMO requirements could be considered to conform to Dublin Core and that the draft ISO standard could be applied to WMO requirements.
The ISO 19115 specifies a process (in ISO 19115 Annex C) where a community can adopt parts of the standard that it feels are relevant (including the "Core Elements") and also extend the elements, keywords and code table instances to suite that community. The team noted that the WMO might need to accept more than one such "Community Profile" (ISO 19115 Annex C.5) for the different WMO Programmes, but that there should be a Community Core Profile which could be adopted by all of WMO, with the potential for further extensions under ISO 19115 Annex C where necessary. With this process in mind the team developed a "WMO Community Core Metadata" profile. An overview of the ISO and WMO core elements and their corresponding names within the draft ISO standard is provided in the report of the meeting.
The team agreed that the various Programmes of WMO have a wide range of requirements for documenting their datasets. Thus, it would be difficult to develop a standard for comprehensive metadata that would meet the needs of all Programmes and development of a comprehensive WMO metadata standard would be a difficult, lengthy and expensive undertaking. The team will investigate this issue further over the next several months.
The team will also review the proposed core metadata standard with respect to existing datasets, develop a draft list of keywords to describe WMO datasets, develop list of proposed extensions needed for ISO code lists, and propose an XML schema and examples.
1. ORGANIZATION OF THE MEETING (agenda item 1)
1.1 Opening remarks
1.1.1 A meeting of the CBS Expert Team on Integrated Data Management opened at 0900 on Monday, 5 November 2001 in the headquarters of the WMO in Geneva. Mr S. Foreman (UK) chaired the meeting. Mr D. Schiessl, Director of the WWW Basic Systems Department, opened the meeting on behalf of the Secretary-General and welcomed the experts to Geneva. He outlined the objectives of the meeting and noted the primary purpose was to develop a metadata standard to be used in future WMO information systems as well as a standard Internet technology representation of the metadata for use on the World Wide Web for consideration by CBS-Ext.(2002).
1.2 Adoption of the agenda
1.2.1 The experts adopted the agenda as reproduced at the beginning of this report.
2. REVIEW OF WMO REQUIREMENTS FOR A METADATA STANDARD (agenda item 2)
2.1 The experts, understanding that metadata means different things to different people first considered a working definition of metadata for its deliberations. It agreed that at the most basic level metadata is data about data. More specifically, it is the descriptive data necessary to allow us to find, process and use data, information and products. While metadata generally can describe products, services and software as well as data at different stages of manipulation, it can also be a specification. Metadata can be extensive and all-inclusive, or it can be specific to a more limited function.
2.2 WMO Programmes and Members currently maintain a tremendous volume and variety of metadata. However little of this is in an electronic, standardized form which can be used to find data ("Navigational" metadata), and it is seldom distributed with the data or in a manner to allow the data to be recognized and understood by computer applications or even human users ("Associative" metadata).
2.3 The Expert Team has been tasked to provide a metadata framework which at the highest level is applicable to all WMO tasks, but which can be extended to specific areas and to new ventures in an acceptable and standardized form. This will not be a metadata repository of the all-inclusive form, but the basis of a description which can be extended by users to cover their own unique applications.
2.4 The group discussed the various requirements for metadata and noted that it is needed for many purposes. A brief summary of the most common applications of metadata is given below.
Description and documentation of data and products
Exchange of data and products
- Between applications within an organization
- Between organizations within a community
- Between communities
Conserving/husbanding data
- Describe what it is and how it was collected and produced
- Define conditions for use
- Define retention period and validity period
Location/Discovery of data and products
- Describe what is available and how to find it
- Allow users to find data and products of interest without their knowing where to look
- Define conditions for use
Make data/products available
- Support Web sites
- Allow cross-discipline use
Processing data
Support use of available tools
- Off the shelf software
- Geographic Information Systems
- Interface with external standards
Transform data and products
- Conversion
- Apply data and products to different purposes
3. REVIEW OF EXISTING RELEVANT METADATA STANDARDS (agenda item 3)
3.1 The experts, recognizing that there were existing metadata standards that could be applicable to WMO requirements, reviewed a number of relevant standards. They examined the Federal Geographic Data Committee (FGDC) standard, the Global Change Master Directory Directory Interchange Format (DIF), the Australia, New Zealand Spatial Information Council (ANZLIC) standard, the "Dublin Core" standard and the International Standards Organization (ISO) Draft International Standard 19115 for Geographic Metadata.
3.2 Representatives of the Synthetic Environment Data Representation and Interchange Specification (SEDRIS) made a presentation to the meeting on their organization and activities. They informed the group that SEDRIS is fundamentally about two key issues: representation of environmental data, and the interchange of these environmental data sets. To achieve the first goal, SEDRIS offers a data representation model, augmented with an environmental data coding specification and spatial reference model. Therefore, the data representation aspect of SEDRIS is about capturing and communicating meaning and semantics. For the second part, experience indicates that it is not enough to be able to clearly represent or describe the data, we must also be able to share such data with others in an efficient manner. So the second aspect of SEDRIS concerns interchange of data that can be described using the SEDRIS data representation model. For interchange, the SEDRIS Application Programming Interface (API), its format, and all the associated tools and utilities play the primary role, while being semantically coupled to the data representation model. In other words, SEDRIS provides a language for describing the environment, a mechanism to share this description and an infrastructure technology to assist implementation of this exchange mechanism.
3.3 The SEDRIS experts also provided advice based on their extensive experience in development of a comprehensive environmental data and application standard. They noted that:
One should separate the context of how data is used from the implementation and representation
It is more important to be systematic and consistent than perfect and complete
The standard should be entirely public and controlled with no user-definable fields
3.4 The expert team found the presentation on SEDRIS to be very interesting and informative. The team felt that it was clear that a tremendous amount of work had been committed to SEDRIS and that it could provide many capabilities that might be applied to WMO requirements. At the same time the team was concerned that SEDRIS had developed and used its own API rather than relying on an external standard such as Extensible Mark-up Language (XML). While SEDRIS had created representations of their data models in XML, the extensible nature of XML seems to be in conflict with one of their dictums (expressed in 3.3 c above). The expert team recognized that there were advantages to the SEDRIS approach but felt that when applied to the issue of a WMO metadata standard the disadvantages outweighed the advantages. It agreed that XML would be the most appropriate representation form to use for WMO metadata since it could take best advantage of ongoing activities in the wider Internet community.
3.5 The United States Department of Defence (DoD) Meteorological and Oceanographic Community (METOC) have jointly collaborated on logical data standards since 1995, and the SEDRIS community have adopted their conceptual data model JMDCM (Joint METOC Conceptual Data Model). Within their data dictionary, they have included terms to represent all elements of existing WMO codes. This considerable body of work will repay further investigation, particularly as the Expert Team considers controlled vocabularies and keywords in extending the proposed WMO metadata model.
3.6 A fundamental assumption underlying all of the metadata standards that were discussed was the definition of the dataset, that is, the collection of information that a single metadata entry describes. The team recognized that different groups have different requirements and perspectives. Consequently, different groups organize their data differently and have very different definitions of what constitutes a dataset. The expert team, recognizing that a restrictive definition of a dataset could not be applied to the widely varied requirements of all WMO Programmes, agreed on a minimal and flexible working definition of a dataset as follows:
A dataset is a collection of information (data, products, etc.) that the owner considers as a unit.
Each dataset would have one and only one metadata description. However there may exist several versions of a description for several languages.
3.7 After considering the several metadata standards noted above the expert team decided to examine the Dublin Core and ISO standards most closely. The draft ISO standard 19115 is based on a number of subsidiary standards defining Geographic terms and processes starting at ISO 19000. This set of standards provides an enormous range of definitions and specifications of metadata elements, provides a schema and establishes a common set of metadata terminology, definitions, and extension procedures. In the words of the standard it is meant to:
Provide data producers with appropriate information to characterise their geographic data properly.
Facilitate the organization and management of metadata for geographic data.
Enable users to apply geographic data in the most efficient way by knowing its basic characteristics.
Facilitate data discovery, retrieval and reuse. Users will be better able to locate, access, evaluate, purchase and utilize geographic data.
Enable users to determine whether geographic data in a holding will be of use to them.
3.8 The team also considered whether the elements needed to meet WMO requirements would be considered to conform to Dublin Core. While there were some ambiguities in assigning terms to Dublin Core elements (recognized by the Dublin Core Metadata Initiative), the team concluded that it was possible to do so.
3.9 The experts felt that, while comprehensive in its coverage, the draft ISO standard is very complex. They therefore concentrated their discussion on the high-level or core elements that were most important for meeting WMO requirements. The team agreed that, although some optional items were not relevant to WMO requirements, there was no fundamental shortcoming that would preclude its application for WMO purposes. However, the group felt that two mandatory items should be optional: dataset language and metadata point of contact. The dataset language should not be mandatory unless the language code allows for cases where a language is not applicable, which is the case for most WMO datasets. The metadata point of contact should be optional because there are some cases where an agreed point of contact does not exist and designation of this person could be difficult.
3.10 The ISO 19115 specifies a process (in ISO 19115 Annex C) where a community can adopt parts of the standard which it feels relevant (including the "Core Elements") and also extend the elements, keywords and code table instances to suite that community. The team noted that the WMO might need to accept more than one such "Community Profile" (ISO 19115 Annex C.5) for the different WMO Programmes, but that there should be a Community Core Profile which could be adopted by all of WMO, with the potential for further extensions under ISO 19115 Annex C where necessary. The team recommended that the following additional information should also be included as elements within the "WMO Core Metadata" profile.
Dataset source or originator
Keywords to identify the "Theme" or overall subject area of the dataset
Unique dataset identifier
Expanded or more specific date and time items
Expanded description of geographic coverage
Security or access rights/restrictions
Processing level (observations versus analyses)
Distribution mechanisms
3.11 The group, recognizing that the draft ISO metadata standard might be able to address most WMO requirements, closely examined the ISO standard to determine if it could accommodate these additional core items. It decided that all of the WMO core items could be accommodated although some WMO extensions to the ISO code lists would be required. The table below provides an overview of the ISO and WMO core elements and their corresponding names within the draft ISO standard. It should be noted that some of the items listed in the table describe individual elements while others describe "classes", which contain multiple elements.
WMO Community Core Metadata Profile (WMO Core Metadata)
Required Elements | |||
Generic Name |
ISO Field/Class Name and Reference Lines |
Occurrence |
Definition |
Title | CI_Citation.title (360) | 1 |
Name of the dataset |
Identifier | CI_Citation.identifier (365) and identifierType (366) |
1 |
Unique identifier for the dataset |
Reference Date | CI_Date.date (394) and dateType (395) | N |
Reference date for the dataset (could be creation, publication or revision date) |
Responsible party | CI_ResponsibleParty (374-379) | N |
Identification of and means of communication with person(s) and organizations associated with the dataset. As a minimum, role code of "originator" should be defined. |
Abstract | MD_Identification.Abstract (25) | 1 |
Brief narrative summary of the contents of the dataset |
Keywords | MD_Keywords (53-55) | N |
Keywords, their type and reference source (a defined WMO thesaurus) |
Geographic Extent | EX_GeographicExtent (339-349) | N |
Geographic area covered by the dataset (can be bounding box, polygon or geographic identifiers (names of areas, ISO 19112) |
Topic Category | MD_DataIdentification.topicCategory (41) | N |
Numeric code for the discipline covered by this dataset. Acceptable code values already defined within the ISO standard as code table B.5.27. |
Temporal Extent | Ex_TemporalExtent (350-351 and reference to ISO 19108) | N |
Time Period covered by the dataset |
Metadata Language | MD_Medadata.language (3) | 1 |
Language of this metadata item |
Metadata character set | MD_Medadata.characterSet (4) | 1 |
Character set of this metadata item (Default of ISO 10646-1) |
Metadata Point of Contact | MD_Metadata.contact (8) > CI_ResponsibleParty (374-379) | N |
Party responsible for this metadata item |
Metadata date stamp | MD_Metadata.dateStamp (9) | 1 |
Date that this metadata item was created |
Access rights or restrictions | MD_Constraints (67-77) | N |
Restrictions on the access and use of the resource or metadata (Could specify WMO Additional Data as free text or could expand code list B.5.24 to include this item) |
Distribution mechanisms | MD_Distribution (271-273) | N |
Information about the methods and formats to be used in the distribution of this dataset |
Processing level | LI_Lineage (82-85) | N |
Information about the level of processing applied to the dataset |
Elements that should be defined whenever applicable | |||
Dataset Language | MD_DataIdentification.language (39) | N |
Language(s) used in the dataset, if applicable. |
Character Set | MD_DataIdentification.characterSet (40) | 1 |
Character set used in the dataset, if applicable |
Additional Optional Elements | |||
Spatial Resolution |
MD_DataIdentification.spatialResolution (38) | N |
Factor that provides a general understanding of the spatial density of the data in the dataset (e.g. grid spacing) |
Spatial representation type |
MD_DataIdentification.spatialRepresentationType (37) | N |
Method used to spatially represent data in the dataset |
Reference System |
MD_ReferenceSystem (187-188) | N |
Information about the reference systems used (temporal, coordinate and geographic) |
Vertical Extent |
Ex_VerticalExtent (354-358) | N |
Vertical domain of the dataset |
On-line resource |
CI_OnlineResource (396-402) | N |
Location (address) for on- line access using a Uniform Resource Locator address or similar addressing scheme such as http:// www. statkart. no/ isotc211 |
Metadata file identifier | MD_Metadata.fileIdentifier (2) | 1 |
Unique identifier for this metadata item |
Metadata standard name | MD_Metadata.metadataStandardName (10) | 1 |
Name of the metadata standard (including profile name) used |
Metadata standard version | MD_Metadata.metadataStandardVersion (11) | 1 |
Version (profile) of the metadata standard used |
Validity Time |
MD_Usage (62-66) | N |
Specific date or time ranges that the dataset is valid |
Notes:
1. The proposed WMO standard would provide a definition for directory searches and exchange and would not specify how the information should be archived or presented to users.2. The core elements listed above define a minimum set of information required to exchange data for WMO purposes and are not exhaustive. To fully meet the requirements of all WMO Programmes for metadata, application of far more comprehensive standards would be required. Although the draft ISO standard might include all of the elements essential for WMO this can not be determined unless a detailed analysis of WMO metadata requirements is undertaken (see section 4 below).
4. RECOMMENDATIONS ON COMPREHENSIVE WMO METADATA STANDARDS (agenda item 4)
4.1 The various Programmes of WMO have a wide range of requirements for documentation of their datasets. Thus, it would be difficult to develop a standard for comprehensive metadata that would meet the needs of all Programmes and development of a comprehensive WMO metadata standard would be a difficult, lengthy and expensive undertaking. One possible approach would be for each WMO Programme to conduct analysis of its own metadata requirements and then have an inter-programme team develop a comparative analysis. However, the team felt that a careful and well thought out examination of the potential benefits should be carried out before committing substantial resources to such an endeavour. The team decided to add further investigation of this issue to its near-term work programme (see section 6 below).
5. XML STRUCTURES FOR REPRESENTING METADATA (agenda item 5)
5.1 XML is rapidly becoming a standard for exchanging information between applications, as well as for providing information on which the formatting of data for display in a browser may be defined. Industry standards (e.g. Simple Object Access Protocol (SOAP)) are being defined to allow the exchange of information between applications using the XML standard, with the expectation that many business transactions will use XML as their standard means of data exchange.
5.2 There are many possible ways of representing WMO metadata in XML. To ensure interoperability the experts agreed that a framework for mapping the proposed metadata standard into XML should be developed and added this item to its near-term work programme.
5.3 Annex D of the draft ISO 19115 standard provides an "Abstract Test Suite", which should be used to determine if the proposed WMO Core Metadata standard conforms to the ISO standard. The team has adopted an item in its short-term work programme to check this conformity.
6. FURTHER DEVELOPMENT OF METADATA STANDARDS (agenda item 6)
In light of the experience gained during its deliberations, the team agreed to carry out the following tasks over the next 6 months.
Task |
Responsible |
Deadline |
Review proposed standard with respect to existing datasets |
All team members |
1 February 2002 |
Develop draft list of keywords to describe WMO datasets and circulate for review |
Secretariat |
1 February 2002 |
Develop list of proposed extensions needed for ISO code lists (e.g. list of standard parameter codes) |
All team members |
1 February 2002 |
Develop recommendation on required modifications to the Guide on WWW Data Management |
G. Ross |
1 February 2002 |
Propose XML schema, representation and examples |
G. Ross |
1 March 2002 |
Investigate software to be used in implementation |
All team members |
1 April 2002 |
Develop recommendation on pursuit of a comprehensive WMO metadata standard |
S. Foreman |
1 May 2002 |
Evaluate WMO Core Metadata conformity to ISO 19115 |
S. Foreman |
1 June 2002 |
7. CLOSURE OF THE MEETING
The meeting closed on Thursday 8 November 2001.
LIST OF PARTICIPANTS
Steve Foreman, Chair | Met Office London Road Bracknell, Berkshire RG12 2SZ United Kingdom Tel: (+44 1344) 854680 Fax: (+44 1344) 856099 E-mail: sjforeman@meto.gov.uk |
Alexander Besprozvannykh | RIHMI-WDC 6, Korolev St. Obninsk, Kaluga reg, 249020 Russian Federation Tel: (08439) 74959 Fax: (095) 255 22 25 E-Mail: alexander@meteo.ru |
Wang Guofu | China Meteorological Administration 46 Baishiqiaolu Road Western Suburb Beijing 100081 China Tel: (+86 10) 684 0874 85 Fax: (+86 10) 621 759 30 E-mail: Wanggf@cma.gov.cn |
Henry Hayhoe representing CAgM |
Agriculture and Agri-Food Canada Research Branch Eastern Cereal and Oilseed Research Centre Ottawa, Ontario K1A 0C6 Canada Tel: (+1 613) 7591524 Fax: (+1 613) 7591924 Email: hayhoeh@em.agr.ca |
Gil Ross | Met Office London Road Bracknell, Berkshire RG12 2SZ United Kingdom Tel: (+44)(0) 1344 856 973 Fax: (+44)(0) 1344 856 119 E-mail: gil.ross@metoffice.com |
Jürgen Seib | Deutscher Wetterdienst Kaiserleistr. 42 63067 Offenbach Germany Tel: (+49 69) 80622243 Fax: (+49 69) 80622801 E-mail: juergen.seib@dwd.de |
WMO Secretariat | World Meteorological Organization 7 bis Avenue de la Paix Case postale No. 2300 CH-1211 Geneva 2 Switzerland |
David McGuirk | WWW-B Dept. Tel: (+41 22) 730 8241 Fax: (+41 22) 730 8021 E-mail: mcguirk_d@gateway.wmo.ch |
Valerie Gerard | WCP Dept. Tel: (+41 22) 730 8278 Fax: (+41 22) 730 8042 E-mail: gerard_v@gateway.wmo.ch |
Hama Kontongomde | WCP Dept. Tel: (+41 22) 730 8251 Fax: (+41 22) 730 8042 E-mail: Kontonpomde_h@gateway.wmo.ch |
Stephen Lellyett | WCP Dept. Tel: (+41 22) 730 8151 Fax: (+41 22) 730 8042 E-mail: lellyett_s@gateway.wmo.ch |
Additional experts | |
Jack Cogman (ISO SC24/WG8 Convenor) |
Thales Training and Simulation Gatwick Road Crawley, West Sussex, RH109RL United Kingdom Tel: (+44) (0) 1293 563 343 Fax: (+44) (0) 1293 563 800 E-mail: jack.cogman@thales-tts.com |
Louis A. Hembree | Naval Research Laboratory Marine Meteorology Division Monterey, CA 93943-5502 USA Tel: (+1 831)-656-4787 Fax: (+1 831)-656-4769 E-mail: hembree@nrlmry.navy.mil |
Farid Mamaghani | SEDRIS 19223 SE 45th Ct. Issaquah, WA 98027 USA Tel: (+1 425) 641-6192 E-mail: farid@sedris.org |
Parsh Mehta | Accuweather 385 Science Park Road State College, PA 16803-2215 USA Tel: (+1 215) 868-0248 Fax: (+1 814) 235 8699 E-mail:pmehta@accuwx.com |
ANZLIC |
Australia, New Zealand Spatial Information Council |
CAgM |
Commission for Agricultural Meteorology |
CBS |
Commission for Basic Systems |
DIF |
Directory Interchange Format |
Dublin Core |
Metadata standard developed by the library community |
ET |
Expert team |
FGDC |
Federal Geographic Data Committee (USA) |
HTML |
Hypertext markup language |
ISO |
International Standards Organization |
METOC |
US Department of Defense Meteorological and Oceanographic Community |
NMHS |
National meteorological and hydrological services |
NMS |
National meteorological service |
SEDRIS |
Synthetic Environment Data Representation and Interchange Specification |
SOAP |
Simple Object Access Protocol |
WMO |
World Meteorological Organization |
WWW |
World Weather Watch |
XML |
Extensible mark-up language |
Geographic Information Draft European Prestandards (http://forum.afnor.fr/afnor/WORK/AFNOR/GPN2/Z13C/index.htm)
The Spatial Data Infrastructure Cookbook (http://www.gsdi.org/pubs/cookbook/cookbook0515.pdf)
Problems in Ocean Data Management: Seeking Solutions (http://www.bom.gov.au/OOPC/NVODS_WS/)
Weather Observation Definition Format (OMF) (http://zowie.metnet.navy.mil/~spawar/JMV-TNG/XML/OMF.html)
Dublin Core Metadata Initiative: (http://dublincore.org/about/ and http://www.ukoln.ac.uk/metadata/dcdot/)
US Department of Defense Meteorology and Oceanography Data Administration (https://www.cnmoc.navy.mil/da/jmcdm.htm)
ISO 639, Code for the representation of names of languages
ISO 3166, Codes for the representation of names of countries and their subdivisions
ISO 8879, Information processing Text and office systems Standard Generalized Markup Language (SGML)
ISO/IEC 10646-1, Information technology Universal Multiple-Octet Coded Character Set (UCS) Part 1: Architecture and Basic Multilingual Plane
ISO/IEC 11179 (all parts), Information technology ¯ Specification and standardization of data elements
ISO/TS 19103:1) , Geographic information Conceptual schema language
ISO 19106:1), Geographic information Profiles
ISO 19107:1), Geographic information Spatial schema
ISO 19108:1), Geographic information Temporal schema
ISO 19109:1), Geographic information Rules for application schema
ISO 19111:1), Geographic information Spatial referencing by coordinates
ISO 19112:1), Geographic information Spatial referencing by geographic identifiers
ISO 19115:1), Geographic information Metadata
1. To be published.