CDMS Requirements Analysis

CAUTION

The content presented on this page is a work in progress.
It may change at any stage until finalised.


NB: ET-CDMS Team Members

If you want to edit the table below, please contact Bruce first.
If not, direct edits to the table below will be lost at the next update.


TODO

- add Radim and Rashid's initial analysis
- tie the CDMS to essential service needs to support a range of industries, eg drought monitoring, seasonal forecasting, monitoring of extremes, enquiries from Ministers - appropriate user stories required
- detailed analysis of all components
- explore specific climate data management user stories in depth
- rationalise user stories to those that best illustrate requirements
- high level architecture for each key user story, showing use of components


General notes:

1. Use mind maps for user stories (Tools : open source Freemind version 1.01 recommended), Download available here.

2. If required, the current version of these docs may be found here:

• WMO #1131 Climate Data Management System Specifications
• WIGOS Observations Metadata Standard

3. During the analysis work, we expect that we will develop many user stories with more than one available per component. In time these User Stories will be rationalised to a set that best reflects the expected requirements of a Modern CDMS.


Status Classification used:

• Preliminary Analysis
• Detailed Analysis
• Complete
• Approved

--

CDMS Component	Status	Stage (%)	Mind Map Analysis	User Story	Conceptual Diagram	Date last revised	Comments
3. CDMS Governance	Preliminary analysis	100	.mm .pdf			2016-08
4. Time-series climate data	Detailed Analysis	50	.mm .pdf			2016-11
4.1 Observations data	Detailed Analysis	50		User story: 4.1 Observations data	An Observation	2016-11
4.2 Logical data models	Preliminary analysis	100	.mm .pdf	User story: Logical Data Model - Neutral CDMS Archive		2017-01	The Logical Data Model - Neutral CDMS Archive data format will probably need considerable engagement with CBS - IPET-DD, once the data model for a CDMS Database is known The 5.2 analysis and user stories needs considerable work. We need to cater for at least the 'Community agreed standard vocabularies' as illustrated in the _Climate data services conceptual diagram_ below. This functionality is required to underpin federated global climate data services of consistent data sematics
4.3 Climate metadata	Preliminary analysis	100	.mm .pdf	User story: 4.3.3 Climate Data Provenance - W3C		2017-01	In the next version of the CDMS Specifications: - the Climate Observations Metadata components will be adjusted to be consistent with the WMO WIGOS (Observations) Metadata standard - Data Intellectual Property Metadata may be also included as a subcomponent of Climate metadata. See also info in new component section in the table below.
4.3.1 Observations metadata	Preliminary analysis	100
4.3.2 Dataset discovery metadata	Preliminary analysis	100
4.3.3 Data provenance	Preliminary analysis	100
4.4 WMO standard products	Preliminary analysis	100	.mm .pdf
4.5 Derived climate data	Detailed Analysis	30	.mm .pdf				Need to include from Nov 2016 mind map analysis: - derived statistical data (Extremes, Normals, Anomalies, averages, homogenised date etc.) and other types of derived data - Spatial Views from derived data as discrete point coverages (perhaps by variable and observations network) - climate monitoring grids as continuous coverages derived from the spatial views
4.6 Ancillary data	Preliminary analysis	100	.mm .pdf
5. Climate data management	Preliminary analysis	100				2016-08
5.1 Ingest and extract	Detailed Analysis	95	.mm .pdf	User story: 5.1 Ingest and Extract	Ingest and Extract	2016-11	Need to update User Story to match mind map analysis
5.2 Data rescue	Preliminary analysis	100	.mm .pdf	User story: 5.2 Climate Data Rescue - ET-CDMS User story: 5.2 Climate Data Rescue - W3C			Need to approach ET-DARE for help developing appropriate user stories
5.3 Observations quality control	Preliminary analysis	100	.mm .pdf	User story: 5.3 Observations Quality Control - ET-CDMS	Observations QC/QA	2016-11	Work is required to assess to suitability of using the ISO Standard 19157, Geographic Information - Data Quality: as a data quality framework for Climate Data
5.4 Quality assessment	Preliminary analysis	100	.mm .pdf
5.5 Climate metadata	Preliminary analysis	100	.mm .pdf	User story: 4.3.3 Climate Data Provenance - W3C
6. Climate data analysis	Detailed Analysis	10	.mm .pdf	User story: 6.1 Analyse and Generate Climate Data - Generic		2016-11
6.1.1 Climate modelling	Preliminary analysis	100
6.1.2 Generate derived data from climate observations	Preliminary analysis	100		User story: 6.1.2 Generate climate monitoring grids - ET-CDMS-1 User story: 6.1.2 Generate climate monitoring grids - derived from remotely sensed imagery - ET-CDMS-1 User story: 6.1.2 Virtual Observations - W3C User story: 6.1.2 Generate Standard WMO Data Product User story: 6.1.2 Generate Observations Statistical Data			Refer '6.1.2 Generate Observations Statistical Data' to IPET-CDMP for advice on how we determine consistent rules to use for the generation of the statistics NB: We may need to move the user story 6.1.2 Generate WMO Standard Data Product under 5.1.2 Ingest and Extract. To be confirmed. It will depend on the level of analytical capability required to generate the WMO data products
6.1.3 Data homogenization	Preliminary analysis	100		User story: 6.1.3 Climate Data Homogenisation - ET-CDMS-1			This component needs to be split between QC and also analysis
7. Climate data presentation	Preliminary analysis	100	.mm .pdf	User story: 7.1 CDMS GUI - Data Exploration - ET-CDMS-1		2016-08	Also known as the User Interface
7.1.1 Tables and charts	Preliminary analysis	100
7.1.2 Manage content	Preliminary analysis	100
7.1.3 Visualization	Preliminary analysis	100
7.1.4 Integrated search of climate data	Preliminary analysis	100		User story: 7.1.4 Integrated Search - Find and Flag anomalous values - ET-CDMS-1 User story: 7.1.4 Integrated Search - classify observations by data quality - ET-CDMS-1 User story: 7.1.4 Integrated Search - Determine Prevailing Winds at Location - ET-CDMS-1 User story: 7.1.4 Recover from decommissioned station using Integrated Search GUI - ET-CDMS-1 User story: 7.1.4 aggregate climate stations using Integrated Search GUI - ET-CDMS-1
7.1.5 Data download	Preliminary analysis	100
8. Climate data delivery services	Detailed Analysis	75			Climate data services	2016-11
8.1 Open spatial standards	Detailed Analysis	75		User story: 8.1 Federated Global Data Service
8.2 Data discovery	Preliminary analysis	100		User story: 8.2 Discover data provenance - ET-CDMS-1 User story: 8.2 Data Discovery - ET-CDMS User story: 8.2.2 Linked Data access to observations - W3C			This needs considerable work
8.3 Other formats	Preliminary analysis	100
9. Core IT infrastructure	Preliminary analysis	100	.mm .pdf			2016-08
New Components
Configuration	Detailed Analysis	25	.mm .pdf	User story: xxx CDMS Configuration - User Management User story: xxx CDMS Configuration - New Data Product approval User story: xxx CDMS Configuration - New Station User story: xxx CDMS Configuration - New Observed Variable User story: xxx CDMS Configuration - New Sensor		2016-11	Need much more work on analysis. Need to create user stories from current mind map analysis and add current user stories to mind map.
Climate Data Management	Preliminary analysis	25		User story: xxx Climate Data Management - Manage Change in Station Location		2017-01	_Manage Change in Station Location_ user story needs careful thought and analysis. It has significant implications. As a general note for climate data management, assume that: - All observation data undergoes a suite of automated QC and QA processes - Some data will undergo a further manual QC/QA process if it is deemed appropriate - All data will be available via the climate data services and download - The NMHS? or WMO? policy will define what level of data quality (QA Flag) is available for download We should perhaps really aim for a position where all data is available for download, warts and all with appropriate data description in the Discovery Metadata and Data Provenance Metadata. Then it is a case of _user beware_. The data may be useful for specific use cases.
Data Intellectual Property	Not Started	0				2016-11	Functionality is required to store and manage the Intellectual Property of data. This includes: - Who ‘owns' the source data? - Has the person or organisation capturing the historical data assigned their IP rights in the data (this is particularly important with crowd-sourced data)? - Are there any constraints on the use of the data by others? - Copy of Data License Agreement. This component will need to cater for datasets and for data of mixed IP where source data comes from a number of sources. This is particularly important as NMHS move to embrace Citizen Volunteered data (e.g. 3rd Party data).

---

ISO standards. These documents are provided to WMO under a licence that only allows their use for developing WMO standards.