GISC Operations Page

GISC Watch Guide 202o v1 - Download



...page...

GISC Status reports

Template for GISC Status report (Annex 5 of final report ET-EISC-2106)

ET-WISC/TT-GISC 2016 agreed to future TT-GISC centre status reports containing the following structure and content.

• Area of Responsibility
  • AMDCN connectivity
  • DCs WIS compliance
• Training – Capacity building activity
• Operational issues in last period
  • Summary and Status Updates Open Tickets assigned to GISC
  • Status Updates on Open Upper Air BUFR Tickets
  • Metadata Synchronization Status
  • Number of Metadata records available
  • Core profile Version
  • OAI Provider URL
  • Status of own Set at GISCs
  • Status of remote Sets at GISC
  • Actions taken

• GISC Back up and activities
• Monitoring Activity & volume
  • Traffic size and volumes, uptime etc, user registration
• Changes in GISC, including new features since last meeting
• Progress on action items
• Outlook (planned changes/improvements)
• Any other issues of interest
• Summary (Text for report)

Note that all GISCs should provide the status report to TT-GISC annual meetings, even if unable to attend.

GISC connectivity

GlSCs shall connect to other GISCs through the WIS Core Network, which is based on the Main Telecommunication Network (MTN). Data, products and metadata shall flow to a GISC from the DCPCs and NCs that are within its area of responsibility. An Area Meteorological Data Communication Network (AMDCN) shall connect each GISC to DCPCs and NCs in the GISC area of responsibility. An AMDCN may span multiple Regional Meteorological Telecommunication Networks (RMTNs) and parts thereof.

A table of GISC connectivity is maintained by TT-GISC

• TT-GISC2018
• TT-GISC2017

...page...

GISC Backup

GISC Backup procedure: Para.6.3 Part VI, Guide to WIS (No.1061)

6.3 GISC BACKUP PROCEDURES
The Manual on WIS, Part III, 3.5.9.2, requires GISCs to maintain arrangements with one or more backup GISCs that include, as a minimum, the collection and dissemination of information for its AMDCN to be taken up by another GISC in case of an incapacitating system failure.
Note: Responsibilities of the backup GISC are limited to the centres designated in the backup agreement with the GISC.
6.3.1 Backup services
6.3.1.1 Data collection and distribution must continue without interruption to and from centres in the area of the GISC being backed up. Where a centre's routine receipt of data is through subscription (e.g. GTS push), the backup GISC must have a current list of data to be sent to each centre or provide a place for the centres to come and get the data (e.g. GISC Cache).
6.3.1.2 Centres may be unable to change their GTS subscriptions during a period of back up operation, and any changes to subscriptions might not be maintained when normal operations resume.
6.3.1.3 Changes to metadata will not be possible during a backup period.
6.3.1.4 Any ad hoc changes made during a backup period may need to be redone after return to normal operations.
6.3.2 User information
6.3.2.1 If there is a need to exchange user information between GISCs in support of backup, proper security measures should be taken based on the agreement between the two GISCs. However, the centres concerned should ensure that the backup GISC has sufficient information for sending and collecting data from the centres being supported during a backup period.
6.3.2.2 Ad hoc changes to subscriptions, including additions or deletions of subscribers, should be avoided while in backup mode. Any ad hoc changes made during a backup period may need to be redone after return to normal operations.
6.3.3 Networks
Global Information System Centres need to ensure network connectivity to centres in the AMDCN of the GISC they are backing up. This may be through dedicated links, such as GTS, or over the Internet. Such connectivity should be in line with the Guide to Information Technology Security (WMO-No 1115) and the Guide to Virtual Private Networks (VPN) via the Internet between GTS centres (WMO-No 1116), as applicable.

GISC Backup matrix (status) updated by TT-GISC

• TT-GISC2018
• TT-GISC2017

...page...

Metadata harvesting: Para.4.10 Manual on WIS (No. 1060)

4.10 WIS-TECHSPEC-9: CONSOLIDATED VIEW OF DISTRIBUTED DAR METADATA (WIS DISCOVERY METADATA) CATALOGUES
4.10.1 GISCs should exchange metadata catalogue updates using version 2 of the Open Archives Initiative–Protocol for Metadata Harvesting (OAI-PMH).
4.10.2 The exchange of metadata catalogue updates should satisfy the requirement for distributed instances of DAR metadata (WIS discovery metadata) not to diverge in content by more than one day. A mechanism for rapid update on an emergency basis should also be provided.
4.10.3 See also 3.5.6 (Discovery, access and retrieval).
...page...

GISC Cache

The scope of distribution for data within WIS shall be expressed using the following controlled vocabulary:
“GlobalExchange”, “RegionalExchange” and “OriginatingCentre”.
Data marked with RegionalExchange only needs to be in the cache of those GISCs involved in the distribution of such data.
Data marked with “OriginatingCentre” may be cached at the decision of the GISC, but are only expected to be available from the orginating centre.
Data marked with “GlobalExchange” must be cached by all GISCs. (See GISC Core Cache)
Cache file format: agreed by TT-GISC-2015

GISC Cache types

(Comment by chair TT-GISC -TOR which the TT-GISC should have afresh)
1. The Manual on WIS defines the GISC cache as a holding of 24 hours
2. The purpose of the cache is two fold:

1. To support Data Discovery and Access
2. To support reruns of data to centres in a GISC area of responsibility should they go offline or miss data.

3. Although defined as a 24 hour holding, this is the minimum period of retention.

1. A GISC may hold some data longer if the user requirement shows this is needed
2. The GISC cache must support the performance criteria of WIS by having warnings available within two minutes of their publishing into WIS.
3. The period of retention of a GISC cache beyond 24 hours is at the discretion of the GISC itself

4. Thus the GISC cache has two components.

1. The primary is the Core GISC Cache (See next page).
2. The secondary is the cache supporting regional or just those centres in its area of responsibility.

5. GISCs exchange files, but the default file format for holdings in the Cache is the Meteorological Bulletin


Note on Meteorological Bulletins
For meteorological bulletins, the format within 24h cache should allow the full “meteorological bulletin” to be stored. Information stored should be the complete bulletin as defined in the Manual on GTS Attachment II-15 para 5.2 on accumulating messages into files. It must include all text between the TTAAii of the abbreviated heading and end with “=” (equal) at the end of the text part as shown below.



...page...

GISC core cache

Inclusion of data in GISC core cache
The GISC core cache is defined by the keyword (flag) GlobalExchange
In order to ensure completeness of the Core Cache for GISCs:
1. The TT-GISC maintains the list of Core Cache data types, which are information flagged with “GlobalExchange” that all GISCs are required to have common holding (according to the Manual on WIS 3.5.3.1), and publishes the list through the WMO Secretariat.
2. The TT-GISC estimates the peak bandwidth used for the exchange of the Core Cache data by regular analysis of results from WIS Monitoring.
3. The TT-GISC coordinates and agrees on the minimum bandwidth required for each GISC to ensure the exchange of the Core Cache data.
4. Where TT-GISC cannot agree, or are unable to meet the expectatons of the data originator or user community, the decision will be escalated to the ITT-WIS for consideration of the President of CBS through Chair of OPAG-ISS.

Working practices

1. Daily data management is done without a decision of the TT-GISC.
(a). The TT-GISC publishes the data type list which the Core Cache data should include. The list should be clear enough that any new data is judged whether it’s included or not.
(b). When a WIS centre wishes to circulate new information through WIS, the centre may specifies the distribution scope “GlobalExchange” in the metadata, if the data type information corresponds to at least one of a type of Core Cache data type list.
(c). When a GISC receives information with metadata including the distribution scope keyword “GlobalExchange”, the GISC shall treat the information as Core Cache data.
(d). If a WIS centre submits a WIS metadata record which inappropriately has the keyword GlobalExchange, the principal GISC shall guide the WIS centre to an appropriate description.
2. Addition and deletion of Core Cache data type list requires a consensus of TT-GISC.
It was recommended in TT-GISC. In order to keep a history of Core Cache evolution requests, the TT-GISC should provide a template form to be filled with information (such as time input of the data and products, daily volume estimation …) by a WIS centre which wants to add new information in Core Cache.
3. The TT-GISC may add a data type in the following ranges.
This list materializes the definition of information intended for global exchange in the Manual on WIS 3.5.1 (time- and operation-critical information (data and products)). The list below only shows the maximum extent of the Core Cache, since it is not realistic to guarantee the global exchange of some of data, considering available bandwidth.
(a). Observation data that is specified in the Annex I to Resolution 40 (Cg-XII) (1) to (5)
(b). Observation data that is designated as the additional data according to the Resolution
(c). Products that are to be exchanged through the WIS in Manual on GDPFS (currently under development)
4. The first edition of the Core Cache data type list should be the minimal of necessity.
In order to ensure the stable operation, the list will be augmented within the bandwidth agreed at the TT-GISC. (e.g. (a) and (b) preceding paragraph) If some GISC cannot handle the data or there is a concern about the operation, addition will be deferred and will be proposed to the GISC to increase the bandwidth of the core network.
5. The TT-GISC may defer adding the data type which has concern about the communication bandwidth. NWP and Satellite data are examples at the moment.
As it is specified in the Manual on WIS, the Core Cache is required that the all GISCs have to exchange and cache it, and should be acceptable for all GISCs in terms of capacity, such as a server capacity and bandwidth of the core network. Thus it should be estimated to be able to pass through a circuit with the lowest bandwidth. It was recommended in TT-GISC.
6. The TT-GISC will regularly estimate the required bandwidth by analyzing the result from WIS monitoring. To create a mechanism of estimating the bandwidth required that is used for the Core Cache data exchange, GISC should monitor the bandwidth usage of the Core Cache data exchange. Even if a list of the data type is not changed, the demand for communication bandwidth changes over time. The TT-GISC will investigate the actual amount of data transferred in regular interval (e.g. every 3 months or every 6 months), and estimate the maximum communication bandwidth which is expected to be used for exchange of current data types. Separately, the TT-GISC will agree on a communication bandwidth that each GISC shall provide, and it may add data types only when the data congestion does not occur. (A possible criterion of non-congestion is that daily volume of traffic to be passed over any one circuit shall not exceed 80 per cent of its theoretical capacity in accordance with the Manual on GTS 1.3, Principle 4)

Escalation procedure Annex 2 to Rec. 36 (CBS-16)

1. Global Information System Centres’ representatives (i.e. TT-GISC) should, based on discussion with the providers (Data Collection or Production Centres and National Centres) and users, be the group to decide whether a data stream should go in or out of the 24 hour cache and be routinely distributed:
(a) That all GISCs have to maintain (e.g. GlobalExchange flag);
(b) That a number of GISCs have to maintain (e.g. RegionalExchange flag).
2. Normally, the decision to add a new or to remove an existing data stream will be by consensus of GISCs representatives in line with normal operational collaboration:
(a) An implementation time line will be a part of the decision;
(b) Decisions should be made in a short timescale (e.g. less than 2 weeks).
3. If unable to reach consensus or if the requester is not satisfied with the decision, the problem should be escalated to ITT-WIS:
(a) ITT-WIS should then make its recommendations based on information from requesters and GISCs;
(b) The president of CBS will make a decision based on the ITT-WIS recommendation, utilizing fast track procedures as appropriate.

Note that issues may be escalated by a GISC either in response to events, occurred or planned,
where it is anticipated it might impact on the functioning of WIS.
...page...

TT-GISC Email Groups (TT-GISC2018)

TT-GISC discussion and information forum

The primary email group for TT-GISC is cbs-wisc-tt-gisc@wmo.int. This is an email group for the focal points of WMO Global Information System Centres (GISC), and includes members of the CBS Expert Team on WIS Centres' Task Team on GISCs. The email group consists mainly of individuals emails, although some GISCs have subscribed their operational GISC email alias to the group. Only members of the email group can send and receive emails to or from the group. Many individuals also subscribe secondary emails to this group, mostly where they are unable to associate their work email with G-Mail. The secretariat moderate this email group, including spam protection.

GISC Operations email

The second TT-GISC email group is gisc-ops@wmo.int. This email group is to exchange information between GISCs relating specifically to GISC Operations, including GISC Watch. This should be the primary email for announcing to other GISCs important planned changes such as new system configuration (e.g. change URL of OAI harvester) or for handing over the GISC Watch duty. The aim is for these messages to get to the operations centre (or help desk) of each GISC and to be monitored by staff on duty. Thus, subscribers to the group are primarily generic GISC email groups although some individuals have also subscribed. Each GISC can manage internally how emails received from this group are distributed among their staff. The secretariat and Chair moderate this email group and can approve submission from email addresses not registered in the group.
This list of generic GISC email addresses is maintained by the chair of TT-GISC who refers to these as the GISCs help desk email, although this is not really the case for all GISCs. The meeting noted that all GISCs have provided this contact information on their GISC portal and it can be used for 24/7 issues (need to take actions immediately), such as System/network trouble/outage.

GISC communications: agreed by TT-GISC-2015

Technology

Email, phone, fax, wiki, WIS Core network

Message format

The message should be on a standard form for TQM purpose and posted on the WIS-WIKI. For Example:

Source GISC:
Destination GISCs:
Message date & time:	Validity period:
Subject:
Message body:

Communication Language

English should be the primary language and if the GISC server a second language it can be added in the message part.
...page...

...page...

Stop gap metadata should use “WMO-other” (ET-WISC/TT-GISC 2016 agreed)

49. ET-WISC agreed that stop gap metadata should use “WMO-other” rather than “WMO-additional” because the data or product might be more restricted than “WMO-additional”

61. Mr Markus Heene presented TT-GISC Doc06 on data policies. The meeting noted that the paper highlighted two different possibilities for encoding the “WMO data policy for globally exchanged data” (“WMOEssential”, “WMOAdditional”, “WMOOther” and “NoLimitation”) following WMO Core Profile 1.3. the paper included an analysis of the impact on the users and GISCs. In particular it showed an existing operational problem in some GISC implementations which potentially couldn’t handle both encodings. The two methods are 1) to use gmx:Anchor and 2) free text:CharacterString

62. The meeting urged the GISCs to check if their implementation can cope with both possible encodings of the WMO data policy. Furthermore TT-GISC asked IPET-MDRD to consider this example and to avoid in the next version of WMO Core Profile multiple encoding styles for crucial information to minimize the implementation work for the GISCs.
...page...

Metadata Guidance

WMO web site

• WIS metadata
• WIS metadata Guide
• OAI Doc

Part V. METADATA GUIDANCE

• Guide to WIS (No. 1061)

...page...

Metadata Migration Procedure GISC A to GISC B

Requirements from new GISC:

1. Operational OAI-PMH server reachable from internet
2. GISC area of responsibility metadata set available (name shall be WIS-GISC-XXXX where XXXX is the city name of the GISC)

Procedure

1. The new GISC shall inform others GISC for the availability of OAI-PMH server by mailing to : cbs-wisc-tt-gisc@wmo.int and gisc-ops@wmo.int plus copy to wis-help@wmo.int
2. Others GISC shall check that the new OAI-PMH server is reachable and send email confirmation to new GISC.
3. New GISC contact GISC Tokyo and furnish the list of metadata which shall be removed from the WIS-UNASSOCIATED set.
4. GISC Tokyo sets a delete flag in metadata which shall be removed. Tokyo also informs all GISC of the number of metadata which has been removed and ask to others GISC to re-harvest WIS-UNASSOCIATED set by mailing to cbs-wisc-tt-gisc@wmo.int and gisc-ops@wmo.int plus copy to wis-help@wmo.int
5. All GISC shall re-harvest WIS-UNASSOCIATED and check that the metadata have been deleted.
6. New GISC add update flag their all metadata and ask to others GISC to re-harvest their WIS-metadata set by mailing to cbs-wisc-tt-gisc@wmo.int and gisc-ops@wmo.int plus copy to wis-help@wmo.int
7. All GISC shall start harvesting metadata set from new GISC.
8. All GISC shall confirm to the new GISC that actions have been completed.
9. GISC Tokyo physically removes their metadata from WIS UNASSOCIATED.

Recommendations:

• Action 1 shall be done by a week before starting management of metadata.
• Action 2 shall be done in a maximum delay of 5 days.
• Action 5 and 6 shall be done in a maximum delay of 5 days.

...page...

WIS Monitoring

JSON Specification for WIS Monitoring: agreed by ET-WISC 2016

• specification

WIS Common Dashboard (WCD) Centres

• GISC Beijing
• GISC Tokyo

...page...

GISC Watch

GISC Watch tasks

1.      The on-duty GiSC shall
a)      Carry out the GISC Watch by using information exchanged in JSON files. The WCD is advised to be used.
b)      Provide a summary report (template to be defined), and
c)      Formally hands over the GISC Watch operation to the next GISC

2.      The tasks performed by the on-duty GISC for GISC Watch shall include at least:
a)      Monitor the status of a service of each GISC, including OAI PMH, SRU and HTTP portal, on daily basis.
b)      Monitor the number of metadata records of each GISC (should be similar to each other and no sudden big changes)
c)      Send notification on identified problems/issues to the concerning GISC(s), follow up on the actions, and escalate issues as needed.
d)      Use agreed issue tracker system (MANTIS - http://www.inmet.gov.br/giscticket/) to report the incidents.

3.      Specific tasks:
a)      WCDs (JMA and CMA) are requested to store/present the history of those matrixes mentioned in 2.a) for one month.
b)      As agreed in 2017 ET-WISC, GISC Brasilia will host and operate an issue tracking software. (Done. 10/2/2018. http://www.inmet.gov.br/giscticket/)
c)       GISCs currently not issue JSON files, please provide your JSON files to WCD asap.

Roster Rules
a)       Each roster stats from 00:00UTC and ends at 23:59UTC of the planned roster period.
b)       Every GISC should provide a email address (GISC_email) for the purpose of conmmunication, preferablly an alias that include multiple operators so that important information is not missed.
c)       Duty GISC shall hand over the monitoring duty to the next GISC by sending the report to the GISC_email, along with any specific information deemed necessary/useful to the next duty GISC.
d)     To ensure the continuity of GISC Watch, the monitoring duty shall be carried out by the duty GISC at the beginning of the roster period regardless whether it has received handover email from the previous GISC.
e)      The GISC Watch report shall be sent to 1) next Duty GISC (GISC_email), 2) TT-GISC mailing list (CBS-WISC-TT-GISC@wmo.int), 3) WMO Secretariat (wis_help@wmo.int)
f)       Duty GISC shall register any issue in the GISC-Ticket system (http://www.inmet.gov.br/giscticket/) and make update when necessary
g)      For any issue related to GISC Watch that requires escalation, the Chair of TT-GISC is the first escalation point for all GISCs

Report template

Duty GISC	GISC Tokyo	Period 2018-02-01 — 2018-02-15
Issue ID	Date	GISC	Issues Action Taken	Status	Further Action
2018-1	2018-02-01	GISC A	GISC portal off air GISC A notified	Resolved
2018-2	2018-02-02	GISC B	OAI provider inactive GISC B notified	Open	Follow up
2018-3	2018-02-03	GISC C	Can't reach GISC A GISC C and A notified	Resolved
2018-4	2018-02-04	GISC D	Abnormal increase of the number of Metadata GISC D notified	Open	Follow up
XX
XX
XX
XX

….
Notes:
1. For every calendar year, the Issue ID should be continuous through out the year, which gives each issue an unique ID.
This is easy for communicating the issues among the GISC for taken action. The ID may change once we start using Issue Tracking System
2. "Further Action" is the suggestion from duty GISC to the next GISC on issues that are open and need to be followed up by next duty GISC.

Roster

GISC	2nd - Round	3rd - Round	4th - Round
Tokyo	2018/12/16 - 2018/12/31	2019/08/01 - 2019/08/15	2020/03/16 - 2020/03/31
Beijing	2019/01/01 - 2019/01/15	2019/08/16 - 2019/08/31	2020/04/01 - 2020/04/15
Melbourne	2019/01/16 - 2019/01/31	2019/09/01 - 2019/09/15	2020/04/16 - 2020/04/30
Pretoria	2019/02/01 - 2019/02/15	2019/09/16 - 2019/09/30	2020/05/01 - 2020/05/15
Exeter	2019/02/16 - 2019/02/28	2019/10/01 - 2019/10/15	2020/05/16 - 2020/05/31
Toulouse	2019/03/01 - 2019/03/15	2019/10/16 - 2019/10/31	2020/06/01 - 2020/06/15
Seoul	2019/03/16 - 2019/03/31	2019/11/01 - 2019/11/15	2020/06/16 - 2020/06/30
Casablanca	2019/04/01 - 2019/04/15	2019/11/16 - 2019/11/30	2020/07/01 - 2020/07/15
Offenbach	2019/04/16 - 2019/04/30	2019/12/01 - 2019/12/15	2020/07/16 - 2020/07/31
Brazilia	2019/05/01 - 2019/05/15	2019/12/16 - 2019/12/31	2020/08/01 - 2020/08/15
New Delhi	2019/05/16 - 2019/05/31	2020/01/01 - 2020/01/15	2020/08/16 - 2020/08/31
Washington	2019/06/01 - 2019/06/15	2020/01/16 - 2020/01/31	2020/09/01 - 2020/09/15
Jeddah	2019/06/16 - 2019/06/30	2020/02/01 - 2020/02/15	2020/09/16 - 2020/09/30
Moscow	2019/07/01 - 2019/07/15	2020/02/16 - 2020/02/29	2020/10/01 - 2020/10/15
Tehran	2019/07/16 - 2019/07/31	2020/03/01 - 2020/03/15	2020/10/16 - 2020/10/31

...page...

Fast Track Changes

• a) GISCs should implement fast track changes within 6 months after the date of effect
• b) Secretariat should email notifications of approved fast track changes to TT-GISC email group

...page...

Other miscellaneous reference material for GISCs

• GEOSS API Documentation