cbsdemo-minutes2

WIS » ET-WISC » CBS-demo » cbsdemo-minutes2

Teleconference on GISC synchronization (in preparation of WIS functionality demo to CBS)

Monday 30th of August 8:00am GMT, WMO WEBEX system

Teleconference on GISC synchronization (in preparation of WIS functionality demo to CBS)

List of participants:

Lipeng, Jiang CMA
Siegfried Fechner, DWD
Markus Heene, DWD
Heinrich Knottenberg, DWD
Jose Mauro de Rezende, INMET (absent)
Eizi Toyoda, JMA
Jitsuko Hasegawa, WMO
Timo Proescholdt, WMO

Agenda

Opening
Status of preparations
Discussion of the test
Timeline
Any other business
Closure

Minutes

The participants met again in WMO’s new WEBEX system. Since Eizi Toyoda’s (JMA) microphone was not working and Jose de Mauro Rezende (INMET) could not finally make it, it was decided that Timo Pröscholdt (WMO), who had worked with Jose on the INMET GISC implementation, would brief the participants on his behalf. Everybody would be reading Eizi’s comments in the WEBEX chat window. Now that everybody was connected, Timo welcomed the participants, introduced Jitsuko Hasegawa from the WMO WIS team. He emphasized that the purpose of the tests was to prepare for the demonstration of WIS to CBS. The tests intend to examine technical as well as operational aspects of the WIS DAR synchronization over a longer period, which is also important for the implementation of WIS.
The participants briefed each other on the preparations for the DAR synchronization test.
- Markus Heene (DWD) said that the 100 testmetadata records had been created and the two OIA sets, although not yet there, would be configured soon. He added that DWD’s OAI harvesting and providing procedures were ready for the test and that he had developed a simple procedure for checking if a changed metadata record has propagated successfully.
- Lipeng Jiang (CMA) told the participants that CMA had created 100 testmetadata records, configured WISTEST-CMA and WISTEST-CATALOGUE sets and that he would write a program to verify the propagation of changes, using SRU to fetch the updated records from the other GISCs.
- The teleconference was glad to learn from Eizi Toyoda (JMA) that although JMA was very busy at this time, they would join the test, supporting critical WIS developments. He said that it would be easy to provide 100 metadata and the needed OIA sets.
- On behalf of Jose de Mauro Rezende (INMET), Timo briefed the participants that INMENT had created 100 testmetadata with the kind assistance of DWD, and that WISTEST-INMET and WISTEST-CATALOGUE OAI sets had been configured. Harvesting procedures are also working, the only thing that remained to be done was to define a (probably manual) procedure to regularly update the metadata.
The meeting then set to discuss aspects of the synchronization tests.
- Markus noted that Geonetwork, the software used by INMET for its GISC implementation, had a problem with its OAI component. The OAI identifier, which the participants agreed should be identical to the ISO19139 fileIdentifier, does not respect the case in the fileIdentifier element and is always lower-case instead. Noting that this was not a critical problem, since the uniqueness of the identifier is maintained even without case-sensitivity, Timo promised to work on a fix, so that all OAI-identifiers over the participating GISCs would be identical, even though the teleconference noted that equality of fileIdentifier and OAI-identifier was not an IPET-MDI recommendation.
- The meeting also agreed that the WISWIKI would be used to share essential information about the test. The minimum information to share was identified to be fileIdentifier, date of the change, nature of the change, actual change, and a field capturing the status of the propagation to all other participating GISCs. An example of this information can be found here.
- The meeting noted that an update of a metadata record involves at least the following two changes. First, the dateStamp element has to be set to the current date. Second the actual change must be performed. The minimal verification of changes comprises checking that all GISCs have picked up the updated/new metadata record by retrieving the record (for example via the portal site or SRU) and checking that the new timestamp and the element content are present. Deletion would work according to IPET-MDI recommendations. DWD was of the opinion that OAI could also be used to retrieve the changed metadata from the other GISCs and informed the meeting that is was planning to do this. Timo explained that the suggestion to use SRU or the portal was due to a methodology by which a protocol should not be tested by itself. The meeting noted that it was important to reliably verify the changes and expressed confidence in DWD to find a good procedure to this end.
- To facilitate the testing of insertions into the catalogue, the participants agreed that initially each GISC would only insert 50 records into its OAI-set, while the remaining 50 would be used to test the insertion of records. The participants also noted that the maximum number of test metadata records would be 100 per centre, since the test was of functional rather than load-testing nature.
- Note from Timo: We did not discuss the frequency of the changes. For the tests to be meaningful there need to be sufficient updates. The norm should be to have 1-3 updates/insertions/deletions per day, considering work-load, with automatic mass updates encouraged.
The participants then set a timeline for the tests. Timo noted that, CBS being in November, there was enough time and no need for rush, so that everybody could make the necessary preparations. Finally it was agreed that the test would start in two phases. Each GISC would share some of its test metadata records via the OAI sets WISTEST-#GISCNAME# and WISTEST-CATALOGUE on Thursday 2nd of September, to check if the other GISCs can import the data. The actual test would then start on Wednesday 22 of September.
Timo gave an outline of the secretariat’s plan for CBS. Steve Foreman, chairman of OPAG-ISS, has asked the WMO secretariat to organize for CBS a demonstration of WIS along the lines of the EC demo. The current tests will be the base for the demonstration of WIS during the WIS session at the CBS–teco. He emphasized that this demonstration of WIS is independent from the formal demonstration of WIS centres to the ET-GDDP. He continued that a new version of “the script” (for EC) would have to be developed at a later point in time, this time featuring multiple GISCs, and that there will be an exhibition space for WIS centres to show posters and other materials. The meeting was pleased to learn that Méteo France and the MetOffice are also working on a demonstration and welcomed them to join into the tests. Heinrich Knottenberg (DWD, chair of ET-WISC) added that the test should also be seen as step towards the implementation of WIS and that CBS would be pleased to learn about this, too. Markus suggested that the GISCs work together in the development of software to verify the propagation of changes. The others welcomed the suggestion and it was decided that further discussion would take place by email.
The meeting was closed at 9:16 am GMT, thanking all the participants for taking part in the tests and emphasizing that the work done by the group constituted a leap forward in the implementation of WIS.

Action Items:

until 2nd of September
- provide some testmetadata records in OAI-set WISTEST-#GISCNAME#
until 22nd of September
- provide the initial 50 testmetadata records in OAI-set WISTEST-#GISCNAME#
- have the 50 other records ready
- provide OAI-set WISTEST-CATALOGUE accumulating (via OAI harvesting) metadata from all WISTEST-#GISCNAME# sets
- implement test procedure: have software/manpower/time to do roughly 3 updates per day (or more)
- verification procedure (manual or automatic) to check propagation to all other GISCS

Information:

Annex (WEBEX chat)

from Markus to Everyone:
hi to everyone
from Lipeng to Everyone:
Hello
from Lipeng to Everyone:
Hi Markus
from Lipeng to Everyone:
yes, I can
from Markus to Everyone:
eizi you can hear us?

<conversation between Eizi and Timo to the effect that Eizi can listen inn but the mic does not work>

from TOYODA Eizi to Everyone:
memo: Markus don't want to make separated logfile (of probably OAI)
from TOYODA Eizi to Everyone:
hard to say
from TOYODA Eizi to Everyone:
we're very busy fixing programmes
from TOYODA Eizi to Everyone:
but whatever experiment we will join.
from TOYODA Eizi to Everyone:
no its easy to give 100 metadata
from TOYODA Eizi to Everyone:
sharing is welcome of course
from TOYODA Eizi to Everyone:
Google Docs is no good for DWD
from TOYODA Eizi to Everyone:
but I guess Google Groups may works
from TOYODA Eizi to Everyone:
yes
from TOYODA Eizi to Everyone:
I thought it works like mailing list, am I wrong?
from TOYODA Eizi to Everyone:
so the idea is mailing list with log facility
from TOYODA Eizi to Everyone:
wiki is also fine
from TOYODA Eizi to Everyone:
great to hear you're working on it.
from TOYODA Eizi to Everyone:
good. and for the time being we start with WIS wiki and email without log.
from TOYODA Eizi to Everyone:
So DWD wants to have lowercased (gmd:fileIdentifer) as oai:identifier?
from TOYODA Eizi to Everyone:
FYI we are going to use gmd:fileIdentifier as oai:identifier. There are still old metadata records, but we plan to migrate "urn:...." form.
from Markus to Everyone:
case sensetive or case insenseitve?
from TOYODA Eizi to Everyone:
currently case sensitive since we have no case-folding code.
from TOYODA Eizi to Everyone:
But we're glad to change it to case-insensitive if that contributes entire network stability
from TOYODA Eizi to Everyone:
Sorry I'm difficulty getting your idea
from TOYODA Eizi to Everyone:
I see
from TOYODA Eizi to Everyone:
sounds good
from TOYODA Eizi to Everyone:
because insertion and modification is different thing.
from Markus to Everyone:
Question start Test 27th of September?
from TOYODA Eizi to Everyone:
sounds good. I can't guarantee but I hope we sort out our challenges by that season.
from TOYODA Eizi to Everyone:
can the metadata the same as our GTS ones?
from TOYODA Eizi to Everyone:
year so we make "tweaked" GTS metadata
from TOYODA Eizi to Everyone:
that won't be too long
from TOYODA Eizi to Everyone:
enable OAI set by thursday?
from TOYODA Eizi to Everyone:
okay I'll try
from TOYODA Eizi to Everyone:
bye
from Markus to Everyone:
bye
from WMO Webex 5 to Everyone:
bye

This is the static archive copy of the old wiswiki, decommissioned on June 1 2020