Telco on the 12th of April regarding WMO EC demonstration

Participants:
DWD: Markus Heene, Siegfried Fechner, Christoph Müller
INMET: Jose Mauro de Rezende
WMO: Eliot Christian, Timo Pröscholdt

Tentative Agenda:

• welcome
• 24h cache synchronization
• DAR Metadata
• AOB

Welcome:

DWD enquired about the status of Brazils RMDCN connection, noting that so far the tests will be conducted using the internet. Jose told the others that a proposal has been negotiated and that implementation and getting connected is a matter of signing the contract.

24h cache synchronization.

The participants discussed a test of 24h global cache synchronization between INMET and DWD, in preparation for the WMO EC demonstration as well as capacity building for INMET’s GISC implementation.
Timo wanted to know if AFD (Automatic File Distributor, a DWD tool) or MW (Moving Weather), will finally be used, since Jose had remarked that usage of AFD would make it easier for non Moving Weather countries to join in the syncing. Jose remarked that as far as Brazil is concerned both are ok.
DWD said that the test should be done using current GTS procedures, thus MV. This was ok for Jose.
It was also agreed that the dataset which was used by DWD in the tests with CMA (see meeting notes 31 March for description) should be used for the INMET / DWD tests. By using the same dataset, the tests become comparable. It was also decided to use FTP. DWD asked if the INMET servers are time synced with NTP, which Jose affirmed.
The participants discussed which information should be measured during the tests. It is important to know the transfer time of messages/files from the origin till they are picked up by the other system. It is also critical to know if files get corrupted or lost. DWD has a script which measures the transfer time of files and messages as well as file corruption. It was successfully used in the CMA / DWD tests and the participants decided that it will also be used for the DWD / INMET cache tests. The script runs for 24 hours, simulating a normal day of GTS traffic. (peeks in data flow as well as a full transfer burst).
DWD and INMET will proceed to exchanging ftp account and MV channel names. DWD will also send the test script together with instructions on how to use it to Brazil. It was noted that some system programs have to be replaced in order for the script to work.
A date for a tentative first test sync was agreed: Wednesday 00 UTC.

DAR metadata

Markus told the meeting that DWD has successfully closed its action items. The Metadata for Brazil has been generated (143 records) and made accessible via OIA and FTP. The DWD OAI server is now accessible from everywhere. DWD is ready for tests with INMET.
Jose will delete his geonetwork DB and load the new records into it, which he will download from the DWD ftp server.
Timo has ported the Oracle DB dump as well as the DWD metadatagenerator so that it works with MySQL or other non Oracle DBs. He will provide the dump files and the code to Jose, so that INMET can start generating their own metadata.
The meeting then discussed issues that currently prevent DWD/INMET DAR sync. Jose has tried to harvest with the Geonetwork harvester from the DWD JOAI provider. Some data is transferred but there are error messages and not all data get harvested. Timo is currently looking into what is wrong with the GN and JOAI implementations. He thinks the problems are due to the way GN validates OIA responses against XML schema. Will report back this week what needs to be done to fix this, expressing hope that a change of the XML schema will suffice.
DWD has successfully harvested from INMET’s GN installation. There were no problems because DWD has fixed their harvester so that it does not have to session problems usually experienced with the GN provider. Timo noted that this part in GN also needs fixing, since it is better to fix the problem at the source rather than changing all possible harvesters. He will look into it this week.

Any other business

After that the meeting then discussed the synchronization scenario for the EC demonstration.

The role of the DCPC’s in the scenario was considered. WMO highlighted that that the DCPC’s taking part in the demo should be DCPC’s already working together closely with the GISC’s, since this makes problem solving easier. This meeting agreed on this.
Potential DCPC where then discussed.
Brazil has the CPTEC space institute as potential Brazilian DCPC and there is also a cooperation with Argentina as DCPD. DWD will probably propose the GPCC as DCPC. There is also National Climate Data Center with real time data generation.
The meeting decided that Brazil should focus on the CPTEC, since the data is more complementary to the data already available. Interesting data is important to show the idea of WIS to delegates. DWD should work with GPCC, since the National Climate Center is not a DCPC and this might be irritating the EC.
DWD will prepare one special MD record for the dry-run. This MD will have some unique characteristics, so that it can easily searched and its presence or absence in the catalogue be easily shown. The MD record should come from the GPCC. It was understood that the catalogue would not be empty during the EC demo, but that the special MD record would be sufficiently “different” so that it can be found without other records being in the search result set.


The meeting then talked about the online resource link in the MD.
The matter is currently under discussion in the relevant ET’s. In a nutshell, if the metadata described data intended for global distribution, two ways of providing a link for accessing the data being described by a MD record exist. One is to have several online resource links in the relevant MD element. One would be a link to the real source of the data (often a DCPC), the other one would be the responsible GISC and its partner GISC. This is easy to accomplish, since only the links have to be inserted. However, some knowledge of the way the data can be accessed at the partner GISC is needed.
The other way is to use an “INSTANCE PATTERN” which allows each GISC, when displaying the search results, to blend in a URL allowing the user to download the data from the local GISC cache.
For the EC-demonstration we will use the first method, since it is easier technically and conceptually.

Markus will talk to CMA and JMA about the access to global Data at their GISC’s, since the links need to be known in order to be inserted into the test metdadata for the demonstration. WMO will also organize a telco with JMA and CMA.

the participants promised to consider, discuss and convey to WMO their technical and non-technical requirements for the EC demonstration. The bulleded list at the bottom of (et_wisc_wisdemo) is food-for-thought.

Next telco 26th of April.