Minutes for teleconference on Wednesday 31/3/2010 regarding EC-demonstration

Participants:

From DWD: Markus Heene, Siegfried Fechner
From Inmet: Jose Mauro Rezende
From WMO: Eliot Christian, Timo Pröscholdt

The tentative agenda was

1. Welcome
2. DAR Metadata
3. OAI Harvester / Provider
4. GISC to GISC communication
   1. AFD
   2. Moving Weather
5. any other business

No changes were made to the tentative agenda:

1. Welcome

Markus stated that the telco should be about technical details and that the EC and CBS demonstrations should be discussed once the technical details were solved. The meeting agreed to this. WMO will set up a WIKI page and send out an email to INMET, CMA, DWDD and JMA asking for comments and suggestions on the demonstration.

2. DAR Metadata

INMET now has a working Geonetwork running which can by synced from DWD. Markus has looked at the metadata and noticed some problems with validation. The reason the data is not valid is that Brazil used outdated German or French metadata obtained at WMO.

The meeting discussed how INMET can be provided with up to date metadata. Two options were considered.
First, providing INMET with a set of already generated Metadata for the synchronization test. DWD uses an Oracle DB as storage of its metadata, so it could provide an Oracle Table dump of the metadata. But this would be difficult for INMET to import, since they do not currently have Oracle. The meeting discussed the feasibility of data exchange and some way will be found to exchange the data, probably by producing a non proprietary comma separated dump of the oracle DB which can be imported into any other DB.
Second, providing INMET with the newest version of the metadata generator. The metadata generator also works off an Oracle DB, but it is coded in a portable way, so that it can be plugged into any other JDBC DB. DWD will send the code to WMO, so that WMO can look at the code and assist INMET in setting it up with a MySQL DB.
Siegfried remarked that using the metadata generator, which imports metadata from Volume A and C is not trivial.
For testing between INMET and DWD, INMET will receive a dataset from DWD. It should however be kept in mind that INMET, in order to become a GISC, needs to master Metadata generation.
DWD will also make available
By the end of next week two datasets on its OAI server. One set contains the German DAR (ccc_DWD). The other set is the set_cbbr containing a set with metadata generated for Brazilian data.

3. GISC to GISC

The meeting discussed ways to sync the Brazilian and German GISC caches.
There are two ways of doing this. Either using the Moving Weather software or to use AFD.
The issue of the testdataset that will be used in the syncing was also considered.
DWD also proposed to use the syncing of the global cache as use case for the EC demo.


DWD has in the past tested the syncing with CMA. At the time AFD was used and a dataset corresponding to one month of testing was employed. This dataset has 190.000 files weighing 13GB and having satellite image files of up to 17MB. Syncing was done in both ways using ftp and ftps protocols. Warnings can be synced in less than 6 secs.
It was stated that for GISC to GISC syncing MSS over GTS has to be used, as agreed by ET-WISC. However, CMA and DWD can also use AFD.

The meeting agreed that the technical experts (networkers) have to talk to organize the actual sync testing, because the network transport has to be provided. Due to time constraints around Easter, the tests will now take place in the week of the 12th of April.
The test will use the internet as transport since both the Brazilian and German hosts are reachable from the internet. The firewall people will make the machines mutually reachable.

Brazil can do the syncing with both AFD and Moving Weather. Their hosts are:
(Moving Weather): 200.252.242.85
(AFD) 200.252.242.74

After discussion of the protocols during which DWD proposed to use SFTP for data consistency, it was agreed that initially FTP should be used, since this causes less operational issues with the AFD and Moving Weather programs and the machines they are installed on.
INMET and DWD will exchange the IP addresses, usernames and passwords that will be used in the syncing by email shortly.

Then the dataset that should be used for syncing was discussed, noting that there is so far no MSS traffic between DWD and INMET. A question was if the data had to be representative for INMET / DWD traffic? It was stated that 13GB are more than what is currently exchanged between DWD and INMET via MSS. Satellite images are not usually sent via MSS, so not representative.
DWD said that it might be useful to do the sync via MSS so that we see that it can handle big images.
The meeting agreed to use the DWD / CMA dataset for the INMET / DWD cache sync test.

The size of the dataset previously used for CMA / DWD sync would not be a problem for INMET or DWD. DWD has a test system, so there will be no interference with production use. Brazil will do the tests on the live server, but no risk that traffic gets injected into the live system since channels can be monitored and switched off in case of trouble, since there are not so many.

4. AOB ( any other business )

Nothing.


Next telco on Monday 12th of April. 12 UTC.