The Global Telecommunication System (GTS) is currently the backbone of the real-time transmission of World Meteorological Organization (WMO) data globally. The Sixth World Meteorological Congress in 1971 approved the Manual on GTS and started its operational life. Since then, the GTS has proved to be a reliable real-time exchange mechanism of essential data. It provides observations to all WMO Members and Global Data-processing and Forecasting System centres, including ECMWF, as well as disseminating processed information to national meteorological and hydrological services (NMHSs). Despite some evolution of the technologies used for data exchange, the GTS has kept its basic technical foundations unchanged. The emergence of increasingly rapid, high-bandwidth global connectivity through the Internet offers new opportunities for the future evolution of the GTS. This will be achieved by upgrading the WMO Information System (WIS), which uses the GTS, to WIS 2.0. Centres migrating to WIS 2.0 will have the possibility to switch off their GTS reception and transmission equipment after the migration. The process is expected to finish by 2033, while most centres will have completed the migration by 2030.
Exchanging data through the GTS
The mechanism to exchange data through the GTS is based on a ‘store and forward’ mechanism. A message received by a centre is stored and forwarded to the ‘next’ centre in a complex point-to-point topology designed in 1969 and still operational today with very few changes. This mechanism, which predates the Internet, uses private networks to ensure the high availability of connections between NMHSs. Today, however, migrating to the Internet could provide a similar level of resilience at lower costs.
In GTS, the messages are routed from one point to another of the network using identifiers called ‘GTS headers’. Based on groups of six letters, these headers are statically assigned to bulletins, and ‘routing tables’ are maintained in each transmission centre to direct the messages along the planned route through the network. However, the static nature of routing tables and the relatively simple syntax of the GTS identifiers are not scalable to the current explosion in both volume and variety of data.
A further limitation of the GTS is the complexity of the topology, which requires a level of coordination between WMO Members that is sometimes difficult to reach for various technical and political reasons. However, the Internet and web technologies as a backbone for global data and information exchange offer a straightforward way to help the WMO resolve many of the fundamental data exchange issues related to the architecture of the GTS.
The WMO Information System
A significant improvement of the system was initiated by the WMO Congress in 2007 and led to the development of the WMO Information System (WIS), which was intended to complement the GTS. WIS provides a searchable catalogue and global cache to enable additional discovery, access and retrieval services through web portals maintained by 15 designated Global Information System Centres (GISCs), each operated by a WMO Member.
The WIS also defined new roles for WMO Centres worldwide, recognising the need to improve Members' coordination and facilitate data exchange beyond the World Weather Watch. However, the WIS still uses the GTS as its underlying operational service for data exchange with only minor improvements, thereby inheriting most of its intrinsic limitations.
WIS 2.0 has been designed to address the current WIS and GTS issues, to support the WMO Unified Data Policy, and to meet the demand for high data volume, variety, velocity and veracity.
WIS 2.0 principles
A set of principles is at the foundation of the WIS 2.0 technical framework and was used to define its architecture. The principles can be summarised with the following three foundational pillars:
- Simpler data exchange
- Open standards
- Cloud-ready solutions.
Simpler data exchange
WIS 2.0 prioritises the use of public telecommunication networks, unlike the private networks used for GTS links. As a result, using the Internet will enable the best choice for a local connection, using commonly available and well-understood technology.
WIS 2.0 aims to improve the discovery, access and utilisation of weather, climate and water data by adopting web technologies proven to provide a truly collaborative platform for a more participatory approach. Data exchange using the web also facilitates easy access mechanisms. Browsers and search engines allow web users to discover data without specialised software. The web also enables additional data access platforms, e.g. desktop geographic information systems (GIS), mobile applications, forecaster workstations, etc.
The web provides access control and security mechanisms that can be utilised to freely share the core data as agreed by the WMO Unified Data Policy and protect the data with more restrictive licensing constraints. Web technologies also allow for authentication and authorisation. This enables the provider to retain control of who can access published resources, and to request users to accept a licence specifying the terms and conditions for using the data as a condition for providing access to them.
WIS 2.0 uses a ‘publish–subscribe’ pattern, where users subscribe to a topic to receive new data in real time. The mechanism is similar to WhatsApp and other messaging applications. It is a reliable and straightforward way to allow users to choose their data of interest and to receive them reliably.
Leveraging open standards
WIS 2.0 leverages open standards to avoid building bespoke solutions that create niche markets and force NMHSs to procure special equipment. In today's standards development ecosystem, standards bodies work closely together to minimise overlap and build on their respective areas of expertise. For example, the World Wide Web Consortium provides the framework of web standards, which the Open Geospatial Consortium and other standards bodies leverage. WIS 2.0 relies on open standards with industry adoption and wide, stable, and robust implementations, thus extending the reach of WMO data sharing and lowering the barrier to access by Members.
Cloud-ready solutions
The cloud provides a reliable environment for data sharing and processing. It reduces the need for expensive local IT infrastructure, which constitutes a barrier to developing effective and reliable data processing workflows for some WMO Members. Cloud-ready solutions that can be used in the cloud or on premises have also been developed.
WIS 2.0 encourages WMO centres to adopt cloud technologies where appropriate to meet their users' needs. Whilst WMO technical regulations will not mandate the use of cloud services, WIS 2.0 will promote the adoption of cloud technologies that provide the most effective solution.
Cloud-ready infrastructure enables easy portability of technical solutions, ensuring that a system implemented by a specific country can be packaged and deployed easily in other countries with similar needs. In addition, using cloud technologies allows WIS 2.0 to deploy infrastructure and systems efficiently, with minimum effort for NMHSs, by shipping ready-made services and implementing consistent data processing and exchange techniques.
It should be clear that hosting data and services on the cloud does not affect data ownership. Even in a cloud environment, organisations retain ownership of their data, software, configuration, and change management as if they were hosting in their infrastructure. As a result, data authority and provenance stay with the organisation, and the cloud is simply a technical means to publish and make available the data.
WIS 2.0 development and implementation methodology
WIS 2.0 adopts an improved development and implementation approach compared to WIS. The lessons learned in the first implementation of WIS and its limited success in meeting the needs of the WMO community are taken into account. A collaborative implementation approach is adopted, enabling lower barriers and increased system participation by WMO Members and partner organisations.
The development and implementation of WIS 2.0 follow five phases: setting principles; demonstration projects; architecture and drafting of technical regulations; pilot phase; and operational implementation.
Setting principles
During the initial phase, the WMO Expert Team on WIS Evolution worked on setting the WIS 2.0 Principles underpinning the technical framework. The principles were established with the WIS 2.0 strategy of creating a collaborative system of systems using web architecture and open standards to provide simple, timely and seamless sharing of trusted weather, water and climate data and information through web services.
The WIS 2.0 principles comprise a set of technical and working practices intended to modernise access to promote the discoverability and accessibility of data and information while improving the efficiency of data exchange (see Box A).
A
The principles underpinning WIS 2.0 design
- WIS 2.0 adopts Web technology and leverages industry best practices and open standards.
- WIS 2.0 uses Uniform Resource Locators (URL) to identify resources (i.e., Web pages, data, metadata, APIs) use.
- WIS 2.0 prioritises the use of public telecommunications networks (i.e., Internet) when publishing digital resources.
- WIS 2.0 requires provision of Web service(s) to access or interact with digital resources (e.g., data, information, products) published using WIS.
- WIS 2.0 encourages NCs (National Centres) and DCPCs (Data Collection and Production Centres) to provide 'data reduction' services via WIS that process 'big data' to create results or products that are small enough to be conveniently downloaded and used by those with minimal technical infrastructure.
- WIS 2.0 adds open standard messaging protocols that use the publish-subscribe message pattern to the list of data exchange mechanisms approved for use within WIS and GTS.
- WIS 2.0 requires all services that provide real-time distribution of messages (containing data or notifications about data availability) to cache/store the messages for a minimum of 24-hours and allow users to request cached messages for download.
- WIS 2.0 adopts direct data exchange between provider and consumer and phases out the use of routing tables and bulletin headers.
- WIS 2.0 provides a catalogue containing metadata that describes both data and the service(s) provided to access that data.
- WIS 2.0 encourages data providers to publish metadata describing their data and Web services in a way that can be indexed by commercial search engines.
Demonstration projects
To ensure the completeness and soundness of WIS 2.0 principles and to provide material for the design of the architecture, several demonstration projects were established. They cover the areas of data exchange, data discovery, Least Developed Countries (LDC) and Small Island Developing States (SIDS), all Earth System domains and Services. The demonstration projects were established to validate and evolve the principles and provide a basis for the technical architecture. In addition, they highlighted the benefits for WMO Members and proved their effectiveness in fostering international cooperation for data sharing.
A workshop was held at the end of the demonstration projects phase, in 2021. The workshop concluded that the principles are sound and underpin data sharing at the national level and in several WMO communities. In addition, the need to develop a unifying architecture to allow interoperability between different systems was recognised and the necessity of having a reference implementation to test and validate the architecture was stated.
Architecture and drafting of technical regulations
The WMO Expert Team on WIS2 Architecture and Transition started an intense activity to finalise the technical architecture and draft the technical regulations for the Manual on the WMO Information System, which was approved in October 2022 by the Commission on Observations, Infrastructure and Information Systems (INFCOM) and finally by the WMO Congress in May 2023. In parallel, a project called ‘WIS2 in a box’ was initiated to provide a reference implementation to verify WIS 2.0 architecture and provide a low-cost solution for LDCs, SIDS and developing countries. ‘WIS2 in a box’ is Free and Open Source Software (FOSS) made available at https://docs.wis2box.wis.wmo.int. It is based on a cloud-ready solution, using open standards as required by WIS 2.0, and is made with Free and Open Source Software supported by active communities.
Pilot phase and operational implementation
The new WIS 2.0 data sharing framework can support the growing requirements in all WMO disciplines and domains associated with the WMO Unified Data Policy and the Global Basic Observing Network (GBON). The WIS 2.0 implementation plan was defined around the need to support these two WMO initiatives. The outcomes of the demonstration projects phase and the technical regulation drafting in conjunction with establishing the ‘WIS2‑in‑a‑box’ project allowed the WMO Standing Committee for Information Management and Technology (SC-IMT) to accelerate the implementation.
A transition strategy from the GTS to WIS 2.0 was developed by SC‑IMT to ensure that Centres migrating to WIS 2.0 can switch off their GTS reception and transmission equipment shortly after migrating without having to wait for the end of the migration.
The implementation has started with a one‑year pilot phase in 2023, followed by a pre-operational phase in 2024, which will lead to the operational phase beginning in 2025, when WMO Members will be required to migrate from GTS to WIS 2.0. The aim is to have 90% of Members migrate to WIS 2.0 by 2030 and to switch off all the GTS infrastructure by 2033.
The various components of WIS 2.0 architecture
Every WIS Centre that is part of WIS 2.0 will operate a ‘WIS2 Node’ to receive and to send data. ‘WIS2 in a box’, introduced above, provides a reference implementation of a WIS2 Node. It should be noted that using ‘WIS2 in a box’ is not mandatory and each WIS Centre will choose the solution they see fit. Some of the WIS2 Nodes that are part of the pilot phase are using this software, others are using a solution provided by commercial partners, and some are using solutions developed in‑house.
In order to provide a reliable, efficient service for all WIS users, the following Global Services have been defined:
- Global Broker: Centres will be responsible to make sure that all messages announcing the availability of new data and metadata can be easily obtained by all users. The Global Broker will provide a subscription service using the MQTT (Message Queuing Telemetry Transport) standard and a Free and Open Source Software solution with additional companion software (specific to WIS 2.0) to ensure uniqueness of messages as well as verifying the correct format of those messages.
- Global Cache: In order to provide quick and reliable access to core data as defined by the WMO Unified Data Policy, a copy of this data will be made available by a Global Cache. Storing data from originating WIS2 Nodes, the Global Cache will then make available the core data to all WIS Users.
- Global Discovery Catalogue: Each dataset available on WIS 2.0 must be described by a metadata record, using the OGC API - Records standard (soon to be ratified). The Global Discovery Catalogue will provide a discovery and metadata service using Free and Open Source Software, and it will provide quality assessment capabilities in support of continuous improvement of WIS 2.0 metadata.
- Global Monitoring: WIS 2.0 being an operational solution, it must be monitored. Each WIS2 Node and Global Service will provide metrics relevant to their operations. Global Monitoring Centres will collect the metrics and make available a visual dashboard presenting those metrics and alert the Centres when an unexpected event occurs in support of corrective action.
At the time of writing (May 2023), the following NMHSs are operating the Global Services as part of the Pilot phase (see Figure 1):
- Chinese Meteorological Agency and Météo-France (Global Brokers)
- Japan Meteorological Agency and Deutscher Wetterdienst (Global Cache)
- Environment and Climate Change Canada (Global Discovery Catalogue)
- Maroc‑Météo (Global Monitoring)
Currently the Global Broker operated by Météo-France and a WIS2 Node are being implemented in the European Weather Cloud, a federated cloud computing infrastructure set up by ECMWF and EUMETSAT.
It is planned that additional Global Services (either operated by NMHSs or the private sector) will be made available by the end of 2023 in order to ensure sufficient redundancy of the architecture at the onset of the next phase of the implementation project, the pre‑operational phase, starting in 2024.
Conclusion
After more than 50 years of operations of the GTS, and considering the data exchange challenges ahead, it is time to decommission this service and open the WIS 2.0 era. Thanks to the technical choices made based on Open Standards, the availability of a reference implementation (‘WIS2 in a box’), and the commitment of experts from many NMHSs, the plan is for most NMHSs to complete the migration from GTS to WIS 2.0 in five years (2025–2030). During that period, all WMO Members and affiliated organisations, such as ECMWF, should aim to implement this new Information System.