• English 
  • Spanish 

Large-scale and Data-driven Load Balancing Strategies for the THREDDS Data Server

The THREDDS project is developing middleware to bridge the gap between data providers and data users. The goal is to simplify the discovery and use of scientific data and to allow scientific publications and educational materials to reference scientific data. The THREDDS Data Server (TDS) lacks horizontal scalability and automatic configuration management and deployment, making frequently downtimes and time consuming configuration tasks, being a limiting factor when an intensive use is done as is usual within the scientific community (e.g. climate). Instead of the regular installation and configuration of a single or multiple independent TDS, manually configured, this work presents an automatic provisioning, deployment and orchestration of a cluster of TDS instances. The framework presented allows to control the deployment and configuration setup on a infrastructure and to manage the datasets available in TDS instances. The framework allows to configure different backends and frontends load balancing setups and solutions. This implementation allows to define different infrastructure and deployment scenario setups, and more TDS instances are easily added to the cluster by simply declaring them and updating the configuration dynamically and with zero downtime service design. In this work different load balancing strategies are being evaluated, considering different key performance indicators to characterise throughput, scalability, availability and reliability.

  • Institutions:
  • Tags: