esmvalcore.io.protocol#

Protocols for accessing data.

An input data source can be defined in the configuration by using esmvalcore.config.CFG

>>> from esmvalcore.config import CFG
>>> CFG["projects"]["example-project"]["data"]["example-source-name"] = {
        "type": "example_module.ExampleDataSource"
        "argument1": "value1"
        "argument2": "value2"
    }

or as a YAML configuration file

projects:
  example-project:
    data:
      example-source-name
        type: example_module.ExampleDataSource
        argument1: value1
        argument2: value2

where example-project is a project, e.g. CMIP6, and example-source-name is a unique name describing the data source. The datasource type, in the example above called example_module.ExampleDataSource needs to implement the esmvalcore.io.protocol.DataSource protocol. Any remaining key-value pairs in the configuration, argument1: value1 and argument2: value2 are passed as keyword arguments to the data source when it is created.

Dedeplication of search results happens based on the esmvalcore.io.protocol.DataElement.name attribute and the "version" facet in esmvalcore.io.protocol.DataElement.facets of the data elements provided by the data sources. If there is a tie, the data element provided by the data source with the lowest value of esmvalcore.io.protocol.DataSource.priority is chosen.

Classes:

DataElement(*args, **kwargs)

A data element represents some data that can be loaded.

DataSource(*args, **kwargs)

A data source can be used to find data.
class esmvalcore.io.protocol.DataElement(*args, **kwargs)[source]#

Bases: Protocol

A data element represents some data that can be loaded.

A file is an example of a data element.

Attributes:

attributes

Attributes are key-value pairs describing the data.

facets

Facets are key-value pairs that can be used for searching the data.

name

A unique name identifying the data.

Methods:

prepare()

Prepare the data for access.

to_iris([ignore_warnings])

Load the data as Iris cubes.
attributes: dict[str, Any]#

Attributes are key-value pairs describing the data.

facets: dict[str, str | Sequence[str] | int]#

Facets are key-value pairs that can be used for searching the data.

name: str#

A unique name identifying the data.

prepare() → None[source]#

Prepare the data for access.

Return type:

None

to_iris(ignore_warnings: list[dict[str, Any]] | None = None) → CubeList[source]#

Load the data as Iris cubes.

Parameters:

ignore_warnings (list[dict[str, Any]] | None) – Keyword arguments passed to warnings.filterwarnings() used to ignore warnings issued by iris.load_raw(). Each list element corresponds to one call to warnings.filterwarnings().

Returns:

The loaded data.

Return type:

iris.cube.CubeList

class esmvalcore.io.protocol.DataSource(*args, **kwargs)[source]#

Bases: Protocol

A data source can be used to find data.

Attributes:

debug_info

A string containing debug information when no data is found.

name

A name identifying the data source.

priority

The priority of the data source.

project

The project that the data source provides data for.

Methods:

find_data(**facets)

Find data.
debug_info: str#

A string containing debug information when no data is found.

find_data(**facets: str | Sequence[str] | int) → Iterable[DataElement][source]#

Find data.

Parameters:

**facets (str | Sequence[str] | int) – Find data matching these facets.

Returns:

The data elements that have been found.

Return type:

typing.Iterable of esmvalcore.io.base.DataElement

name: str#

A name identifying the data source.

priority: int#

The priority of the data source. Lower values have priority.

project: str#

The project that the data source provides data for.