Searching...

No Matches

For a module reference see yac.

This package provides Python bindings for the YAC coupler. They are generated using cython and can be enabled with the --enable-python-bindings configure flag. This enables the configure checks, the compilation of the Python extension module (.so file) and the corresponding tests. Hard dependencies for the Python bindings are the packages

For full functionality and the test suite, we recommend using

The function Component.comp_comm and get_comps_comm require the mpi4py module, which must be compiled with the same MPI version as YAC for proper functionality. To convert ISO8601 date/time strings returned by YAC functions, such as YAC.start_datetime, we recommend the isodate library (https://github.com/gweis/isodate/) or the Python bindings of mtime (https://gitlab.dkrz.de/icon-libraries/libmtime). All packages (except mtime) can be installed using pip.

To use the Python bindings in a Python program, add the path containing the Python extension module file to the PYTHONPATH environment variable. To install yac into a virtual environment set the --prefix during configuration (./configure) to the directory of the venv and execute make install. E.g.\ to setup a fresh venv, install all Python dependencies and install yac you can use the following snippet:

#setup venv
python3 -m venv <path_to_your_venv>  # user-defined path where the venv will be created
source <path_to_your_venv>/bin/activate
pip install setuptools wheel uxarray scipy matplotlib netcdf4 cython numpy
 
# build and install yac
git clone git@gitlab.dkrz.de:YAC/YAC-dev.git
cd YAC-dev/
./autogen.sh
mkdir build
cd build
../configure CC=mpicc FC=mpif90 --prefix=<path_to_your_venv> --enable-python-bindings \
  --with-yaxt-root=<yaxt_inst_dir>
make
make install

The bindings are designed to build a thin layer between Python and YAC. I.e. methods are directly forwarded to the corresponding YAC functions with a few exceptions:

For all data structures for that YAC returns an id, Python creates an object that holds this id and provides member functions that use this id.
Whenever a c-pointer to memory is expected by the YAC function, the argument of the Python function is converted to a contiguous numpy array using numpy.ascontiguousarray (involving a copy if it is not already contiguous) and converted to a cython memory view to extract the size and pointer.
MPI communicators are converted into mpi4py.MPI.Comm and vice-versa.

Coroutines

The Python interface provides the coroutines yac::Field::get_coro and yac::Field::put_coro that can be used to write asynchronous code. This is in particular useful if multiple fields with different timesteps need to be handled. The following example shows how to use the coroutines assuming a source field field_source and target field field_target are defined.

import asyncio
 
async def recv_loop(field):
    while True:
        buf, info = await field.get_coro()
        print(f"recv (field_id={field.field_id}):", buf, info)
        if info == yac.Action.GET_FOR_RESTART:
           break
 
 
async def send_loop(field):
    while True:
        info = await field.put_coro(np.random.rand(4))
        print(f"send (field_id={field.field_id}):", info)
        if info == yac.Action.PUT_FOR_RESTART:
            break
 
 
# asyncio entry point
async def main():
    # create task group
    await asyncio.gather(
        send_loop(field_source),
        recv_loop(field_target),
    )
 
# entry point
asyncio.run(main())

Note that the code does not need to know anything about the timestepping. Both fields are processed independendly from each other. Effectively the event loop of ayncio spinns between the two coroutines and proceeds where possible.

Note: The Python binding coroutines do not depend on asyncio and can also be used with other coroutine frameworks.

Logging

The Python logging package has been integrated into the Python bindings. Most log records are generated at the DEBUG level. Please note that a limitation in cython results in inaccurate stack information passed to the logger (specifically, incorrect filenames and module names).

To enable debug messages, use the following code:

import logging
 
logging.basicConfig(level=logging.WARNING, format=f'%(name)s rank={rank} - %(levelname)s - %(message)s')
logging.getLogger("yac").setLevel(logging.DEBUG)

This will print warnings from all packages and all logging messages from the yac module.

Using mpi4py with yac

Installing mpi4py can be challenging, so we recommend against installing it when using YAC. However, if you require manual MPI communication, you can utilize mpi4py. To do this, it’s crucial to build mpi4py using the same MPI implementation as YAC. Within a virtual environment (venv), this can be accomplished by configuring the necessary compilers (e.g., loading a module or spack package) and then installing mpi4py with pip. For example, on Levante:

spack load openmpi/mnm

pip install --no-binary --no-cache-dir mpi4py

The --no-binary flag instructs pip to build the package from source.

OpenMPI 5 with yac

OpenMPI version 5 requires that libmpi.so be loaded with the RTLD_GLOBAL flag when loaded dynamically using dlopen (as is the case when using Python). Currently, YAC does not handle this scenario, which can cause issues such as segmentation faults. The simplest solution is to use mpi4py (built against the compatible OpenMPI version) and import it before YAC. mpi4py automatically manages this requirement, ensuring that libmpi.so is properly loaded for YAC. We are actively working on a more robust solution for this situation.

Python Examples

In the examples directory a framework of classes can be found that act as model component. To start a configuration with different components the driver.py can be used. It allows sequential coupling as well as parallel coupling.

The framework contains the following example components:

Table of Contents

Coroutines

Logging

Using mpi4py with yac

OpenMPI 5 with yac

Python Examples