sundials-devs-python-callbacks

name

sundials-devs-python-callbacks

description

Generate nanobind C++ wrapper code for user-supplied callback functions in the sundials4py Python bindings. Use this skill whenever adding a new user-supplied function (callback) to any sundials4py submodule (arkode, cvodes, idas, kinsol, core/sundials), wrapping a C function-pointer-based API for Python, adding a `python` member to a SUNDIALS C struct that does not yet have one, or when a function cannot be autogenerated by litgen because it takes a function pointer argument. Also trigger when the user mentions "callback wrapper", "user_supplied_fn_caller", "BIND_ARKODE_CALLBACK", "BIND_CVODE_CALLBACK", "BIND_IDA_CALLBACK", "BIND_KINSOL_CALLBACK", or references any *_usersupplied.hpp or *_impl.h file in the context of Python bindings. If the user says something like "wrap this Set callback for Python", "add a new user function to cvodes", or "this object needs a python member for callbacks", this is the skill to use.

sundials4py User-Supplied Function Wrapper Skill

This skill guides you through creating the nanobind C++ wrapper code needed to expose a SUNDIALS C callback function to Python. These wrappers cannot be autogenerated by litgen because they accept function pointers, so they must be hand-coded following the patterns described below.

Why these wrappers exist

nanobind cannot directly bind C functions that accept raw function pointers. The solution is a multi-layer indirection pattern:

0. python member on the C struct - a void* field on the SUNDIALS object that holds a pointer to the function table. Some objects (e.g. SUNStepper) already have this; others may need it added first.
1. Function table - a struct of nb::object members, one per callback, stored either in the integrator's user_data or in the object's python member.
2. Wrapper function - a C-linkage function matching the original C typedef signature that retrieves the Python callable from the function table and invokes it.
3. nanobind m.def - a lambda registered with nanobind that accepts a std::function, stores it in the function table, and passes the wrapper function to the underlying C "Set" API.

Every new user-supplied function requires additions to layers 1-3. If the target object does not yet have a python member, layer 0 must be addressed first.

Step-by-step process

Note on examples: The examples below use ARKODE and CVODES to illustrate the patterns, but the same structure applies identically to all packages (ARKODE, CVODES, IDAS, KINSOL). Each has its own function table struct, wrapper functions, convenience macro, and get_*_fn_table helper. When working in a different package, substitute the corresponding names (e.g. ida_user_supplied_fn_table, get_ida_fn_table, BIND_IDA_CALLBACK, etc.).

When the user asks you to wrap a new callback, gather these inputs first:

• Module: which submodule (arkode, cvodes, idas, kinsol, or a core module like sundials).
• C "Set" function name: e.g. ARKodeSetPostprocessStepFn.
• C function-pointer typedef: e.g. ARKPostProcessFn.
• Parameter list of the typedef: needed to detect "special" parameters (see below).
• Storage mechanism: whether the function table is accessed via user_data (like ARKODE/CVODE/IDA/KINSOL) or via a python member on the struct (standalone objects like SUNStepper, SUNAdaptController, etc.).

Then produce code fragments as described in the following sections. If the target object does not yet have a python member, start with Step 0. Otherwise, skip to Step 1.

0. Add a python member to the C struct (if it does not already have one)

Some SUNDIALS objects (e.g. SUNStepper) already carry a void* python member for the function table. Others may not. If the struct you are targeting has no python member, you must add one before any binding work can proceed. This involves changes to three C-side files.

Use SUNStepper as the reference implementation for the pattern. All of the relevant code can be found in include/sundials/sundials_stepper.h, src/sundials/sundials_stepper_impl.h, and src/sundials/sundials_stepper.c.

0a. Add the field to the private implementation struct

Open the *_impl.h header that defines the struct internals (e.g. src/sundials/sundials_logger_impl.h). Add a void* python; field. Place it near the content field if one exists, since they serve analogous purposes (content is for user-supplied C data, python is for the binding-layer function table).

Before (SUNLogger example):

struct SUNLogger_
{
  /* ... existing fields ... */
  void* content;

  SUNLoggerQueueMsgFn queue_msg;
  SUNLoggerFlushMsgFn flush_msg;
};

After:

struct SUNLogger_
{
  /* ... existing fields ... */
  void* content;

  /* Python binding function table (managed by sundials4py) */
  void* python;

  SUNLoggerQueueMsgFn queue_msg;
  SUNLoggerFlushMsgFn flush_msg;
};

0b. Initialize python to NULL in the Create function

Open the .c source file containing the object's Create function. Initialize the new field to NULL alongside the other field initializations.

Also add a forward declaration for the destroy helper, guarded by #if defined(SUNDIALS_ENABLE_PYTHON). This function will be defined on the C++ binding side (see Step 0c) and is called during object destruction to prevent leaks.

Example - modifying SUNLogger_Create in src/sundials/sundials_logger.c:

/* Forward declaration of function used to destroy any data allocated for Python */
#if defined(SUNDIALS_ENABLE_PYTHON)
void SUNLoggerFunctionTable_Destroy(void* ptr);
#endif

SUNErrCode SUNLogger_Create(SUNComm comm, int output_rank, SUNLogger* logger_ptr)
{
  /* ... existing allocation code ... */

  logger->content = NULL;
  logger->python  = NULL;   /* <-- add this */

  /* ... rest of initialization ... */
}

The naming convention for the forward-declared destroy function is <ObjectName>FunctionTable_Destroy. Follow what SUNStepper uses: SUNStepperFunctionTable_Destroy.

0c. Clean up python in the Destroy function

In the same .c file, find the object's Destroy function. Add a guarded block that calls the destroy helper and nulls out the pointer. This must happen before the free() of the object itself, but after any user-level destroy ops have run (so the user callback is not called after its function table is freed).

Example - modifying SUNLogger_Destroy in src/sundials/sundials_logger.c:

SUNErrCode SUNLogger_Destroy(SUNLogger* logger_ptr)
{
  /* ... existing checks and cleanup ... */

  SUNLogger logger = *logger_ptr;
  if (logger)
  {
#if SUNDIALS_LOGGING_LEVEL > 0
    if (sunLoggerIsOutputRank(logger, NULL))
    {
      SUNHashMap_Destroy(&logger->filenames);
    }
#endif

#if defined(SUNDIALS_ENABLE_PYTHON)
    SUNLoggerFunctionTable_Destroy(logger->python);
#endif
    logger->python = NULL;

#if SUNDIALS_MPI_ENABLED
    if (logger->comm != SUN_COMM_NULL) { MPI_Comm_free(&logger->comm); }
#endif

    free(logger);
    *logger_ptr = NULL;
  }

  return retval;
}

0d. Define the FunctionTable_Destroy function on the binding side

The forward-declared *FunctionTable_Destroy function must be defined in a C++ source file in the bindings. Typically this goes in the same .cpp file where the nanobind m.def registrations live, or in a dedicated helpers file for that module. It casts the void* to the function table struct and deletes it.

Example - in bindings/sundials4py/sundials/sundials_logger.cpp:

// Must be extern "C" since the forward declaration in the C source is in a C file
extern "C" void SUNLoggerFunctionTable_Destroy(void* ptr)
{
  delete static_cast<SUNLoggerFunctionTable*>(ptr);
}

For reference, SUNStepper follows this exact pattern - SUNStepperFunctionTable_Destroy is forward-declared in sundials_stepper.c and defined in the binding-side C++ code.

1. Add a member to the function table struct

Open the appropriate *_usersupplied.hpp header (e.g. bindings/sundials4py/arkode/arkode_usersupplied.hpp). Add an nb::object member to the function table struct. The member name should be a lowercase, underscore-separated version of the callback's short name.

File: bindings/sundials4py/<module>/<module>_usersupplied.hpp

struct <module>_user_supplied_fn_table
{
   // ... existing members ...
   nb::object <member_name>;  // <-- add this
};

Example - adding postprocessstepfn to ARKODE:

struct arkode_user_supplied_fn_table
{
   // ... existing members ...
   nb::object postprocessstepfn;
};

2. Write the wrapper function

Immediately below the struct (in the same *_usersupplied.hpp file), add a wrapper function template. The wrapper delegates to sundials4py::user_supplied_fn_caller.

Standard callbacks (no special parameters)

For callbacks whose parameters map 1:1 between C and C++, use the generic template form:

template<typename... Args>
inline <return_type> <module>_<member_name>_wrapper(Args... args)
{
  return sundials4py::user_supplied_fn_caller<
    std::remove_pointer_t<CTypeDef>,        // the C function-pointer typedef, de-pointered
    <FunctionTableStruct>,                  // the function table struct type
    <MemStruct>,                            // integrator memory struct (e.g., ARKodeMem) or object type (e.g., SUNStepper)
    <user_data_offset>                      // (optional) positional offset of user_data arg; omit or set to 0 for `python`-member objects
  >(&<FunctionTableStruct>::<member_name>, args...);
}

Key parameters of user_supplied_fn_caller:

Template param	Meaning
1st	The C typedef with pointer removed (std::remove_pointer_t<...>)
2nd	The function table struct type
3rd	The integrator memory struct type or SUNDIALS object type
4th (optional)	Integer indicating which argument (0-indexed from the end) is the user_data pointer. Only needed for integrator callbacks; omit for python-member objects. Check the C typedef to find the position.

Example - standard ARKODE callback via user_data:

template<typename... Args>
inline int arkode_postprocessstepfn_wrapper(Args... args)
{
  return sundials4py::user_supplied_fn_caller<
    std::remove_pointer_t<ARKPostProcessFn>, arkode_user_supplied_fn_table,
    ARKodeMem, 1>(&arkode_user_supplied_fn_table::postprocessstepfn, args...);
}

Example - SUNStepper callback via python member (no offset needed):

template<typename... Args>
inline SUNErrCode sunstepper_evolve_wrapper(Args... args)
{
  return sundials4py::user_supplied_fn_caller<
    std::remove_pointer_t<SUNStepperEvolveFn>, SUNStepperFunctionTable,
    SUNStepper>(&SUNStepperFunctionTable::evolve, std::forward<Args>(args)...);
}

Special callbacks (array or output-pointer parameters)

Some C typedefs have parameters that need translation for Python:

C parameter pattern	C++ / Python analog
N_Vector* used as a 1D array of N_Vectors	std::vector<N_Vector>
Output pointer (e.g., sunbooleantype* used to return a value)	Pack into std::tuple return
sunrealtype* used as an array	std::vector<sunrealtype> or pointer-to-first with length

When any of these appear, you cannot use the generic variadic template. Instead:

1. Define a custom std::function type alias that mirrors the C typedef but replaces the special parameters with their C++ analogs. Output parameters become additional elements in a std::tuple return type.

2. Write a fully-explicit wrapper function (not a variadic template) that:

   • Casts user_data / accesses the python member to get the function table.
   • Extracts the Python callable and casts it to the custom std::function type.
   • Converts C arrays to std::vector before calling.
   • Unpacks std::tuple return values back into the output pointers after calling.

Example - CVODES CVLsPrecSetupFnBS with an N_Vector array and an output pointer:

// Custom std::function type: replaces N_Vector* array with std::vector,
// and moves the jcurPtrB output pointer into a tuple return value.
using CVLsPrecSetupStdFnBS = std::tuple<int, sunbooleantype>(
  sunrealtype t, N_Vector y, std::vector<N_Vector> yS_1d, N_Vector yB,
  N_Vector fyB, sunbooleantype jokB, sunrealtype gammaB, void* user_dataB);

inline int cvode_lsprecsetupfnBS_wrapper(sunrealtype t, N_Vector y,
                                         N_Vector* yS_1d, N_Vector yB,