sundials-devs-python-callbacks

name

sundials-devs-python-callbacks

description

Generate nanobind C++ wrapper code for user-supplied callback functions in the sundials4py Python bindings. Use this skill whenever adding a new user-supplied function (callback) to any sundials4py submodule (arkode, cvodes, idas, kinsol, core/sundials), wrapping a C function-pointer-based API for Python, adding a `python` member to a SUNDIALS C struct that does not yet have one, or when a function cannot be autogenerated by litgen because it takes a function pointer argument. Also trigger when the user mentions "callback wrapper", "user_supplied_fn_caller", "BIND_ARKODE_CALLBACK", "BIND_CVODE_CALLBACK", "BIND_IDA_CALLBACK", "BIND_KINSOL_CALLBACK", or references any *_usersupplied.hpp or *_impl.h file in the context of Python bindings. If the user says something like "wrap this Set callback for Python", "add a new user function to cvodes", or "this object needs a python member for callbacks", this is the skill to use.

sundials4py User-Supplied Function Wrapper Skill

This skill guides you through creating the nanobind C++ wrapper code needed to expose a SUNDIALS C callback function to Python. These wrappers cannot be autogenerated by litgen because they accept function pointers, so they must be hand-coded following the patterns described below.

Why these wrappers exist

nanobind cannot directly bind C functions that accept raw function pointers. The solution is a multi-layer indirection pattern:

0. python member on the C struct - a void* field on the SUNDIALS object that holds a pointer to the function table. Some objects (e.g. SUNStepper) already have this; others may need it added first.
1. Function table - a struct of nb::object members, one per callback, stored either in the integrator's user_data or in the object's python member.
2. Wrapper function - a C-linkage function matching the original C typedef signature that retrieves the Python callable from the function table and invokes it.
3. nanobind m.def - a lambda registered with nanobind that accepts a std::function, stores it in the function table, and passes the wrapper function to the underlying C "Set" API.

Every new user-supplied function requires additions to layers 1-3. If the target object does not yet have a python member, layer 0 must be addressed first.

Step-by-step process

Note on examples: The examples below use ARKODE and CVODES to illustrate the patterns, but the same structure applies identically to all packages (ARKODE, CVODES, IDAS, KINSOL). Each has its own function table struct, wrapper functions, convenience macro, and get_*_fn_table helper. When working in a different package, substitute the corresponding names (e.g. ida_user_supplied_fn_table, get_ida_fn_table, BIND_IDA_CALLBACK, etc.).

When the user asks you to wrap a new callback, gather these inputs first:

• Module: which submodule (arkode, cvodes, idas, kinsol, or a core module like sundials).
• C "Set" function name: e.g. ARKodeSetPostprocessStepFn.
• C function-pointer typedef: e.g. ARKPostProcessFn.
• Parameter list of the typedef: needed to detect "special" parameters (see below).
• Storage mechanism: whether the function table is accessed via user_data (like ARKODE/CVODE/IDA/KINSOL) or via a python member on the struct (standalone objects like SUNStepper, SUNAdaptController, etc.).

Then produce code fragments as described in the following sections. If the target object does not yet have a python member, start with Step 0. Otherwise, skip to Step 1.

0. Add a python member to the C struct (if it does not already have one)

Some SUNDIALS objects (e.g. SUNStepper) already carry a void* python member for the function table. Others may not. If the struct you are targeting has no python member, you must add one before any binding work can proceed. This involves changes to three C-side files.

Use SUNStepper as the reference implementation for the pattern. All of the relevant code can be found in include/sundials/sundials_stepper.h, src/sundials/sundials_stepper_impl.h, and src/sundials/sundials_stepper.c.

0a. Add the field to the private implementation struct

Open the *_impl.h header that defines the struct internals (e.g. src/sundials/sundials_logger_impl.h). Add a void* python; field. Place it near the content field if one exists, since they serve analogous purposes (content is for user-supplied C data, python is for the binding-layer function table).

Before (SUNLogger example):

struct SUNLogger_
{
  /* ... existing fields ... */
  void* content;

  SUNLoggerQueueMsgFn queue_msg;
  SUNLoggerFlushMsgFn flush_msg;
};

After:

struct SUNLogger_
{
  /* ... existing fields ... */
  void* content;

  /* Python binding function table (managed by sundials4py) */
  void* python;

  SUNLoggerQueueMsgFn queue_msg;
  SUNLoggerFlushMsgFn flush_msg;
};

0b. Initialize python to NULL in the Create function

Open the .c source file containing the object's Create function. Initialize the new field to NULL alongside the other field initializations.

Also add a forward declaration for the destroy helper, guarded by #if defined(SUNDIALS_ENABLE_PYTHON). This function will be defined on the C++ binding side (see Step 0c) and is called during object destruction to prevent leaks.

Example - modifying SUNLogger_Create in src/sundials/sundials_logger.c:

/* Forward declaration of function used to destroy any data allocated for Python */
#if defined(SUNDIALS_ENABLE_PYTHON)
void SUNLoggerFunctionTable_Destroy(void* ptr);
#endif

SUNErrCode SUNLogger_Create(SUNComm comm, int output_rank, SUNLogger* logger_ptr)
{
  /* ... existing allocation code ... */

  logger->content = NULL;
  logger->python  = NULL;   /* <-- add this */

  /* ... rest of initialization ... */
}

The naming convention for the forward-declared destroy function is <ObjectName>FunctionTable_Destroy. Follow what SUNStepper uses: SUNStepperFunctionTable_Destroy.

0c. Clean up python in the Destroy function

In the same .c file, find the object's Destroy function. Add a guarded block that calls the destroy helper and nulls out the pointer. This must happen before the free() of the object itself, but after any user-level destroy ops have run (so the user callback is not called after its function table is freed).

Example - modifying SUNLogger_Destroy in src/sundials/sundials_logger.c:

SUNErrCode SUNLogger_Destroy(SUNLogger* logger_ptr)
{
  /* ... existing checks and cleanup ... */

  SUNLogger logger = *logger_ptr;
  if (logger)
  {
#if SUNDIALS_LOGGING_LEVEL > 0
    if (sunLoggerIsOutputRank(logger, NULL))
    {
      SUNHashMap_Destroy(&logger->filenames);
    }
#endif

#if defined(SUNDIALS_ENABLE_PYTHON)
    SUNLoggerFunctionTable_Destroy(logger->python);
#endif
    logger->python = NULL;

#if SUNDIALS_MPI_ENABLED
    if (logger->comm != SUN_COMM_NULL) { MPI_Comm_free(&logger->comm); }
#endif

    free(logger);
    *logger_ptr = NULL;
  }

  return retval;
}

0d. Define the FunctionTable_Destroy function on the binding side

The forward-declared *FunctionTable_Destroy function must be defined in a C++ source file in the bindings. Typically this goes in the same .cpp file where the nanobind m.def registrations live, or in a dedicated helpers file for that module. It casts the void* to the function table struct and deletes it.

Example - in bindings/sundials4py/sundials/sundials_logger.cpp:

// Must be extern "C" since the forward declaration in the C source is in a C file
extern "C" void SUNLoggerFunctionTable_Destroy(void* ptr)
{
  delete static_cast<SUNLoggerFunctionTable*>(ptr);
}

For reference, SUNStepper follows this exact pattern - SUNStepperFunctionTable_Destroy is forward-declared in sundials_stepper.c and defined in the binding-side C++ code.

1. Add a member to the function table struct

Open the appropriate *_usersupplied.hpp header (e.g. bindings/sundials4py/arkode/arkode_usersupplied.hpp). Add an nb::object member to the function table struct. The member name should be a lowercase, underscore-separated version of the callback's short name.

File: bindings/sundials4py/<module>/<module>_usersupplied.hpp

struct <module>_user_supplied_fn_table
{
   // ... existing members ...
   nb::object <member_name>;  // <-- add this
};

Example - adding postprocessstepfn to ARKODE:

struct arkode_user_supplied_fn_table
{
   // ... existing members ...
   nb::object postprocessstepfn;
};

2. Write the wrapper function

Immediately below the struct (in the same *_usersupplied.hpp file), add a wrapper function template. The wrapper delegates to sundials4py::user_supplied_fn_caller.

Standard callbacks (no special parameters)

For callbacks whose parameters map 1:1 between C and C++, use the generic template form:

template<typename... Args>
inline <return_type> <module>_<member_name>_wrapper(Args... args)
{
  return sundials4py::user_supplied_fn_caller<
    std::remove_pointer_t<CTypeDef>,        // the C function-pointer typedef, de-pointered
    <FunctionTableStruct>,                  // the function table struct type
    <MemStruct>,                            // integrator memory struct (e.g., ARKodeMem) or object type (e.g., SUNStepper)
    <user_data_offset>                      // (optional) positional offset of user_data arg; omit or set to 0 for `python`-member objects
  >(&<FunctionTableStruct>::<member_name>, args...);
}

Key parameters of user_supplied_fn_caller:

Template param	Meaning
1st	The C typedef with pointer removed (std::remove_pointer_t<...>)
2nd	The function table struct type
3rd	The integrator memory struct type or SUNDIALS object type
4th (optional)	Integer indicating which argument (0-indexed from the end) is the user_data pointer. Only needed for integrator callbacks; omit for python-member objects. Check the C typedef to find the position.

Example - standard ARKODE callback via user_data:

template<typename... Args>
inline int arkode_postprocessstepfn_wrapper(Args... args)
{
  return sundials4py::user_supplied_fn_caller<
    std::remove_pointer_t<ARKPostProcessFn>, arkode_user_supplied_fn_table,
    ARKodeMem, 1>(&arkode_user_supplied_fn_table::postprocessstepfn, args...);
}

Example - SUNStepper callback via python member (no offset needed):

template<typename... Args>
inline SUNErrCode sunstepper_evolve_wrapper(Args... args)
{
  return sundials4py::user_supplied_fn_caller<
    std::remove_pointer_t<SUNStepperEvolveFn>, SUNStepperFunctionTable,
    SUNStepper>(&SUNStepperFunctionTable::evolve, std::forward<Args>(args)...);
}

Special callbacks (array or output-pointer parameters)

Some C typedefs have parameters that need translation for Python:

C parameter pattern	C++ / Python analog
N_Vector* used as a 1D array of N_Vectors	std::vector<N_Vector>
Output pointer (e.g., sunbooleantype* used to return a value)	Pack into std::tuple return
sunrealtype* used as an array	std::vector<sunrealtype> or pointer-to-first with length

When any of these appear, you cannot use the generic variadic template. Instead:

1. Define a custom std::function type alias that mirrors the C typedef but replaces the special parameters with their C++ analogs. Output parameters become additional elements in a std::tuple return type.

2. Write a fully-explicit wrapper function (not a variadic template) that:

   • Casts user_data / accesses the python member to get the function table.
   • Extracts the Python callable and casts it to the custom std::function type.
   • Converts C arrays to std::vector before calling.
   • Unpacks std::tuple return values back into the output pointers after calling.

Example - CVODES CVLsPrecSetupFnBS with an N_Vector array and an output pointer:

// Custom std::function type: replaces N_Vector* array with std::vector,
// and moves the jcurPtrB output pointer into a tuple return value.
using CVLsPrecSetupStdFnBS = std::tuple<int, sunbooleantype>(
  sunrealtype t, N_Vector y, std::vector<N_Vector> yS_1d, N_Vector yB,
  N_Vector fyB, sunbooleantype jokB, sunrealtype gammaB, void* user_dataB);

inline int cvode_lsprecsetupfnBS_wrapper(sunrealtype t, N_Vector y,
                                         N_Vector* yS_1d, N_Vector yB,
                                         N_Vector fyB, sunbooleantype jokB,
                                         sunbooleantype* jcurPtrB,
                                         sunrealtype gammaB, void* user_dataB)
{
  auto cv_mem   = static_cast<CVodeMem>(user_dataB);
  auto fn_table = get_cvode_fn_table(user_dataB);
  auto fn =
    nb::cast<std::function<CVLsPrecSetupStdFnBS>>(fn_table->lsprecsetupfnBS);
  auto Ns = cv_mem->cv_Ns;

  // Convert C array to std::vector
  std::vector<N_Vector> yS(yS_1d, yS_1d + Ns);

  auto result = fn(t, y, yS, yB, fyB, jokB, gammaB, nullptr);

  // Unpack output parameter from tuple
  *jcurPtrB = std::get<1>(result);

  return std::get<0>(result);
}

3. Register the nanobind binding in the .cpp file

Open the corresponding .cpp file (e.g. bindings/sundials4py/arkode/arkode.cpp for ARKODE, bindings/sundials4py/cvodes/cvodes.cpp for CVODES, etc.). Add a m.def (or use the package's BIND_<PACKAGE>_CALLBACK convenience macro for standard integrator callbacks) that:

• Accepts the integrator/object handle and a std::function wrapping the de-pointered C typedef.
• Retrieves (or lazily creates) the function table.
• Stores the Python callable in the appropriate member via nb::cast(fn).
• Calls the real C "Set" function with either the wrapper or nullptr.
• Marks the function argument as .none() so Python users can pass None to unset it.

Using the integrator convenience macro

Each integrator package defines a convenience macro for standard callbacks (no special params): BIND_ARKODE_CALLBACK, BIND_CVODE_CALLBACK, BIND_IDA_CALLBACK, and BIND_KINSOL_CALLBACK. These macros all follow the same pattern - they expand to a m.def lambda that stores the callable in the function table and passes the wrapper to the C "Set" function. Use the macro for the package you are working in:

BIND_<PACKAGE>_CALLBACK(<SetFunctionName>, <CTypeDef>,
                        <member_name>, <wrapper_function_name>,
                        nb::arg("<mem_param_name>"), nb::arg("<python_param_name>").none());

Example (ARKODE):

BIND_ARKODE_CALLBACK(ARKodeSetPostprocessStepFn, ARKPostProcessFn,
                     postprocessstepfn, arkode_postprocessstepfn_wrapper,
                     nb::arg("arkode_mem"), nb::arg("postprocessstep").none());

Manual m.def (standalone objects, or special callbacks)

For standalone objects that use the python member (e.g. SUNStepper), for special callbacks with array/output-pointer parameters, or in any case where the convenience macro does not apply, write the m.def explicitly.

Pattern for integrator callbacks (user_data storage):

m.def(
  "<SetFunctionName>",
  [](void* mem, std::function<std::remove_pointer_t<CTypeDef>> fn)
  {
    auto fn_table = get_<module>_fn_table(mem);
    fn_table-><member_name> = nb::cast(fn);
    if (fn) { return <SetFunctionName>(mem, <wrapper_function_name>); }
    else    { return <SetFunctionName>(mem, nullptr); }
  },
  nb::arg("<mem_param_name>"), nb::arg("<fn_param_name>").none());

Pattern for python-member callbacks (e.g. SUNStepper):

m.def(
  "<SetFunctionName>",
  [](ObjectType obj,
     std::function<std::remove_pointer_t<CTypeDef>> fn) -> ReturnType
  {
    if (!obj->python) { obj->python = new <FunctionTableStruct>; }
    auto fntable = static_cast<<FunctionTableStruct>*>(obj->python);
    fntable-><member_name> = nb::cast(fn);
    if (fn)
    {
      return <SetFunctionName>(obj, <wrapper_function_name>);
    }
    else { return <SetFunctionName>(obj, nullptr); }
  },
  nb::arg("<obj_param_name>"), nb::arg("fn").none());

The critical difference: for python-member objects you must lazily allocate the function table (if (!obj->python) { obj->python = new ...; }) because it is not automatically created with the object. For integrator callbacks, the function table is managed by the integrator's initialization and accessed through a get_*_fn_table helper.

Nullable arguments

Every callback argument registered with nanobind should have .none() appended to its nb::arg(...) so that Python users can pass None to unregister the callback. This is important - without .none(), passing None from Python will raise a type error.

nb::arg("fn").none()

Checklist

Before finishing, verify:

• If the object lacked a python member: void* python added to the C struct in the *_impl.h header.
• If the object lacked a python member: python initialized to NULL in the Create function.
• If the object lacked a python member: Forward declaration of <Object>FunctionTable_Destroy added (guarded by SUNDIALS_ENABLE_PYTHON) in the .c source, and the destroy helper called in the Destroy function before free().
• If the object lacked a python member: <Object>FunctionTable_Destroy defined as extern "C" in the binding-side C++ source.
• New nb::object member added to the function table struct in *_usersupplied.hpp.
• Wrapper function defined in *_usersupplied.hpp below the struct.
• If the C typedef has array or output-pointer parameters, a custom std::function type alias is defined and the wrapper is fully explicit (not variadic).
• m.def or BIND_<PACKAGE>_CALLBACK added in the corresponding .cpp file.
• The nb::arg for the function parameter has .none() so None can be passed.
• For python-member objects, lazy allocation (if (!obj->python)) is present.
• The function is excluded from autogeneration in the relevant generate.yaml file (add it to the exclude list if not already there).
• A test exercising the new callback from Python is added in bindings/sundials4py/test/.

Common mistakes

• Forgetting to add void* python to the C struct: Without this field, the nanobind binding code has nowhere to store the function table. The user_supplied_fn_caller helper accesses object->python, so a missing field is a compile error on the C++ side.
• Not initializing python to NULL in Create: An uninitialized pointer will cause the lazy allocation check (if (!obj->python)) to skip allocation and dereference garbage.
• Missing FunctionTable_Destroy in the Destroy function: The function table is allocated with new on the C++ side. If Destroy does not delete it, you leak memory. The destroy call must be guarded by #if defined(SUNDIALS_ENABLE_PYTHON) because the function only exists when building with Python support.
• Forgetting extern "C" on the destroy definition: The forward declaration in the .c file uses C linkage. If the C++ definition does not use extern "C", the linker will fail to find the symbol due to C++ name mangling.
• Destroying the function table after free(): The FunctionTable_Destroy call must come before the free() of the parent struct, otherwise you are reading freed memory to get the python pointer.
• Forgetting std::remove_pointer_t: The C typedef is a pointer to a function. nanobind needs the function type itself, so always wrap with std::remove_pointer_t<CTypeDef>.
• Wrong user_data offset: Count the position of the void* user_data argument from the end of the C typedef's parameter list (0-indexed from the last arg). Getting this wrong causes segfaults.
• Not forwarding args: For python-member wrappers, prefer std::forward<Args>(args)... to avoid unnecessary copies.
• Missing .none() on nb::arg: This will cause Python None to throw a type error instead of gracefully unsetting the callback.
• Not excluding from generate.yaml: If the function is not excluded from litgen autogeneration, you will get duplicate/conflicting bindings at compile time.

File location reference

Binding-side files (C++)

Module	Function table header	Binding source
arkode	bindings/sundials4py/arkode/arkode_usersupplied.hpp	bindings/sundials4py/arkode/arkode.cpp
cvodes	bindings/sundials4py/cvodes/cvode_usersupplied.hpp	bindings/sundials4py/cvodes/cvodes.cpp
idas	bindings/sundials4py/idas/ida_usersupplied.hpp	bindings/sundials4py/idas/idas.cpp
kinsol	bindings/sundials4py/kinsol/kinsol_usersupplied.hpp	bindings/sundials4py/kinsol/kinsol.cpp
sundials (SUNStepper, etc.)	bindings/sundials4py/sundials/sundials_stepper_usersupplied.hpp	bindings/sundials4py/sundials/sundials_stepper.cpp
shared helpers	bindings/sundials4py/sundials4py_helpers.hpp	-

C-side files (for adding python member to new objects)

Object	Public header	Private impl header	Source (Create/Destroy)
SUNStepper	include/sundials/sundials_stepper.h	src/sundials/sundials_stepper_impl.h	src/sundials/sundials_stepper.c
SUNLogger	include/sundials/sundials_logger.h	src/sundials/sundials_logger_impl.h	src/sundials/sundials_logger.c

When adding a python member to a new object, locate the equivalent three files for that object (public header, private _impl.h struct definition, and .c source with Create/Destroy) and follow the pattern from Step 0.