add-device-info

name	add-device-info
description	Add device information query functions through HAL → ins:: API → language bindings

Add Device Information Functions

Pattern for exposing hardware information queries through the Insight7 plugin architecture to user-facing APIs and language bindings.

Architecture

HAL (device_ext.h)  →  C API Wrapper  →  ins:: API  →  Bindings (Python/Lua/Julia)
  C_DeviceInterface       ProfilerWrapper            free/classes      module-level
  function pointers       stores iface ptr           (profiler.h)      ins.Profiler

C API Wrapper Pattern (for opaque handles)

When the HAL returns an opaque handle (C_Profiler, C_Event etc.) and the frontend needs to call back into the HAL, create a wrapper struct:

// src/core/profiler.cpp
struct Wrapper {
  C_Profiler handle;          // HAL backend's opaque handle
  const C_DeviceInterface *iface;  // HAL interface pointer for dispatch
};

C_Status api_function(C_Profiler prof) {
  auto *w = reinterpret_cast<Wrapper*>(prof);
  return w->iface->some_function(w->handle);
}

This is required because the HAL's opaque handles don't carry the interface pointer — only the backend plugin knows it at creation time. The C API layer must store it alongside the handle.

Exception Safety (Lua/sol2)

sol2 does NOT automatically catch C++ exceptions from lambdas or raw lua_pushcfunction callbacks. A C++ exception escaping to LuaJIT shows as "C++ exception" and crashes the process. All sol2 lambdas must wrap C++ calls in try/catch.

Step 1: HAL Layer (if new query needed)

If the HAL already has the function pointer (e.g., get_compute_capability, get_runtime_version, device_memory_stats), skip to Step 2.

Otherwise, add a new function pointer to C_DeviceInterface in include/insight/c_api/device_ext.h:

C_Status (*get_device_name)(C_Device device, char *buf, size_t buf_size);

Place it near related functions (after get_max_grid_dim_size, before Profiler section).

Step 2: CUDA Backend Implementation

In backends/cuda/device/cuda_device.cpp:

static C_Status cuda_get_device_name(C_Device device, char *buf,
                                     size_t buf_size) {
  if (!buf || buf_size == 0 || !device) return C_FAILED;
  cudaDeviceProp prop;
  cudaError_t err = cudaGetDeviceProperties(&prop, device->id);
  if (err != cudaSuccess) {
    gpu_set_last_error(cudaGetErrorString(err));
    return C_FAILED;
  }
  std::strncpy(buf, prop.name, buf_size - 1);
  buf[buf_size - 1] = '\0';
  return C_SUCCESS;
}

Register in the iface setup function:

iface->get_device_name = cuda_get_device_name;

CPU backend stubs are not needed — the ins:: layer handles missing function pointers gracefully.

Step 3: ins:: Upper-Level API

In include/insight/core/place.h — declare free functions:

std::string device_name(DeviceKind kind, int device_id = 0);
int cuda_version();
int driver_version();
int compute_capability(int device_id = 0);

struct DeviceMemoryInfo { size_t total; size_t free; };
DeviceMemoryInfo device_memory(int device_id = 0);

In src/core/place.cpp — implement using HAL:

std::string device_name(DeviceKind kind, int device_id) {
  if (kind == DeviceKind::CPU) return "CPU";
  const C_DeviceInterface *iface = get_device_interface(kind);
  if (!iface || !iface->get_device_name) return "";
  C_Device_st dev;
  dev.id = device_id;
  char buf[256] = {};
  C_Status status = iface->get_device_name(&dev, buf, sizeof(buf));
  return (status == C_SUCCESS) ? std::string(buf) : "";
}

Pattern: get C_DeviceInterface* via get_device_interface(), check function pointer is non-null, create C_Device_st with dev.id = device_id, call HAL, return result or fallback.

Step 4: Language Bindings

Python (pybind11)

m.def("device_name",
    [](const std::string &kind, int device_id) {
      DeviceKind dk = (kind == "gpu" || kind == "cuda")
                          ? DeviceKind::GPU : DeviceKind::CPU;
      return device_name(dk, device_id);
    },
    py::arg("kind") = "cpu", py::arg("device_id") = 0);
m.def("cuda_version", []() { return cuda_version(); });
m.def("device_memory", [](int id) {
    auto info = device_memory(id);
    return py::make_tuple(info.total, info.free);
}, py::arg("device_id") = 0);

Lua (sol2)

m["device_name"] = [](sol::optional<std::string> kind,
                      sol::optional<int> device_id) {
  std::string k = kind.value_or("cpu");
  DeviceKind dk = (k == "gpu" || k == "cuda") ? DeviceKind::GPU : DeviceKind::CPU;
  return device_name(dk, device_id.value_or(0));
};

Julia (C ABI + wrapper)

C wrapper in insight_julia_capi.cpp:

void insight_jl_device_name(int32_t device_id, char *buf, size_t buf_size) {
  std::string name = device_name(DeviceKind::GPU, device_id);
  std::strncpy(buf, name.c_str(), buf_size - 1);
  buf[buf_size - 1] = '\0';
}

Julia wrapper in Insight.jl:

function device_name(device_id::Int=0)::String
    buf = Vector{UInt8}(undef, 256)
    ccall((:insight_jl_device_name, LIB_INSIGHT), Cvoid,
          (Int32, Ptr{UInt8}, Csize_t), Int32(device_id), buf, 256)
    return String(buf[1:findfirst(==(0x00), buf)-1])
end

Step 4a: device_memory_info — Extended API (v2, PR #46)

The original device_memory() convenience wrapper returns a DeviceMemoryInfo struct. A lower-level device_memory_info() function was added to support C API dispatch:

C API (include/insight/c_api/memory.h)

C_Status insight_device_memory_info(int32_t device_type, int32_t device_id,
                                    size_t *total, size_t *free);

C++ API (include/insight/core/place.h)

static void device_memory_info(DeviceKind kind, int device_id,
                               size_t *total, size_t *free);

CPU Backend Implementation (⚠️ Common Pitfall: VmRSS Bug)

Use MemTotal + MemAvailable, NOT VmRSS!

The original buggy implementation returned VmRSS (process RSS, ~200MB) as total_memory and MemAvailable (~800GB) as free_memory. This caused used = total - free to underflow since total < free, producing negative used memory in all language bindings.

✅ Correct Linux implementation (reads /proc/meminfo):

std::ifstream proc_mem("/proc/meminfo");
std::string line;
size_t mem_total = 0, mem_free = 0;
while (std::getline(proc_mem, line)) {
  if (line.find("MemTotal:") == 0) {
    std::istringstream iss(line.substr(9));
    iss >> mem_total;
    mem_total *= 1024;  // kB → bytes
  } else if (line.find("MemAvailable:") == 0) {
    std::istringstream iss(line.substr(13));
    iss >> mem_free;
    mem_free *= 1024;  // kB → bytes
  }
  if (mem_total > 0 && mem_free > 0) break;
}

✅ Correct Windows implementation (uses GlobalMemoryStatusEx only, no GetProcessMemoryInfo):

MEMORYSTATUSEX mem_stat;
mem_stat.dwLength = sizeof(mem_stat);
GlobalMemoryStatusEx(&mem_stat);
*total_memory = static_cast<size_t>(mem_stat.ullTotalPhys);
*free_memory = static_cast<size_t>(mem_stat.ullAvailPhys);

Why VmRSS was wrong: GPU backends (CUDA) return device-global total/free (e.g. 80GB total / 40GB free). For consistency, CPU must also return system-global total/available RAM, not per-process RSS. The used = total - free computation in the demo layer is only valid when total ≥ free.

Language Binding Signatures

CRITICAL: Parameter types differ per language!

Language	Call Signature	Notes
C++	device_memory_info(DeviceKind::CPU, 0, &total, &free)	Uses enum + pointer
Python	ins.device_memory_info(0, 0) → returns (total, free)	0=CPU, 1=GPU — int, not string!
Lua	ins.device_memory_info(0, 0) → returns total, free	0=CPU, 1=GPU — int, not string!
Julia	Insight.device_memory_info(Int64(0), Int64(0)) → returns (total, free)	Int64 not Int32!

# ✅ CORRECT Python usage
cpu_total, cpu_free = ins.device_memory_info(0, 0)   # CPU
gpu_total, gpu_free = ins.device_memory_info(1, 0)   # GPU

-- ✅ CORRECT Lua usage
local cpu_total, cpu_free = ins.device_memory_info(0, 0)   -- CPU
local gpu_total, gpu_free = ins.device_memory_info(1, 0)   -- GPU

# ❌ WRONG — Int32 arguments
cpu_total, cpu_free = Insight.device_memory_info(Int32(0), Int32(0))  # MethodError!

# ✅ CORRECT — Int64 arguments
cpu_total, cpu_free = Insight.device_memory_info(Int64(0), Int64(0))
gpu_total, gpu_free = Insight.device_memory_info(Int64(1), Int64(0))

Python binding implementation

m.def("device_memory_info",
    [](int device_kind, int device_id) {
      size_t total = 0, free = 0;
      DeviceKind dk = (device_kind == 1) ? DeviceKind::GPU : DeviceKind::CPU;
      device_memory_info(dk, device_id, &total, &free);
      return py::make_tuple(total, free);
    },
    py::arg("device_kind") = 0, py::arg("device_id") = 0);

Lua binding implementation

m["device_memory_info"] = [](int device_kind, int device_id) {
  size_t total = 0, free = 0;
  DeviceKind dk = (device_kind == 1) ? DeviceKind::GPU : DeviceKind::CPU;
  device_memory_info(dk, device_id, &total, &free);
  return std::make_tuple(total, free);
};

Julia C API wrapper

extern "C" void insight_jl_device_memory_info(
    int64_t device_kind, int64_t device_id,
    int64_t *total, int64_t *free) {
  size_t t = 0, f = 0;
  DeviceKind dk = (device_kind == 1) ? DeviceKind::GPU : DeviceKind::CPU;
  device_memory_info(dk, (int)device_id, &t, &f);
  *total = (int64_t)t;
  *free = (int64_t)f;
}

Note: Julia C API uses int64_t (matching Int64), NOT int32_t (matching Int32). This is because the Insight Julia binding uses Int64 for all device_memory_info parameters due to how ccall types are defined in Insight.jl.

Step 5: Test

# Quick smoke test
import insight as ins
ins.load_backend("cuda")
print(ins.device_name("gpu"))       # "NVIDIA A800-SXM4-80GB"
print(ins.cuda_version())            # 11080
print(ins.compute_capability())      # 80
print(ins.device_memory())           # (total, free) in bytes

Key HAL Functions Already Available

HAL pointer	Returns	Notes
get_device_count	device count	Already wrapped as ins::device_count()
get_compute_capability	SM version (e.g., 80)	major*10+minor
get_runtime_version	CUDA runtime ver	major1000+minor10
get_driver_version	CUDA driver ver	major1000+minor10
device_memory_stats	(total, free) bytes	via cudaMemGetInfo
get_multi_process	SM count	optional
get_max_threads_per_mp	threads/SM	optional
get_max_threads_per_block	threads/block	optional
get_max_grid_dim_size	grid dims [3]	optional
get_device_name	GPU name string	new — needs HAL addition