py calls c(0)

As We All Know#

Python, as a powerful glue language, makes it very convenient to call various other popular programming languages.
My memory is ridiculously poor; if I don't record what I've learned, I'll forget it in a couple of days.
The C language is the programming language closest to machine language (don't argue about assembly or FPGA scripts; those aren't written by humans).

I plan to thoroughly understand how to glue Python with C so that I can comfortably write C code, which is still a relatively troublesome task. Therefore, a series of articles is needed to document the learning process.

Python can call C language code in the form of source code or dynamic libraries. I initially used the dynamic library form for my learning. I believe using Python to call pre-prepared so/dll files is not difficult; most tutorials or documents prepared by others describe this simple process.

The challenge lies in finding a widely applicable method to call any known C language library.

Compiling Dynamic Libraries#

First, it is necessary to compile the existing C language library into the form of a dynamic library.

This compilation process varies widely. Below, I will introduce how to compile a dynamic library using CMake. Most good C language libraries will provide a CMakeLists.txt that collects all source files, such as in the common folder of modbusX. Define an interface library[^interface library] to collect all the source files.

[^interface library]: It is not compiled itself and does not produce any behavior; it merely collects files for direct use by other projects.

cmake_minimum_required(VERSION 3.22)

project(MODBUSX)
add_library(modbusx INTERFACE) # INTERFACE means this library itself will not be compiled but will be used as a dependency by other targets, allowing MBx_user.h to be defined by itself and change the library behavior.

# Recursively find all source files
file(GLOB_RECURSE SRC ${CMAKE_CURRENT_LIST_DIR}/source/*.c)
# Non-recursive example
# file(GLOB SRC ${CMAKE_CURRENT_LIST_DIR}/source/*.c)

target_include_directories(modbusx INTERFACE
    ${CMAKE_CURRENT_LIST_DIR}/include
)

if(WIN32)
    target_include_directories(modbusx INTERFACE
        ${CMAKE_CURRENT_LIST_DIR}/../port/win32/inc
    )
else()
    target_include_directories(modbusx INTERFACE
        ${CMAKE_CURRENT_LIST_DIR}/../port/generic/inc
    )
endif()

target_sources(modbusx INTERFACE
     ${SRC}
)

if(CMAKE_C_STANDARD LESS 11)
    message(ERROR "Generated code requires C11 or higher")
endif()

Then, in the top-level project, you can call this interface library and add compilation properties to directly generate the dynamic library file. Here is an example of the top-level directory CMakeLists.txt. This CMakeLists.txt does not specify compiler and optimization properties; parameters need to be passed in the compilation statement or specified in the CMake plugin (also passing parameters in the compilation statement).

cmake_minimum_required(VERSION 3.22)
project(MODBUSX_WRAPPER)

# Include original library configuration
add_subdirectory(common)   # Introduce the CMakeLists that collects all files

# Create a new shared library target
add_library(modbusx_shared SHARED ${SRC}) # The dynamic library target named modbusx_shared

# Inherit the INTERFACE properties of the original library
target_link_libraries(modbusx_shared PUBLIC modbusx) # Introduce the interface library into the dynamic library

# Set symbol visibility (key!)
if(WIN32)
    target_compile_definitions(modbusx_shared PRIVATE "MBX_API=__declspec(dllexport)")
else()
    target_compile_definitions(modbusx_shared PRIVATE "MBX_API=__attribute__((visibility("default")))")
endif()

# Force C11 standard
set_target_properties(modbusx_shared PROPERTIES
    C_STANDARD 11
    C_VISIBILITY_PRESET hidden
)

Here is a key issue: in typical header files, function declarations like extern void func(uint8_t* data, uint16_t len); do not expose this function in the dynamic library. You need to add modifiers to make the declaration look like dllexport void func(uint8_t* data, uint16_t len); or default void func(uint8_t* data, uint16_t len); dllexport or default depends on the platform—win/linux.

After compiling the dll/so, you need to use the dump tool to check whether the generated dll/so exposes the expected symbols. For example, I compiled modbusx_x86.dll on the win platform using the above CMakeLists and then used the dumpbin tool in the vs terminal to check. Confirm that it exposes some functions I need, and this dll can be used.

PS E:\PY64_PROJECT\CFFI_TEST> dumpbin /exports modbusx_x86.dll
Microsoft (R) COFF/PE Dumper Version 14.42.34435.0
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file modbusx_x86.dll

File Type: DLL

  Section contains the following exports for libmodbusx_shared.dll

    00000000 characteristics
    6808B5B8 time date stamp Wed Apr 23 17:41:12 2025
        0.00 version
           1 ordinal base
          15 number of functions
          15 number of names

    ordinal hint RVA      name

          1    0 00005510 MBx_Master_Error_Get
          2    1 00002200 MBx_Master_Member_Add
          3    2 00001F20 MBx_Master_RTU_Init
          4    3 000024C0 MBx_Master_Read_Coil_Request
          5    4 00002530 MBx_Master_Read_Disc_Input_Request
          6    5 00002610 MBx_Master_Read_Input_Reg_Request
          7    6 000025A0 MBx_Master_Read_Reg_Request
          8    7 00002090 MBx_Master_TCP_Init
          9    8 00002770 MBx_Master_Write_Coil_Mul_Request
         10    9 00002680 MBx_Master_Write_Coil_Request
         11    A 000027F0 MBx_Master_Write_Reg_Mul_Request
         12    B 00002700 MBx_Master_Write_Reg_Request
         13    C 00003C30 MBx_Slave_RTU_Init
         14    D 00003E10 MBx_Slave_TCP_Init
         15    E 00004770 MBx_Ticks

  Summary

        1000 .CRT
        1000 .bss
        1000 .data
        2000 .debug_abbrev
        1000 .debug_aranges
        1000 .debug_frame
        8000 .debug_info
        2000 .debug_line
        1000 .debug_line_str
        2000 .debug_loclists
        1000 .debug_rnglists
        1000 .debug_str
        1000 .edata
        1000 .idata
        1000 .pdata
        2000 .rdata
        1000 .reloc
        7000 .text
        1000 .tls
        1000 .xdata

Python's ctypes Method#

Using this prepared dll file is very simple. You can import it in Python using the following code.

try:
    lib = ctypes.CDLL("./lib/modbusx_x86.dll")
except OSError as e:
    print(f"Failed to load DLL: {e}")
    exit(1)

Then you can freely call the exposed functions with lib..

while True:
    lib.MBx_Ticks(1000)
    time.sleep(0.001)  # About 1ms cycle

Combining with ctypesgen#

Our C language programs usually have a large number of type definitions, structure definitions, macro definitions, etc. These definitions are very important in C language processing, and Python also needs to pass the same type variables (objects) to call functions in the library. Manually replicating all types and structures in Python is not very realistic. The ctypesgen tool can be used for this. Similar to the following command.

ctypesgen -I . -o mbx_bindings.py MBx_api.h  MBx_port.h

These properties are similar to gcc, or rather, exactly the same; ctypesgen calls gcc to complete this process.

-I . indicates adding an include path, which is the root directory.
-o mbx_bindings.py indicates that the output file name is mbx_bindings.py.
MBx_api.h MBx_port.h are the header files that need to be processed, which contain all the definitions required by the library.

The generated mbx_bindings.py has small segments like the following, replicating the structure types in C language.

# E:\\PY64_PROJECT\\CDLL_TEST\\h\\MBx_api.h: 292
class struct_anon_12(Structure):
    pass

struct_anon_12.__slots__ = [
    'SlaveID',
    'Func',
    'AddrStart',
    'RegNum',
    'Value',
]
struct_anon_12._fields_ = [
    ('SlaveID', uint8_t),
    ('Func', uint8_t),
    ('AddrStart', uint16_t),
    ('RegNum', uint16_t),
    ('Value', uint8_t * int((0x7F * 2))),
]

_MBX_MASTER_REQUEST_RING_NODE = struct_anon_12# E:\\PY64_PROJECT\\CDLL_TEST\\h\\MBx_api.h: 292

# E:\\PY64_PROJECT\\CDLL_TEST\\h\\MBx_api.h: 299
class struct_anon_13(Structure):
    pass

struct_anon_13.__slots__ = [
    'Queue',
    'Head',
    'Tail',
]
struct_anon_13._fields_ = [
    ('Queue', _MBX_MASTER_REQUEST_RING_NODE * int(40)),
    ('Head', uint8_t),
    ('Tail', uint8_t),
]

_MBX_MASTER_REQUEST_RING = struct_anon_13# E:\\PY64_PROJECT\\CDLL_TEST\\h\\MBx_api.h: 299

/**
 * @brief The circular stack of requests waiting to be sent from the host to the slave
 */
typedef struct
{
    uint8_t  SlaveID;                             // The ID of the slave to be sent
    uint8_t  Func;                                // The function code to be sent
    uint16_t AddrStart;                           // The starting address to be sent
    uint16_t RegNum;                              // The number of registers to be sent
    uint8_t  Value[MBX_MASTER_MULTI_REG_MAX * 2]; // The value range to be sent
} _MBX_MASTER_REQUEST_RING_NODE;

typedef struct
{
    _MBX_MASTER_REQUEST_RING_NODE Queue[MBX_MASTER_REQUEST_QUEUE_MAX]; // The request queue to be sent
    uint8_t                       Head;                                // The head pointer of the circular queue (input)
    uint8_t                       Tail;                                // The tail pointer of the circular queue (output)
} _MBX_MASTER_REQUEST_RING;

There are also similar small segments replicating C language macro definitions.

# ./MBx_port.h: 114
try:
    MBX_PORT_RETURN_DEFAULT = 0x00
except:
    pass

# ./MBx_port.h: 115
try:
    MBX_PORT_RETURNT_ERR_INDEFINITE = 0x01
except:
    pass

#define MBX_PORT_RETURN_DEFAULT         0x00 // Default no error
#define MBX_PORT_RETURNT_ERR_INDEFINITE 0x01 // Default undefined error (for temporary use or minimal function return error)

So later in the Python program, you can seamlessly replicate C language operations to call resources from the dll. For example, the most complex callback functions and various memory space bindings can have code like the following.

# Serial port function
@MBX_SEND_PTR
def SerialSendPort(data: ctypes.POINTER(ctypes.c_uint8), length: ctypes.c_uint32) -> ctypes.c_uint32:
    try:
        buffer = ctypes.string_at(data, length)
        bytes_written = ser.write(buffer)
        if bytes_written == length:
            print(f"Sent: {buffer.hex()} ({bytes_written} bytes)")
            return MBX_PORT_RETURN_DEFAULT
        else:
            print(f"Send incomplete: expected {length} bytes, sent {bytes_written} bytes")
            return MBX_PORT_RETURNT_ERR_INDEFINITE
    except serial.SerialException as e:
        print(f"Serial error: {e}")
        return MBX_PORT_RETURNT_ERR_INDEFINITE
    except Exception as e:
        print(f"Unexpected error: {e}")
        return MBX_PORT_RETURNT_ERR_INDEFINITE


@MBX_GTEC_PTR
def SerialGetcPort(data: ctypes.POINTER(ctypes.c_uint8)) -> ctypes.c_uint32:
    if ser.in_waiting > 0:
        byte = ser.read(1)
        data[0] = byte[0]
        print(f"Received: {byte.hex()}")
        return MBX_PORT_RETURN_DEFAULT
    return MBX_PORT_RETURNT_ERR_INDEFINITE


# Initialize slave
slave = MBX_SLAVE()
SRxBuffer = (ctypes.c_uint8 * 84)()
STxBuffer = (ctypes.c_uint8 * 84)()

# Initialization
result = lib.MBx_Slave_RTU_Init(
    ctypes.byref(slave),
    1,  # Slave ID
    MapList,
    SerialSendPort,
    SerialGetcPort,
    115200,  # Baud rate
    SRxBuffer,
    84,
    STxBuffer,
    84,
)

/* Serial port function */
uint32_t SerialSendPort(const void *Data, size_t Len)
{
    WINBOOL b     = FALSE; // Send operation flag
    DWORD   wWLen = 0;     // Actual length of data sent
    /* Attempt to send */
    b = WriteFile(comHandle, Data, Len, &wWLen, NULL);
    if(b && wWLen == Len)
        return MBX_PORT_RETURN_DEFAULT;
    else
        return MBX_PORT_RETURNT_ERR_INDEFINITE;
}

uint32_t SerialGetcPort(uint8_t *Data)
{
    WINBOOL b     = FALSE; // Receive operation flag
    DWORD   wRLen = 0;     // Actual length of data received
    /* Attempt to receive */
    b = ReadFile(comHandle, Data, 1, &wRLen, NULL);
    if(b == TRUE && wRLen == 1)
    {
        return MBX_PORT_RETURN_DEFAULT;
    }
    else
    {
        return MBX_PORT_RETURNT_ERR_INDEFINITE;
    }
}

_MBX_SLAVE MBxSlave;
uint8_t *SRxBuffer = (uint8_t *)malloc(84 * sizeof(uint8_t));
uint8_t *STxBuffer = (uint8_t *)malloc(84 * sizeof(uint8_t));
state = MBx_Slave_RTU_Init(&MBxSlave,     // Slave object
                          1,              // Slave ID
                          MapList,        // Address mapping table
                          SerialSendPort, // Send function
                          SerialGetcPort, // Receive function
                          9600,           // Baud rate
                          SRxBuffer,      // Library internal receive buffer allocation
                          84,             // Maximum length of receive buffer
                          STxBuffer,      // Library internal send buffer allocation
                          84)             // Maximum length of send buffer

[!NOTE]

It is important to note that the type in C is _MBX_SLAVE, while in Python I use MBX_SLAVE.

Here I manually modified it in mbx_bindings.py because in Python, identifiers starting with an underscore are considered private and not exposed outside the file, which can lead to calling errors. Therefore, if there are types or other definitions in C that start with an underscore, you may need to manually correct them in mbx_bindings.py.

This article is updated synchronously by Mix Space to xLog. The original link is https://www.yono233.cn/posts/novel/25_5_6_py4c0