DeepStream Python Apps

The deepstream-test4 sample demonstrates a metadata-to-cloud pipeline:

1. Metadata Creation: The application creates NVDS_META_EVENT_MSG type metadata and attaches it to the buffer. This metadata can include custom objects via the extMsg and extMsgSize fields.
2. nvmsgconv Plugin: This plugin reads the NVDS_META_EVENT_MSG metadata and generates a JSON payload following the "DeepStream Schema". Static properties for the schema are read from a configuration file (e.g., dstest4_msgconv_config.txt). The resulting payload is attached to the buffer as NVDS_META_PAYLOAD type metadata.
3. nvmsgbroker Plugin: This plugin extracts the NVDS_META_PAYLOAD metadata and sends it to the backend server using the specified protocol adaptor APIs.

Custom Metadata: To extend NvDsEventMsgMeta with custom structures, assign your structure pointer to extMsg and set extMsgSize. If your custom object requires complex memory management, you must implement/extend event_msg_meta_copy_func() and event_msg_meta_release_func() in bindschema.cpp.

When binding string fields (like sensorStr in NvDsEventMsgMeta) that require deep copying between Python and C, use a macro to define getter and setter logic. This ensures that when a Python string is assigned to a C struct, memory is correctly allocated on the C side.

The STRING_PROPERTY Macro Pattern: This macro defines a getter that returns the C address and a setter that uses calloc to allocate memory and str.copy to move the content.

Binding a string property:

py::class_<NvDsEventMsgMeta>(m, "NvDsEventMsgMeta", pydsdoc::metaschema::EventmsgDoc::descr)
    .def_property("sensorStr", STRING_PROPERTY(NvDsEventMsgMeta, sensorStr))

Python Usage Pattern: To safely copy a Python string into a C-allocated struct field, use a helper like pyds.get_string() to resolve the address before assignment:

test_string = 'test message ' + str(frame_number)
data = pyds.alloc_custom_struct(user_meta)
data.message = test_string # python string object at first
data.message = pyds.get_string(data.message)

#define STRING_PROPERTY(TYPE, FIELD)                               \n        [](const TYPE &self)->size_t {                             \n            return (size_t)self.FIELD;                             \n        },                                                         \n        [](TYPE &self, std::string str) {                          \n            int strSize = str.size();                              \n            self.FIELD = (char*)calloc(strSize + 1, sizeof(char)); \n            str.copy(self.FIELD, strSize);                         \n        },                                                         \n        py::return_value_policy::reference 

// Binding example
py::class_<NvDsEventMsgMeta>(m, "NvDsEventMsgMeta",
                                pydsdoc::metaschema::EventmsgDoc::descr)
    .def_property("sensorStr",
                    STRING_PROPERTY(NvDsEventMsgMeta, sensorStr))

// Python usage
test_string = 'test message ' + str(frame_number)
data = pyds.alloc_custom_struct(user_meta)
data.message = test_string
data.message = pyds.get_string(data.message)

Directly type-casting certain GStreamer data types (like GstEvent *) in Python can be difficult. To circumvent this, implement a binding in PyDS that accepts a GstElement (passed as a size_t from Python) and performs the event creation and sending within the C++ binding layer.

For example, to send a stream reset event using gst_nvevent_new_stream_reset(source_id), create a binding that wraps gst_element_send_event and the specific NVEVENT creation function. This allows the Python application to receive a simple boolean indicating if the event was handled successfully.

m.def("gst_element_send_nvevent_new_stream_reset",
        [](size_t gst_element, int source_id) {
            auto *element = reinterpret_cast<GstElement *>(gst_element);
            return gst_element_send_event(element, gst_nvevent_new_stream_reset(source_id));
        },
        pydsdoc::methodsDoc::gst_element_send_nvevent_new_stream_reset);

When you need to attach data structures that are not part of the standard C API headers to DeepStream buffers, you can use NvDsUserMeta. To make these structures accessible in Python, you must write custom Pybind11 bindings.

This process involves:

1. Defining the C++ structure: Create a header file (e.g., custom_data.hpp) defining your struct.
2. Creating a bindings header: Create a header (e.g., bindcustom.hpp) following the bind{header_name}.hpp convention to declare the binding function.
3. Implementing the bindings: In a .cpp file, use Pybind11 to map the struct members and implement memory management functions.
4. Memory Management: You must provide a copy_func and a release_func to NvDsUserMeta to handle deep copying and memory deallocation of your custom data.
5. Integration: Add the new files to CMakeLists.txt and register the submodule in pyds.cpp.

To expose a C-style struct to Python, use the pybind11::class_ method. You must provide the module, the name of the struct, and a docstring.

Step 1: Declare the binding function

Create a header (e.g., include/bind/bindnvdsmeta.hpp) to declare the submodule binding function:

#include "pyds.hpp"
#include "../docstrings/pydocumentation.h"

namespace py = pybind11;

namespace pydeepstream {
    void bindnvdsmeta(py::module &m);
}

Step 2: Define the binding function

In the corresponding source file (e.g., src/bindnvdsmeta.cpp), implement the function:

#include "bind_string_property_definitions.h"
#include "bindnvdsmeta.hpp"

namespace py = pybind11;

namespace pydeepstream {

void bindnvdsmeta(py::module &m) {
    // Binding code goes here
}
}

Step 3: Bind the struct and its members

Use py::class_ to bind the struct. To expose data members (Plain Old Data types), use .def_readwrite().

void bindnvdsmeta(py::module &m) {
    py::class_<NvDsObjectMeta>(m, "NvDsObjectMeta",
                                pydsdoc::nvmeta::ObjectMetaDoc::descr)
        .def(py::init<>()) // Constructor wrapper
        .def_readwrite("base_meta", &NvDsObjectMeta::base_meta);
}

Note: If you bind a member that is itself a struct (like base_meta), you will only be able to access its internal members in Python once that specific struct type is also bound.

void bindnvdsmeta(py::module &m) {
    py::class_<NvDsObjectMeta>(m, "NvDsObjectMeta",
                                pydsdoc::nvmeta::ObjectMetaDoc::descr)
        .def(py::init<>()); // Constructor wrapper, empty for struct
                                .def_readwrite("base_meta", &NvDsObjectMeta::base_meta);
}