repository·master·Indexed 43 days ago
https://github.com/galis/opentiktokAn open-source video editing project for Android designed to provide a CapCut-like experience. Version 1.0 includes features such as frame-by-frame preview, text and sticker effects, video composition, and transitions. The project includes a C++ core with the ARContext class for AR engine lifecycle management, Filter3D for 3D object overlays, and FilterBase for implementing custom video filters.
To test the project's functionality, you can use the testfile.mp4 located in the raw/ directory. For Android-based testing, push this file to the following cache directory:
/data/data/com.galix.opentiktok/cache
push testfile.mp4 to /data/data/com.galix.opentiktok/cacheThe upcoming v2.0 release is planned to include:
The following features are implemented and available in the v1.0 release:
The ARContext class is the primary interface for managing the Augmented Reality (AR) engine lifecycle. It handles the creation, rendering, and destruction of the AR environment.
Key lifecycle methods:
create(): Initializes the AR engine.draw(int textureId): Renders the AR effects onto the provided OpenGL texture ID.destroy(): Cleans up resources and shuts down the engine.onSurfaceChanged(int width, int height): Updates the internal viewport dimensions when the rendering surface changes.slar::ARContext context;
context.create();
// ... in render loop ...
context.draw(textureId);
// ... on resize ...
context.onSurfaceChanged(newWidth, newHeight);
// ... cleanup ...
context.destroy();The Filter3D class is a specialized filter used to overlay 3D models onto video frames. It extends FilterBase and manages 3D transformations (Model-View-Projection) to render objects using .obj files.
To use it, you must load a 3D model file, set the spatial transformation data (position, scale, and rotation), and handle the drawing lifecycle via onDraw.
// Example usage pattern for Filter3D
slfilter::Filter3D filter3d;
// 1. Load the 3D model
filter3d.loadFile("path/to/model.obj");
// 2. Set transformation data
// center: Point2f (x, y)
// scale: float
// pitch, yaw, roll: rotation angles
cv::Point2f center(0.5f, 0.5f);
filter3d.setData(center, 1.0f, 0.0f, 0.0f, 0.0f);
// 3. In the rendering loop
filter3d.onDraw(textureId);To create a custom video filter, inherit from the slfilter::FilterBase class. You must manage the filter lifecycle by calling init() during initialization and destroy() when the filter is no longer needed.
Key lifecycle hooks to override:
onInit(): Called during the init() process.onInitialized(): Called after initialization is complete.onDraw(GLuint textureId): The primary method for rendering logic. It receives the textureId of the input frame.onDestroy(): Called during the destroy() process.onOutputChanged(int width, int height): Called when the output dimensions change.Note: FilterBase can also serve as a default filter if no specific logic is provided.
class MyCustomFilter : public slfilter::FilterBase {
public:
MyCustomFilter(const std::string& vShader, const std::string& fShader)
: FilterBase(vShader, fShader) {}
void onDraw(GLuint textureId) override {
// Custom rendering logic using textureId
}
};
// Usage:
MyCustomFilter filter("vertex_shader_code", "fragment_shader_code");
filter.init();
GLuint resultTexture = filter.draw(inputTextureId);
filter.destroy();You can control how the filter outputs its data and the resolution of its internal buffers.
Render Modes
Use setRenderMode(RenderMode mode) to choose between:
OUTPUT: Standard output mode.FRAMEBUFFER: Renders into an internal framebuffer.Buffer Management
setFrameBufferSize(const SLSize &size): Sets the resolution of the internal framebuffer.getFrameBuffer(): Returns the GL texture ID of the framebuffer.getFrameBufferColorTexture(): Returns the color texture ID of the framebuffer.getOutputSize(): Returns the current output dimensions.getFrameBufferSize(): Returns the current framebuffer dimensions.The FilterOES class is a hardware-accelerated implementation of FilterBase designed for OES texture filtering. It allows for efficient rendering of textures using OpenGL ES hardware acceleration.
Key methods for managing the filter include:
onDraw(GLuint textureId): Performs the actual drawing operation using the provided texture ID.setTextureInfo(SLSize size): Configures the dimensions of the texture being processed.setBgColor(int color): Sets the background color for the filter operation.Note: This class is part of the slfilter namespace.
#include "FilterOES.h"
// Example usage pattern
slfilter::FilterOES filter;
filter.setTextureInfo({width, height});
filter.setBgColor(0xFF000000);
filter.onDraw(textureId);Once a filter is initialized, you can pass data to its shaders using binding methods. These methods map values to shader uniforms or attributes.
Attribute Binding
bindAttr(string &&key, float values[], int lineWidth = 2): Binds an array of floats as an attribute.Uniform/Data Binding
bindInt(string &&key, int value): Binds an integer.bindFloat(string &&key, float value): Binds a float.bindFloatArray(string &&key, float values[], int length): Binds a float array.bindFloatVec3(string &&key, float *floatValue): Binds a 3-component float vector.bindTexture(string &&key, GLuint texture): Binds a GL texture ID.bindTexture(string &&key, SLMat &mat): Binds a matrix as a texture.bindRect(string &&key, SLRect &rect): Binds a rectangle.bindPoint(string &&key, SLRect &point): Binds a point.bindTMat(string &&key, SLMat &mat): Binds a transformation matrix (specifically for OpenCV integration).All keys must match the names defined in your GLSL shader code.
filter.bindFloat("u_opacity", 0.5f);
filter.bindInt("u_frame_count", 120);
filter.bindTexture("u_sampler", textureId);The ARConfig struct and CONFIG_TYPE enum define the configuration parameters for AR effects. While CONFIG_TYPE defines categories of effects, ARConfig is used to pass specific data like lipstick types and textures.
enum CONFIG_TYPE {
EYELINE,
EYELASH,
EYESHADOW,
LIPSTICK,
SURFACE,
};
struct ARConfig {
string lipstickType;
Mat lipstick;
bool lipstickChange;
};The Filter3D class provides the following public methods for managing 3D model rendering:
loadFile(string obj): Loads a 3D model from an OBJ file path. Returns an integer status code.setData(Point2f ¢er, float scale, float pitch, float yaw, float roll): Configures the 3D object's position and orientation.center: A cv::Point2f representing the anchor point.scale: A float for scaling the object.pitch, yaw, roll: Euler angles for rotation.onDraw(GLuint textureId): The primary rendering call. It draws the 3D object over the provided textureId (the video frame).onOutputChanged(int width, int height): Updates the internal projection matrix when the video output dimensions change.