react-native-mlkit

The react-native-mlkit-face-detection module provides React Native APIs for Google's ML Kit Native Modules (Swift and Kotlin). It allows you to perform face detection within a React Native application using the speed and power of native ML Kit implementations.

Key Capabilities:

• Face Detection: Detect faces in an image and identify key facial features.
• Face Contour Detection: When enabled, provides a list of points for each detected facial feature representing its shape.

Important Note: This API is designed for face detection (identifying facial features and presence), not face recognition (identifying specific individuals).

This repository is a monorepo organized as follows:

• Apps: Contains the ExampleApp used for demonstrations.
• Modules: Contains the individual MLKit modules. Each is a separate npm package published under the @infinitered/ scope (e.g., @infinitered/react-native-mlkit-face-detection).
• Packages: Contains internal, non-published packages (like linter configurations).

To use a specific module, refer to the README within its respective directory in the modules/ folder. Available modules include:

• react-native-mlkit-core
• react-native-mlkit-face-detection
• react-native-mlkit-text-recognition
• react-native-mlkit-document-scanner
• react-native-mlkit-image-labeling
• react-native-mlkit-object-detection

The example-app uses the Ignite boilerplate, which provides a structured directory layout for React Native applications. Key directories include:

• app/: The core application logic, containing components, models (MobX State Tree), navigators (React Navigation), screens, services, and theme configuration.
• assets/: Organized storage for icons and images.
• ignite/: Contains Ignite-specific CLI tools and customizable templates.
• test/: Holds Jest configurations and mocks.
• android/ & ios/: Native platform project files.

The face detection functionality is powered by a React Context system. The FaceDetectionProvider initializes a RNMLKitFaceDetector and provides it to the component tree via FaceDetectionContext.

Consumers can then use hooks like useFaceDetection to grab the raw detector instance or useFacesInPhoto to use high-level detection logic. The context value (FaceDetectionContextValue) primarily exposes the faceDetector instance.

FaceDetectionState represents the lifecycle of the face detection process. Monitoring these states allows you to update your UI (e.g., showing a loading spinner during modelLoading or an error message during error).

React Native MLKit currently supports the following platforms:

• iOS
• Android

Note: The native modules do not work on the web, and there are currently no plans to support web platforms.

The library is modularized into specific packages based on the MLKit functionality required. Currently, the following modules are available:

• Face Detection
• Object Detection
• Image Labeling
• Document Scanner
• Text Recognition

To improve performance or delay setup until a specific user action, you can prevent the face detector from initializing automatically upon mount.

1. Set the deferInitialization prop on the FaceDetectionProvider.
2. Manually call the .initialize() method on the detector instance (obtained via useFaceDetection) when you are ready to start using it.

// 1. Configure the provider to defer initialization
function App() {
  return (
    <FaceDetectionProvider
      options={FACE_DETECTION_OPTIONS}
      deferInitialization
    >
      {/* rest of your app */}
    </FaceDetectionProvider>
  );
}

// 2. Manually initialize when needed
function MyComponent() {
  const detector = useFaceDetection();
  
  useEffect(() => {
    detector.initialize();
  }, [detector]);

  // ...
}