Flashlight

The flashlight cloud service and the flashlight.dev platform are provided completely free of charge.

Because the services run on shared infrastructure (Theodo's servers and AWS Device Farm), users may experience queues when waiting for devices to become available. Projects and clients of Theodo are given priority in the queue.

Flashlight generates a performance score for Android apps by aggregating various metrics. It requires no setup within your application code and can be used to measure performance even on production apps.

You can use Flashlight in three primary ways:

1. Web Interface: Upload an app to app.flashlight.dev to receive a performance score.
2. CLI (Manual Audit): Use flashlight measure for quick audits with real-time measures.
3. CLI (Automated Testing): Use flashlight test to automate measures via e2e performance testing over multiple iterations.
4. CLI (Cloud/CI): Use flashlight cloud to run measures on real devices in the cloud and integrate them into your CI pipeline.

In React Native, JS FPS (JavaScript Frames Per Second) is a critical metric for app responsiveness. If JS FPS drops to 0, the app becomes unresponsive to user interactions (e.g., button clicks).

Flashlight tracks the JS thread CPU usage, which is inversely correlated with JS FPS:

• 0% JS thread CPU Usage correlates to Max JS FPS.
• 100% JS thread CPU Usage correlates to 0 JS FPS.

When running performance measurements, your goal is to ensure the JS thread does not appear in the Processes with high CPU usage section of the report and that your average FPS remains close to 60.

To avoid the high overhead of the top command, you can implement a custom measurement tool by reading directly from the Linux /proc filesystem. This approach provides higher frequency sampling with minimal impact on the device.

Implementation Logic

1. Identify Processes: Locate the process ID (PID) folders within the /proc directory.
2. Identify Threads: For a specific PID, navigate to /proc/<PID>/task/ to find subfolders representing individual threads (each with its own Thread PID).
3. Extract Stats: Read the /proc/<PID>/task/<Thread PID>/stat file.
4. Parse Data: According to Linux documentation, the 14th and 15th columns of the stat file contain the necessary CPU statistics for calculating consumption.

You can measure the startup performance of a React Native application using the Flashlight CLI. Instead of writing a TypeScript test file, you can run a single command by providing the application's bundle ID, the command to launch the activity, and the test duration.

To run a performance test, use the flashlight test command with the following flags:

• --bundleId: The unique identifier for your Android application (e.g., com.reactnativefeed).
• --testCommand: The shell command used to start the application's main activity via ADB (e.g., adb shell am start com.reactnativefeed/.MainActivity).
• --duration: The length of the performance measurement in milliseconds.

flashlight test --bundleId com.reactnativefeed \
  --testCommand "adb shell am start com.reactnativefeed/.MainActivity" \
  --duration 15000

To generate a report from the JSON file produced by your programmatic test:

1. Ensure your Appium server is running (npx appium).
2. Execute your test script using ts-node (e.g., npx ts-node yourScriptName.ts).
3. Once the script completes, it will output a JSON filename.
4. Use the flashlight report command to open the web profiler and view the results.

flashlight report yourResultFileName.json

Start a local development server to preview the website. This command will automatically open a browser window, and most changes will be reflected live via hot reloading without requiring a server restart.

$ yarn start

You can obtain an app start performance score using the Flashlight Cloud web interface. This process involves uploading an APK and defining a trigger to measure Time to Interactive (TTI).

1. Upload APK: Go to app.flashlight.dev and upload your Android APK.
2. Define TTI Trigger: To measure TTI, provide a text string that appears in your app once it has successfully started. Flashlight uses the appearance of this text to determine the app's start state.
   • Note: The text does not need to be an exact match; a partial string (e.g., "world" for "Hello world") is sufficient.
3. Receive Report: After processing (typically ~10 minutes, depending on queue), you will receive a performance report via email. If the test fails, you will receive a video of the attempt.

To install the Flashlight CLI on macOS or Linux, run the following command in your terminal:

curl https://get.flashlight.dev | bash

Note for macOS arm64 (Apple Silicon) users: You must have Rosetta installed. If you do not have it, install it using:

softwareupdate --install-rosetta --agree-to-license

You can integrate Flashlight into your CI pipeline to capture and report app startup scores. Use the --scoring APP_START flag to ensure the scoring method accounts for the test run time.

Setting a --testName allows you to track metric evolution for that specific test type on the evolution dashboard.

flashlight cloud --test start.yaml --app app.apk \
 --projectId <PROJECT_ID> \
 --scoring APP_START \
 --testName "STARTUP"