Vue Camera Gestures for Vue 3

website·Indexed 31 days ago

https://vue3.cameragestures.com/

A Vue 3 component that allows developers to train an AI model in the browser to recognize gestures from a user's camera and perform actions based on those gestures.

Tokens: 3.8K
Snippets: 21
Records: 26
Agent score: 50%

What's inside Vue Camera Gestures

Overview of CameraGestures for Vue
CameraGestures allows developers to integrate AI-powered gesture recognition into Vue applications by using a dedicated Vue component. The system trains and runs the AI model directly in the browser to recognize gestures from the user's camera and trigger corresponding actions. Key characteristics include:
- Privacy/Security: All image processing and AI inference occur on-device; no image data is sent to a server.
- Accessibility: Enables application control for users who cannot use a keyboard or mouse, or provides an alternative interactive experience.
https://vue3.cameragestures.com/
Understand the Vue Camera Gestures training workflow
When the camera-gestures component is rendered, it follows a specific sequence to train and verify gestures:
1. Neutral Position Training: Prompts for a neutral position, waits 1 second, then trains for 3 seconds.
2. Gesture Training: Loops through each defined gesture, prompts the user, waits 1 second, and trains for 3 seconds.
3. Neutral Position Verification: Prompts for neutral position, waits 1 second, and verifies for 1 second.
4. Gesture Verification: Loops through each gesture, prompts the user, waits 1 second, and verifies for 1 second.
5. Accuracy Check: If any gesture is identified with < 90% accuracy, the user gets one extra second to verify. If it fails again, the process restarts from step 1.
6. Completion: Once all gestures are verified > 90% accuracy, instructions disappear and gesture events are fired.
https://vue3.cameragestures.com/guide/

Install Vue Camera Gestures without bundled Tensorflow

To avoid duplicate Tensorflow bundles, you can install the dependencies separately and import from the dist-plain directory. The library size drops to ~4KB (min+gzip) in this mode.

npm i @tensorflow/tfjs @tensorflow-models/knn-classifier @tensorflow-models/mobilenet vue-camera-gestures

// Import from dist-plain
import CameraGestures from 'vue-camera-gestures/dist-plain/vue-camera-gestures.es.js'
import 'vue-camera-gestures/dist-plain/style.css'

app.component('camera-gestures', CameraGestures)

Control when gesture training starts

The component begins training immediately upon rendering. To defer training (e.g., until a button click), wrap the component in a v-if directive.

<template>
  <button v-if="!cameraGesturesOn" @click="cameraGesturesOn=true">Start Training</button>
  <camera-gestures v-if="cameraGesturesOn"></camera-gestures>
</template>

<script>
export default {
  data () {
    return {
      cameraGesturesOn: false
    }
  }
}
</script>

Save and load trained gesture models

You can persist trained models to avoid requiring users to train gestures on every session.

Saving: Listen to the @doneTraining event, which provides the trained model object.
Loading: Pass a JSON model to the model prop. If a model is provided, the component skips training and verification.

<template>
  <!-- Saving model -->
  <camera-gestures @doneTraining="doneTraining"></camera-gestures>

  <!-- Loading model -->
  <camera-gestures v-if="modelLoaded" :model="model"></camera-gestures>
</template>

<script>
export default {
  data() {
    return { model: null, modelLoaded: false }
  },
  methods: {
    async doneTraining(model) {
      await this.$axios.post('myapi/saveModel', model)
    }
  },
  async mounted() {
    const res = await this.$axios.get('myapi/getModel')
    this.model = res.data
    this.modelLoaded = true
  }
}
</script>

Install vue-camera-gestures in Vue 3

Install the vue-camera-gestures package via npm and import the required CSS styles. The package bundles TensorFlow JS libraries, so peer dependency warnings can be ignored.

npm i vue-camera-gestures --save

// Global registration
import CameraGestures from 'vue-camera-gestures'
import 'vue-camera-gestures/dist/style.css'
app.component('camera-gestures', CameraGestures)

// OR Single File Component registration
import CameraGestures from 'vue-camera-gestures'
import 'vue-camera-gestures/dist/style.css'
export default {
  components: {
    CameraGestures
  }
}

Install vue-camera-gestures via CDN

For non-npm projects, include the CSS and JS from unpkg. It is recommended to use a specific version number in production.

<link rel="stylesheet" href="https://unpkg.com/vue-camera-gestures/dist/style.css">
<script src="https://unpkg.com/vue-camera-gestures"></script>

<script>
  const app = Vue.createApp({ /* ... */ })
  app.component('camera-gestures', CameraGesturesComponent)
</script>

Manage camera feed visibility
Control when the camera feed is visible using the following boolean props. Note that the video element remains in the DOM for TensorFlow requirements even when hidden.
```
<camera-gestures
  :showCameraFeedDuringTraining="false"
  :showCameraFeedDuringVerification="false"
  :showCameraFeedAfterTrainingCycle="false"
></camera-gestures>
```
https://vue3.cameragestures.com/guide/
Configure CameraGestures Vue 3 component props
The CameraGestures component for Vue 3 uses the following props to control the gesture training process, verification requirements, and UI behavior.
```
<CameraGestures 
  :doVerification="true"
  :fireOnce="true"
  :requiredAccuracy="90"
  :trainingTime="3000"
  :throttleEvents="100"
/>
```
https://vue3.cameragestures.com/api-reference/
Optimize bundle size and load time
The library is ~215KB (min+gzip) and loads a ~17MB Mobilenet model at runtime. To optimize performance:
1. Code-splitting: Use defineAsyncComponent to load the library only when needed.
2. Pre-loading: Call loadMobilenet() early in your application lifecycle to fetch the AI model before the user reaches the component.
```
// 1. Async Component
import { defineAsyncComponent } from 'vue'
const CameraGestures = defineAsyncComponent(() => import('vue-camera-gestures'))

// 2. Pre-loading the model
import { loadMobilenet } from 'vue-camera-gestures'
loadMobilenet() // returns a promise
```
https://vue3.cameragestures.com/guide/
Control gesture event frequency
By default, events fire once and require a return to the neutral position or a different gesture before firing again. Use fireOnce to enable continuous firing and throttleEvents to limit the frequency in milliseconds. These can be set globally or per-gesture within the gestures array.
```
<camera-gestures 
  :fireOnce="false" 
  :throttleEvents="1000" 
  :gestures="[
    {event: 'left', fireOnce: false, throttleEvent: 1000},
    {event: 'right', fireOnce: false, throttleEvent: 200}
  ]"
  @left="left()"
  @right="right()"
>
</camera-gestures>
```
https://vue3.cameragestures.com/guide/
Customize CameraGestures UI and prompts
Modify the visual feedback and text prompts during the training process:
- neutralTrainingPrompt (String, default: 'Maintain a neutral position'): Text shown during neutral position training.
- neutralVerificationPrompt (String, default: 'Verify neutral position'): Text shown during neutral position verification.
- trainingPromptPrefix (String, default: 'Perform a gesture: '): Prefix added to gesture names during training.
- verificationPromptPrefix (String, default: 'Verify gesture: '): Prefix added to gesture names during verification.
- showCameraFeedDuringTraining (Boolean, default: true): Toggle camera feed visibility during training.
- showCameraFeedDuringVerification (Boolean, default: true): Toggle camera feed visibility during verification.
- showCameraFeedAfterTrainingCycle (Boolean, default: true): Toggle camera feed visibility after the cycle completes.
https://vue3.cameragestures.com/api-reference/