# Real-Time Motion Events

Events for building your own UI on top of KinesteX motion tracking: whether the user is in the picture and which way they face, framing hints, the start position, reps and holds, form, coaching cues, idle time and the hands gesture. State events fire when the state changes, never on every frame. They carry no camera frames or body coordinates (for those, see raw pose data on [Camera & Frame Events](/docs/data-points/camera-events)).

**Try them live:** the [Camera Events Lab](https://ai.kinestex.com/camera-events-lab.html) embeds the Camera component with your company name and API key, plays sample videos (or uses your camera), sends the commands with one click and lists every event with its payload.

**Where they are sent**

| Integration | Real-time motion events |
|-------------|-------------------------|
| [Camera component](/docs/integration/camera-component) | Always. The opt-in events need their flag (below) |
| Workouts, plans, personalized plans, custom workouts, challenges, AI Trainer workouts | Only when launched with `includePoseData` containing `"events"`, for example `includePoseData: ["events"]` |
| Assessments and games, Gym Log | Never |

**Launch options.** Pass them with your other launch parameters. Details: [Real-Time Motion parameters](/docs/customization-parameters/realtime-motion-params).

| Option | Default | Effect | Where |
|--------|---------|--------|-------|
| includePoseData: [..., "events"] | off | Turns on every event on this page | Workout flows (the Camera component always sends them) |
| includeFramingHints | `false` | Turns on `framing_hint` | Camera component; workouts with `"events"` |
| includeFormState | `false` | Turns on `form_state` | Camera component; workouts with `"events"` |
| inactivityTimeout | `15` | Seconds of idleness before `activity` `INACTIVE` | Both |
| enableGestureControl | `false` | Turns on `gesture_detected`. KinesteX reports the gesture; pausing is up to you | Camera component. Workouts pause on the gesture themselves and report it with `"events"` alone |
| frameExitPause | `false` | Pauses rep counting while the user is out of view (`presence` `LEFT`) and restarts a half-done rep on `RETURNED`. Off: counting goes on, only the events are sent | Camera component |
| frameExitPrompt | `false` | On `presence` `LEFT`, KinesteX (with its audio on) says "Pause activated. Get into the camera frame to resume the workout" | Camera component |

**Common fields.** Every event has `type` and `timestamp`. Every event except `framing_hint` and `gesture_detected` has `exercise`: the id you sent as `currentExercise` in the Camera component, or in a workout the exercise's id (the `exercise_id` of its `exercise_completed`). Events with variants name them in `kind` (what happened) or `state` (what the user's state became). More kinds and states may come: ignore the ones you don't know.

Exercise events run while an exercise is tracked. Camera component: from the exercise you send in `currentExercise` until `Pause Exercise` or `Stop Camera`; after `Workout Overview` only reps keep coming, until you send an exercise id. Workouts: during each exercise, not on rests or while paused. Never while the positioning check is open, the camera is off or the app is in the background.

**presence**

The user leaves the picture or comes back. Mid-exercise only.

| Field | Type | Description |
|-------|------|-------------|
| exercise | string | The exercise id |
| state | string | `LEFT`: the user was in view, then the camera showed nobody (or too little of the torso) for 3 s. `RETURNED`: back in view for 1 s. Every `LEFT` gets its `RETURNED` |

Camera component: sent straight after `left_camera_frame` / `returned_camera_frame` for the same moment, so handle one of the two. With `frameExitPause: true` rep counting pauses while the user is away; with `frameExitPrompt: true` KinesteX asks them to come back. Workouts: measured separately from the workout's own `left_camera_frame` / `returned_camera_frame` (which pause and resume the workout), so the two can differ; `frameExitPause` and `frameExitPrompt` do not apply.

```json
{ "type": "presence", "exercise": "16", "state": "LEFT", "timestamp": 1793368803120 }
```

**orientation_check**

The user turns toward or away from the exercise's `camera_view` (from `exercise_setup`), during the positioning check or mid-set. The first reading for each exercise is sent too. Only for exercises whose `camera_view` is `FRONT` or `SIDE`, and not while the camera is off.

| Field | Type | Description |
|-------|------|-------------|
| exercise | string | The exercise id |
| expected | string | The exercise's `camera_view`: `FRONT` or `SIDE` |
| detected | string | `FRONT` or `SIDE` once the new view held for 0.6 s; `UNKNOWN` when the shoulders or hips have been out of sight (or nobody detected) for 1.5 s |
| matches | boolean | `true` when `detected` is `expected` or one of the setup's `alternate_camera_views`. `false` for `UNKNOWN`, which says nothing about the way the user faces: don't prompt a turn on it |

**framing_hint** (opt-in: `includeFramingHints`)

While the positioning check is open, whenever what is out of view changes (after it held for 0.4 s). In the Camera component the check opens at the start when `showSilhouette` is on (the default), and after `reset_check_frame`. No `exercise` field.

| Field | Type | Description |
|-------|------|-------------|
| missing_body_parts | string[] | Any of `HEAD`, `TORSO`, `ARMS`, `LEGS`. Empty when everything is in |
| hint | string or null | `MOVE_TO_CENTER` when the body would fit but stands to one side; `STEP_BACK` when it is too close, cut off at the top or bottom, or nobody is detected; `null` when no move would help |

**start_position_reached**

The user settled (0.4 s) into the exercise's starting position: for rep exercises, the position the motion model completes its reps in; for timed holds, the hold position. A user who goes straight into fast reps gets it with the first counted rep. Sent again after anything that stops tracking (a pause, leaving the frame, the positioning check, the camera off, the app in the background) and after `reset_progress`. Field: `exercise`.

**action**

The user did something that counts toward the exercise. `kind` says what; more kinds may come. There is no `rep` event: reps arrive as `action` with `kind: "rep"`.

**`kind: "rep"`**, a rep is counted:

| Field | Type | Description |
|-------|------|-------------|
| exercise | string | The exercise id |
| value | number | The rep count for the exercise so far, from 1. Camera component: `reset_progress` starts it over. Workouts: the exercise's rep count in the workout |
| accuracy | number | The rep's accuracy score |
| rep_duration_ms | number, optional | Time since the previous rep, without the time tracking was stopped. Left out for the first rep of an exercise, the first after the user came back into the frame, the camera was turned back on, `Stop Camera`, `reset_check_frame`, `reset_progress` or an exercise switch, and the first after an idle stretch (`activity` `INACTIVE`) |

```json
{ "type": "action", "kind": "rep", "exercise": "16", "value": 3, "accuracy": 0.98, "rep_duration_ms": 2840, "timestamp": 1793368803120 }
```

**`kind: "hold"`**, a timed hold (plank, wall sit) starts, breaks or resumes. Start your hold timer on `STARTED` / `RESUMED`, freeze it on `BROKEN`.

| Field | Type | Description |
|-------|------|-------------|
| exercise | string | The exercise id (on a `BROKEN` caused by an exercise switch, the exercise the hold was in) |
| state | string | `STARTED` (first hold of the exercise, after 0.4 s in position), `BROKEN` (out of position for 0.8 s, tracking stopped, an exercise switch or `reset_progress`), `RESUMED` |
| held_ms | number | Total time held in the correct position in this exercise so far. Kept across exercise switches; `reset_progress` starts it over |

**successful_repeat** (legacy alias of `action` rep)

Every rep is also sent as `successful_repeat`, straight before its `action`, with exactly the same fields and `timestamp` (and no `kind`). Handle one of the two, not both, or every rep counts twice. New integrations should use `action`. Swift, Kotlin and Flutter type `successful_repeat` (`.reps` / `Reps`) while `action` arrives as `custom_type` / `CustomType`. In workouts, `successful_repeat` is sent only with `"events"` too.

**feedback**

KinesteX gives the user a coaching cue, whether its audio is on, paused or muted.

| Field | Type | Description |
|-------|------|-------------|
| exercise | string | The exercise id |
| kind | string | `mistake`: a form correction. `tip`: coaching that is not a correction, such as a model's tip cues and hold announcements |
| message | string | The cue text |
| feedback_id | string | Stable id for this cue within the exercise. Map it to your own copy or audio |
| body_region | string or null | `HEAD`, `SHOULDERS`, `ARMS`, `TORSO`, `HIPS`, `KNEES` or `FEET`, or `null` |
| count | number | Times this cue has been announced in the exercise, from 1. `reset_progress` starts it over |
| announce | boolean | `true`: a cue to voice now, at most once per cue per its repeat interval (what KinesteX's coach says). `false`: a live detection of a cue already announced, at most once a second per cue, with the count of its last announcement. Use it for highlights; don't voice it |

```json
{ "type": "feedback", "kind": "mistake", "exercise": "16", "message": "Keep your chest up", "feedback_id": "101", "body_region": "TORSO", "count": 2, "announce": true, "timestamp": 1793368803120 }
```

**mistake**

What KinesteX's coach says, tips included, in the shape older integrations know. Handle `feedback` or `mistake`, not both.

| Field | Type | Description |
|-------|------|-------------|
| exercise | string | The exercise id |
| value | string | The cue text, the phrase the coach says |
| mistake_id | string | The cue's id (`feedback_id` in `feedback`) |
| body_region | string or null | As in `feedback` |
| count | number | Times this cue has been sent as `mistake` in the exercise, from 1 |
| is_tip | boolean | `true` for a tip or hold announcement (`feedback` kind `tip`), `false` for a correction |

With KinesteX's voice on, `mistake` is sent as the coach starts saying the cue (a cue the voice never gets to is sent only as `feedback`). With nothing to hear (audio paused or muted, volume at zero, phrase audio missing), it is sent on detection, straight before its `feedback` with `announce: true`. Live detections go to `feedback` only. In the Camera component, `mistake` is also counted in the `mistakes` of `exercise_change` and `exercise_overview`. In workouts it is sent only with `"events"`, and the workout's own `mistakes` lists are unchanged.

**form_state** (opt-in: `includeFormState`)

Form quality changes, after holding for 0.3 s (`ADJUST` from a correction comes at once). `OUT_OF_POSITION` is also sent when tracking stops, if a state was reported before. Fields: `exercise`, `state`.

| state | Rep exercises | Timed holds |
|-------|---------------|-------------|
| GOOD | In the exercise, no correction showing | In the hold position, correct form |
| ADJUST | The user holds a position the model corrects, or KinesteX just corrected the rep just done (the `feedback` kind `mistake` sent right before it); stays until the next rep without a correction | In the hold position with a form issue |
| OUT_OF_POSITION | Start position not reached yet, or tracking stopped | Not in the hold position, or tracking stopped |

**activity**

The user goes idle, or moves again after being idle. Sent once per idle stretch.

| Field | Type | Description |
|-------|------|-------------|
| exercise | string | The exercise id |
| state | string | `INACTIVE`: in view and idle for `inactivityTimeout` seconds (default 15): still, no counted rep, no feedback from KinesteX, not in a hold's position. `ACTIVE`: the stretch ended (a real move, a rep, the hold position, feedback), and also on any tracking stop, exercise switch or `reset_progress`, so every `INACTIVE` gets its `ACTIVE` |
| idle_seconds | number | Idle seconds so far (for `ACTIVE`, the total before the user moved again) |

**gesture_detected**

The user made the hands-free gesture (arms down by the sides, then forearms crossed in front of the chest). No `exercise` field.

| Field | Type | Description |
|-------|------|-------------|
| gesture | string | `PAUSE` or `RESUME` |

Camera component (opt-in: `enableGestureControl`): sent when the gesture is seen while tracking; it alternates `PAUSE`, `RESUME`, and KinesteX does not pause, that is up to you. While you have the exercise paused (`Pause Exercise`) the next gesture is `RESUME`. Workouts (with `"events"`): sent when the workout itself pauses or resumes on the gesture.

**exercise_setup**

How an exercise starts and where the camera goes, from the exercise's motion model.

| Field | Type | Description |
|-------|------|-------------|
| exercise | string | The exercise id |
| tracked | boolean | `false` when KinesteX has no motion model for it; every field below is then `null` or `[]` |
| start_position | string or null | The model's start position, for example `"PLANK"` or `"ALL_FOURS"` |
| camera_view | string or null | The view the camera should have of the user: `orientation_check` compares against `FRONT` and `SIDE` |
| alternate_camera_views | string[] | Other views that also work |
| camera_side | string or null | The model's camera side |

Camera component: once per `currentExercise` switch (resuming the same exercise after `Pause Exercise` sends nothing), after its model has loaded; `tracked: false` straight away for an id that is not in `exercises`, and when the model fails to load. Workouts (with `"events"`): when each exercise in the list starts.

**get_exercise_setup** (Camera component): send `{ workout_activity_action: "get_exercise_setup", exercises: ["16", "21"] }` to get one `exercise_setup` per id now (an id whose model is still loading is answered when it loads). Ids you never sent in `exercises` or `load_models` get no event, only one `error_occurred` warning naming them. The command needs a message with two keys: Swift `WebViewState.sendMessage`, Kotlin `KinesteXSDK.sendMessage`, React Native `sendAction("workout_activity_action", "get_exercise_setup", { exercises })`, or `postMessage` in HTML/JS. Flutter's and React's `sendAction` cannot attach `exercises`, so they cannot send it.

**What stays the workout's own**

With `"events"` in a workout:
- Leaving the picture: the workout keeps sending its own `left_camera_frame` / `returned_camera_frame` and pausing as before; `presence` is added, not a second `left_camera_frame`.
- The hands gesture pauses the workout itself (`enableGestureControl` is not needed); `gesture_detected` reports it.
- `exercise_completed`, `workout_overview` and `exercise_overview` are unchanged. The Camera component's `exercise_change` and its per-command overviews are not sent.
- Events follow the exercise being tracked: nothing on a rest or while the workout is paused (apart from `presence` while it is paused for leaving the frame), and an open hold or idle stretch closes (`action` hold `BROKEN`, `activity` `ACTIVE`) when the exercise ends, the workout pauses, or before `workout_ended`.
- Cue counts, `held_ms` and rep values are per exercise id, as in the Camera component.

**Handling Real-Time Motion Events**

_Swift (iOS)_
```swift
func handle(_ message: KinestexMessage) {
    // These events have no case of their own: they arrive as .custom_type
    guard case .custom_type(let json) = message, let type = json["type"] as? String else { return }
    switch type {
    case "action":
        if json["kind"] as? String == "rep" {
            print("Rep \(json["value"] ?? 0) of \(json["exercise"] ?? "")")
        } else if json["kind"] as? String == "hold" {
            print("Hold \(json["state"] ?? ""), held \(json["held_ms"] ?? 0) ms")
        }
    case "presence":
        print("User \(json["state"] ?? "")") // "LEFT" or "RETURNED"
    case "feedback":
        if json["announce"] as? Bool == true {
            print("Coach: \(json["message"] ?? "")")
        }
    case "activity":
        print("Activity: \(json["state"] ?? "")")
    default:
        break // ignore anything else, including kinds and states added later
    }
}
```

_Kotlin (Android)_
```kotlin
private fun handleWebViewMessage(message: WebViewMessage) {
    // These events have no class of their own: they arrive as CustomType
    if (message !is WebViewMessage.CustomType) return
    val event = message.data
    when (event["type"]) {
        "action" -> {
            when (event["kind"]) {
                "rep" -> println("Rep ${event["value"]} of ${event["exercise"]}")
                "hold" -> println("Hold ${event["state"]}, held ${event["held_ms"]} ms")
                else -> Unit
            }
        }
        "presence" -> println("User ${event["state"]}") // "LEFT" or "RETURNED"
        "feedback" -> {
            if (event["announce"] == true) println("Coach: ${event["message"]}")
        }
        "activity" -> println("Activity: ${event["state"]}")
        else -> Unit // ignore anything else, including kinds and states added later
    }
}
```

_Flutter_
```dart
void handleWebViewMessage(WebViewMessage message) {
  // These events have no class of their own: they arrive as CustomType
  if (message is! CustomType) return;
  final event = message.data;
  switch (event['type']) {
    case 'action':
      if (event['kind'] == 'rep') {
        print("Rep ${event['value']} of ${event['exercise']}");
      } else if (event['kind'] == 'hold') {
        print("Hold ${event['state']}, held ${event['held_ms']} ms");
      }
      break;
    case 'presence':
      print("User ${event['state']}"); // "LEFT" or "RETURNED"
      break;
    case 'feedback':
      if (event['announce'] == true) print("Coach: ${event['message']}");
      break;
    case 'activity':
      print("Activity: ${event['state']}");
      break;
    default:
      break; // ignore anything else, including kinds and states added later
  }
}
```

_React Native_
```jsx
const handleMessage = (type: string, data: { [key: string]: any }) => {
  switch (type) {
    case "orientation_check":
      if (data.detected !== "UNKNOWN") setTurnPrompt(data.matches ? null : data.expected);
      break;
    case "start_position_reached":
      startCountdown();
      break;
    case "action":
      if (data.kind === "rep") {
        setRepCount(data.value);
      } else if (data.kind === "hold") {
        data.state === "BROKEN" ? freezeHoldTimer(data.held_ms) : runHoldTimer(data.held_ms);
      }
      break;
    case "feedback":
      if (data.kind === "mistake") highlight(data.body_region);
      if (data.announce) playCoachLine(data.feedback_id);
      break;
    case "presence":
      if (data.state === "LEFT") pauseWorkoutTimer();
      if (data.state === "RETURNED") resumeWorkoutTimer();
      break;
    case "activity":
      if (data.state === "INACTIVE") openHelpSheet();
      if (data.state === "ACTIVE") closeHelpSheet();
      break;
    default:
      break; // ignore anything else, including kinds and states added later
  }
};
```

_React (TypeScript)_
```tsx
const handleMessage = (type: string, data: { [key: string]: any }) => {
  switch (type) {
    case "orientation_check":
      if (data.detected !== "UNKNOWN") setTurnPrompt(data.matches ? null : data.expected);
      break;
    case "start_position_reached":
      startCountdown();
      break;
    case "action":
      if (data.kind === "rep") {
        setRepCount(data.value);
      } else if (data.kind === "hold") {
        data.state === "BROKEN" ? freezeHoldTimer(data.held_ms) : runHoldTimer(data.held_ms);
      }
      break;
    case "feedback":
      if (data.kind === "mistake") highlight(data.body_region);
      if (data.announce) playCoachLine(data.feedback_id);
      break;
    case "presence":
      if (data.state === "LEFT") pauseWorkoutTimer();
      if (data.state === "RETURNED") resumeWorkoutTimer();
      break;
    case "activity":
      if (data.state === "INACTIVE") openHelpSheet();
      if (data.state === "ACTIVE") closeHelpSheet();
      break;
    default:
      break; // ignore anything else, including kinds and states added later
  }
};
```

_HTML / JavaScript_
```html
// Inside your window "message" listener, after JSON.parse
switch (message.type) {
  case "action":
    if (message.kind === "rep") showReps(message.value);
    else if (message.kind === "hold") showHold(message.state, message.held_ms);
    break;
  case "presence":
    showAwayBanner(message.state === "LEFT");
    break;
  case "feedback":
    if (message.announce) showCue(message.message, message.body_region);
    break;
  case "activity":
    showIdleHelp(message.state === "INACTIVE");
    break;
  default:
    break; // ignore anything else, including kinds and states added later
}
```

---
Source: https://www.kinestex.com/docs/data-points/realtime-motion-events · Index: https://www.kinestex.com/llms.txt