Real-Time Motion Events
Events for building your own UI on top of KinesteX motion tracking: whether the user is in the picture and which way they face, framing hints, the start position, reps and holds, form, coaching cues, idle time and the hands gesture. State events fire when the state changes, never on every frame. They carry no camera frames or body coordinates (for those, see raw pose data on Camera & Frame Events).
Try them live: the Camera Events Lab embeds the Camera component with your company name and API key, plays sample videos (or uses your camera), sends the commands with one click and lists every event with its payload.
Where they are sent
| Integration | Real-time motion events |
| Camera component | Always. The opt-in events need their flag (below) |
| Workouts, plans, personalized plans, custom workouts, challenges, AI Trainer workouts | Only when launched with includePoseData containing "events", for example includePoseData: ["events"] |
| Assessments and games, Gym Log | Never |
Launch options. Pass them with your other launch parameters. Details: Real-Time Motion parameters.
| Option | Default | Effect | Where |
| includePoseData: [..., "events"] | off | Turns on every event on this page | Workout flows (the Camera component always sends them) |
| includeFramingHints | false | Turns on framing_hint | Camera component; workouts with "events" |
| includeFormState | false | Turns on form_state | Camera component; workouts with "events" |
| inactivityTimeout | 15 | Seconds of idleness before activity INACTIVE | Both |
| enableGestureControl | false | Turns on gesture_detected. KinesteX reports the gesture; pausing is up to you | Camera component. Workouts pause on the gesture themselves and report it with "events" alone |
| frameExitPause | false | Pauses rep counting while the user is out of view (presence LEFT) and restarts a half-done rep on RETURNED. Off: counting goes on, only the events are sent | Camera component |
| frameExitPrompt | false | On presence LEFT, KinesteX (with its audio on) says "Pause activated. Get into the camera frame to resume the workout" | Camera component |
Common fields. Every event has type and timestamp. Every event except framing_hint and gesture_detected has exercise: the id you sent as currentExercise in the Camera component, or in a workout the exercise's id (the exercise_id of its exercise_completed). Events with variants name them in kind (what happened) or state (what the user's state became). More kinds and states may come: ignore the ones you don't know.
Exercise events run while an exercise is tracked. Camera component: from the exercise you send in currentExercise until Pause Exercise or Stop Camera; after Workout Overview only reps keep coming, until you send an exercise id. Workouts: during each exercise, not on rests or while paused. Never while the positioning check is open, the camera is off or the app is in the background.
presence
The user leaves the picture or comes back. Mid-exercise only.
| Field | Type | Description |
| exercise | string | The exercise id |
| state | string | LEFT: the user was in view, then the camera showed nobody (or too little of the torso) for 3 s. RETURNED: back in view for 1 s. Every LEFT gets its RETURNED |
Camera component: sent straight after left_camera_frame / returned_camera_frame for the same moment, so handle one of the two. With frameExitPause: true rep counting pauses while the user is away; with frameExitPrompt: true KinesteX asks them to come back. Workouts: measured separately from the workout's own left_camera_frame / returned_camera_frame (which pause and resume the workout), so the two can differ; frameExitPause and frameExitPrompt do not apply.
1{ "type": "presence", "exercise": "16", "state": "LEFT", "timestamp": 1793368803120 }orientation_check
The user turns toward or away from the exercise's camera_view (from exercise_setup), during the positioning check or mid-set. The first reading for each exercise is sent too. Only for exercises whose camera_view is FRONT or SIDE, and not while the camera is off.
| Field | Type | Description |
| exercise | string | The exercise id |
| expected | string | The exercise's camera_view: FRONT or SIDE |
| detected | string | FRONT or SIDE once the new view held for 0.6 s; UNKNOWN when the shoulders or hips have been out of sight (or nobody detected) for 1.5 s |
| matches | boolean | true when detected is expected or one of the setup's alternate_camera_views. false for UNKNOWN, which says nothing about the way the user faces: don't prompt a turn on it |
framing_hint (opt-in: includeFramingHints)
While the positioning check is open, whenever what is out of view changes (after it held for 0.4 s). In the Camera component the check opens at the start when showSilhouette is on (the default), and after reset_check_frame. No exercise field.
| Field | Type | Description |
| missing_body_parts | string[] | Any of HEAD, TORSO, ARMS, LEGS. Empty when everything is in |
| hint | string or null | MOVE_TO_CENTER when the body would fit but stands to one side; STEP_BACK when it is too close, cut off at the top or bottom, or nobody is detected; null when no move would help |
start_position_reached
The user settled (0.4 s) into the exercise's starting position: for rep exercises, the position the motion model completes its reps in; for timed holds, the hold position. A user who goes straight into fast reps gets it with the first counted rep. Sent again after anything that stops tracking (a pause, leaving the frame, the positioning check, the camera off, the app in the background) and after reset_progress. Field: exercise.
action
The user did something that counts toward the exercise. kind says what; more kinds may come. There is no rep event: reps arrive as action with kind: "rep".
`kind: "rep"`, a rep is counted:
| Field | Type | Description |
| exercise | string | The exercise id |
| value | number | The rep count for the exercise so far, from 1. Camera component: reset_progress starts it over. Workouts: the exercise's rep count in the workout |
| accuracy | number | The rep's accuracy score |
| rep_duration_ms | number, optional | Time since the previous rep, without the time tracking was stopped. Left out for the first rep of an exercise, the first after the user came back into the frame, the camera was turned back on, Stop Camera, reset_check_frame, reset_progress or an exercise switch, and the first after an idle stretch (activity INACTIVE) |
1{ "type": "action", "kind": "rep", "exercise": "16", "value": 3, "accuracy": 0.98, "rep_duration_ms": 2840, "timestamp": 1793368803120 }`kind: "hold"`, a timed hold (plank, wall sit) starts, breaks or resumes. Start your hold timer on STARTED / RESUMED, freeze it on BROKEN.
| Field | Type | Description |
| exercise | string | The exercise id (on a BROKEN caused by an exercise switch, the exercise the hold was in) |
| state | string | STARTED (first hold of the exercise, after 0.4 s in position), BROKEN (out of position for 0.8 s, tracking stopped, an exercise switch or reset_progress), RESUMED |
| held_ms | number | Total time held in the correct position in this exercise so far. Kept across exercise switches; reset_progress starts it over |
successful_repeat (legacy alias of action rep)
Every rep is also sent as successful_repeat, straight before its action, with exactly the same fields and timestamp (and no kind). Handle one of the two, not both, or every rep counts twice. New integrations should use action. Swift, Kotlin and Flutter type successful_repeat (.reps / Reps) while action arrives as custom_type / CustomType. In workouts, successful_repeat is sent only with "events" too.
feedback
KinesteX gives the user a coaching cue, whether its audio is on, paused or muted.
| Field | Type | Description |
| exercise | string | The exercise id |
| kind | string | mistake: a form correction. tip: coaching that is not a correction, such as a model's tip cues and hold announcements |
| message | string | The cue text |
| feedback_id | string | Stable id for this cue within the exercise. Map it to your own copy or audio |
| body_region | string or null | HEAD, SHOULDERS, ARMS, TORSO, HIPS, KNEES or FEET, or null |
| count | number | Times this cue has been announced in the exercise, from 1. reset_progress starts it over |
| announce | boolean | true: a cue to voice now, at most once per cue per its repeat interval (what KinesteX's coach says). false: a live detection of a cue already announced, at most once a second per cue, with the count of its last announcement. Use it for highlights; don't voice it |
1{ "type": "feedback", "kind": "mistake", "exercise": "16", "message": "Keep your chest up", "feedback_id": "101", "body_region": "TORSO", "count": 2, "announce": true, "timestamp": 1793368803120 }mistake
What KinesteX's coach says, tips included, in the shape older integrations know. Handle feedback or mistake, not both.
| Field | Type | Description |
| exercise | string | The exercise id |
| value | string | The cue text, the phrase the coach says |
| mistake_id | string | The cue's id (feedback_id in feedback) |
| body_region | string or null | As in feedback |
| count | number | Times this cue has been sent as mistake in the exercise, from 1 |
| is_tip | boolean | true for a tip or hold announcement (feedback kind tip), false for a correction |
With KinesteX's voice on, mistake is sent as the coach starts saying the cue (a cue the voice never gets to is sent only as feedback). With nothing to hear (audio paused or muted, volume at zero, phrase audio missing), it is sent on detection, straight before its feedback with announce: true. Live detections go to feedback only. In the Camera component, mistake is also counted in the mistakes of exercise_change and exercise_overview. In workouts it is sent only with "events", and the workout's own mistakes lists are unchanged.
form_state (opt-in: includeFormState)
Form quality changes, after holding for 0.3 s (ADJUST from a correction comes at once). OUT_OF_POSITION is also sent when tracking stops, if a state was reported before. Fields: exercise, state.
| state | Rep exercises | Timed holds |
| GOOD | In the exercise, no correction showing | In the hold position, correct form |
| ADJUST | The user holds a position the model corrects, or KinesteX just corrected the rep just done (the feedback kind mistake sent right before it); stays until the next rep without a correction | In the hold position with a form issue |
| OUT_OF_POSITION | Start position not reached yet, or tracking stopped | Not in the hold position, or tracking stopped |
activity
The user goes idle, or moves again after being idle. Sent once per idle stretch.
| Field | Type | Description |
| exercise | string | The exercise id |
| state | string | INACTIVE: in view and idle for inactivityTimeout seconds (default 15): still, no counted rep, no feedback from KinesteX, not in a hold's position. ACTIVE: the stretch ended (a real move, a rep, the hold position, feedback), and also on any tracking stop, exercise switch or reset_progress, so every INACTIVE gets its ACTIVE |
| idle_seconds | number | Idle seconds so far (for ACTIVE, the total before the user moved again) |
gesture_detected
The user made the hands-free gesture (arms down by the sides, then forearms crossed in front of the chest). No exercise field.
| Field | Type | Description |
| gesture | string | PAUSE or RESUME |
Camera component (opt-in: enableGestureControl): sent when the gesture is seen while tracking; it alternates PAUSE, RESUME, and KinesteX does not pause, that is up to you. While you have the exercise paused (Pause Exercise) the next gesture is RESUME. Workouts (with "events"): sent when the workout itself pauses or resumes on the gesture.
exercise_setup
How an exercise starts and where the camera goes, from the exercise's motion model.
| Field | Type | Description |
| exercise | string | The exercise id |
| tracked | boolean | false when KinesteX has no motion model for it; every field below is then null or [] |
| start_position | string or null | The model's start position, for example "PLANK" or "ALL_FOURS" |
| camera_view | string or null | The view the camera should have of the user: orientation_check compares against FRONT and SIDE |
| alternate_camera_views | string[] | Other views that also work |
| camera_side | string or null | The model's camera side |
Camera component: once per currentExercise switch (resuming the same exercise after Pause Exercise sends nothing), after its model has loaded; tracked: false straight away for an id that is not in exercises, and when the model fails to load. Workouts (with "events"): when each exercise in the list starts.
get_exercise_setup (Camera component): send { workout_activity_action: "get_exercise_setup", exercises: ["16", "21"] } to get one exercise_setup per id now (an id whose model is still loading is answered when it loads). Ids you never sent in exercises or load_models get no event, only one error_occurred warning naming them. The command needs a message with two keys: Swift WebViewState.sendMessage, Kotlin KinesteXSDK.sendMessage, React Native sendAction("workout_activity_action", "get_exercise_setup", { exercises }), or postMessage in HTML/JS. Flutter's and React's sendAction cannot attach exercises, so they cannot send it.
What stays the workout's own
With "events" in a workout:
- Leaving the picture: the workout keeps sending its own
left_camera_frame/returned_camera_frameand pausing as before;presenceis added, not a secondleft_camera_frame. - The hands gesture pauses the workout itself (
enableGestureControlis not needed);gesture_detectedreports it. exercise_completed,workout_overviewandexercise_overvieware unchanged. The Camera component'sexercise_changeand its per-command overviews are not sent.- Events follow the exercise being tracked: nothing on a rest or while the workout is paused (apart from
presencewhile it is paused for leaving the frame), and an open hold or idle stretch closes (actionholdBROKEN,activityACTIVE) when the exercise ends, the workout pauses, or beforeworkout_ended. - Cue counts,
held_msand rep values are per exercise id, as in the Camera component.
1func handle(_ message: KinestexMessage) {
2 // These events have no case of their own: they arrive as .custom_type
3 guard case .custom_type(let json) = message, let type = json["type"] as? String else { return }
4 switch type {
5 case "action":
6 if json["kind"] as? String == "rep" {
7 print("Rep \(json["value"] ?? 0) of \(json["exercise"] ?? "")")
8 } else if json["kind"] as? String == "hold" {
9 print("Hold \(json["state"] ?? ""), held \(json["held_ms"] ?? 0) ms")
10 }
11 case "presence":
12 print("User \(json["state"] ?? "")") // "LEFT" or "RETURNED"
13 case "feedback":
14 if json["announce"] as? Bool == true {
15 print("Coach: \(json["message"] ?? "")")
16 }
17 case "activity":
18 print("Activity: \(json["state"] ?? "")")
19 default:
20 break // ignore anything else, including kinds and states added later
21 }
22}