Camera & Frame Events
Events about the camera, the positioning check, the user's place in the frame and the Camera component's commands. The real-time events for custom UIs (presence, start position, reps, holds, form, feedback, idle time) are on Real-Time Motion Events.
Positioning and frame
| Event | Data Fields | Description |
| person_in_frame | message: string ("Person stepped into frame") | The user passed the positioning check and tracking starts (workout flows and the Camera component, also after reset_check_frame) |
| check_frame_skipped | message: string ("User skipped check frame") | User tapped Skip on the workout positioning screen |
| left_camera_frame | date: string ("DD MM YYYY HH:MM:SS"), exercise?: string | The user left the picture. Workout flows: the workout pauses and plays "Pause activated..." (no exercise field). Camera component: sent with exercise, straight before presence LEFT |
| returned_camera_frame | date: string, exercise?: string | The user came back. Workout flows: only after a pause caused by leaving the frame (not one the user or the hands gesture made); the workout resumes. Camera component: sent with exercise, straight before presence RETURNED |
In the Camera component, left_camera_frame / returned_camera_frame and presence report the same moments: handle one of the two. In workouts they come from the workout's own pause logic, while presence (with includePoseData "events") is measured separately, so the two can fire at different moments, or one without the other.
Camera selection
| Event | Data Fields | Description |
| camera_selected | id: string, label: string, isMirrorCamera?: boolean | The camera stream opened (start, return from background, after a switch, after resume_camera), in workout flows, the Camera component, assessments and games. Also sent when the user picks a camera in the selector, without isMirrorCamera |
| camera_selector_opened | message: Array<{ deviceId: string, label: string }> | Camera selector opened, with the available cameras |
| camera_selector_closed | id: string or null, isMirrorCamera: boolean | Camera selector closed; the camera and mirror setting now in use |
| open_camera_settings | - | User tapped "Go to Settings" in the camera help screen. Only shown with shouldShowOpenCameraSettings: true. Open the device camera settings from your app |
Camera component: commands and results
| Event | Data Fields | Description |
| exercise_change | prevExerciseId: string, reps: number, timeSpent: number, averageAccuracy: number, mistakes: Array<{ mistake, count }>, accuracyReps: number[], calories: number | You switched currentExercise to another exercise: the totals of the one before. Top-level fields, no data. timeSpent in seconds since that exercise started |
| exercise_overview | data: Array<{ exerciseId, reps, timeSpent, averageAccuracy, mistakes, accuracyReps, calories }> | After currentExercise: "Workout Overview": one entry per exercise tracked |
| workout_overview | data: { totalReps, totalTimeSpent, totalAverageAccuracy, totalCalories } | Right after that exercise_overview. When nothing was tracked yet, only this event is sent, with all four at 0 |
| stop_camera | message: string ("Camera stopped") | After currentExercise: "Stop Camera" |
| reset_progress | exerciseId: string, date: string | Echo of reset_progress: the current exercise starts over |
| reset_all_progress | exerciseId: string or null, date: string | Echo of reset_all_progress: every exercise starts over |
| reset_check_frame | date: string | Echo of reset_check_frame. person_in_frame follows when the user is back in position |
| pause_camera / resume_camera | date: string | Echo, every time (also when sent before the camera page opened). A resume_camera that turns the camera back on is followed by camera_selected. Other integrations answer with an error_occurred warning |
| mute_workout / unmute_workout | date: string | Echo. A command sent before the camera page opens is echoed when it opens, and two identical ones sent before that give one echo |
exercise_setup (on each exercise switch and in reply to get_exercise_setup) is on Real-Time Motion Events; models_loaded and model_warmedup on Application Lifecycle. "Pause Exercise", "Pause Audio" and "Resume Audio" send no event of their own. pause_workout, resume_workout, mute_speech and unmute_speech are ignored by the Camera component (no echo, no error). The command errors ("<command>: camera is not active", "reset_progress: no active exercise") are on Error & Status Events.
Depth guidance and accuracy
| Event | Data Fields | Description |
| form_guidance_config | id, source, combine, smoothing?, angles[] | Depth-guidance setup for the current exercise. Only with streamFormGuidance: true |
| form_guidance_frame | id, fill, target, tol, neutral, zone, angles | Live depth reading, sent when it changes. Only with streamFormGuidance: true |
| rep_depth_angle | id, rep, combine, angles[] | Deepest joint angle reached in each rep, whenever depth guidance is active: by default in workout flows (showFormGuidance defaults to true everywhere except the Camera component), for exercises whose model has depth guidance. In the Camera component, with showFormGuidance or streamFormGuidance |
| correct_position_accuracy | accuracy: number | Every frame the motion model matches a position: how close the user is to the correct one (0 when the matched position is not a correct one). Workout flows and the Camera component, only with includeRealtimeAccuracy: true |
Depth guidance stream. id is the exercise id (the workout exercise's id, or currentExercise in the Camera component). fill, target and neutral are 0-1 positions on the depth bar (0 = standing, 1 = deepest). tol is the half-width of the "perfect" band on the same scale (target plus or minus tol). zone is "neutral", "approaching", "perfect" or "too_deep". angles in a frame maps each joint id to its measured angle in degrees. Each entry of angles[] in the other two events describes one tracked joint: id, joint (knee, hip, elbow, shoulder, ankle), side, source ("2d" or "3d"), topAngle, neutralAngle, targetAngle, tolerance, bottomAngle and an optional weight. rep_depth_angle adds deepestAngle. See showFormGuidance and streamFormGuidance in Camera & Pose Detection.
Raw pose data (`includePoseData`)
Off by default. Pass includePoseData as a list (or a comma-separated string) with the values you need. It works in the Camera component and in workout flows (workouts, plans, personalized plans, custom workouts, challenges).
| Value | Event | Description |
"poseLandmarks" | pose_landmarks | poseLandmarks: the 33 MediaPipe landmarks as named keys (nose, leftShoulder, rightKnee, ...) under coordinates, each { x, y, z, visibility }, normalized to the picture (x flipped when the camera is mirrored, which is the default) |
"worldLandmarks" | world_landmarks | worldLandmarks: the same object in metres, centred between the hips |
"angles" | inside both | Fills angles3D (joint angles in 3D) |
"events" | see Real-Time Motion Events | Workout flows only: the Camera component's real-time events (the Camera component always sends them) |
Both landmark objects always carry angles2D and angles3D (leftKneeAngle, rightKneeAngle, hips, shoulders, elbows, wrists, ankles and armpits, left and right); values are 0 when not computed. angles3D is computed with "angles". angles2D is computed in workouts, and in the Camera component with "angles" or enableGestureControl. Events are sent for every frame with a person in it (up to 30 a second), so ask only for what you process. In workouts each event also has exercise: the exercise id the frame belongs to (the exercise_id of its exercise_completed), null during the positioning check; frames are sent during the positioning check and the exercises, not on rest screens. The Camera component's events have no exercise field.
1{ "type": "pose_landmarks", "exercise": "101", "poseLandmarks": { "coordinates": { "nose": { "x": 0.51, "y": 0.18, "z": -0.3, "visibility": 0.99 } }, "angles2D": { "leftKneeAngle": 172 }, "angles3D": { "leftKneeAngle": 0 } }, "timestamp": 1793368803120 }