Video Annotator guide

The Video Annotator is used to mark when each movement in a test video starts and ends (for example stand, pour water or drink) and saves those times in a YAML file. Each time is saved as real clock time, so it can be matched to the sensor recordings.

Overview

The annotator is a web page: open continuum-tools.pages.dev/annotator in Chrome or Edge. It is part of the same website as the Clinical Log, and there is nothing to install.

• Works with one video (for example an iPhone clip) or with a folder of synchronized cameras.
• Pick the test (the Condition) and the annotator offers the right movement labels for that test.
• Mark each movement frame by frame, or live while the video plays.
• Saves two YAML files per video: subject_XX_cond_YY_run_ZZ_events.yaml (the movements) and …_Info.yaml (Rec. start and fps).

The screen

On a narrow window the right-hand panels (4 and 5) move below the video.

Opening a video

One video (for example an iPhone clip)

1. Click Single video… and choose the file (.mov, .mp4…).
2. The file name appears in the grey pill above the video, Recording is filled with the file name, and Rec. start is read from the video itself when the file has it (see Rec. start).
3. Check the fps box (top right of the video). It is 30 by default; change it if the video was recorded at another frame rate.

A folder of synchronized cameras

1. Click Load folder… and choose the synchronized_videos folder of the recording. Chrome asks to confirm uploading the folder: this only lets the page read it, nothing is sent anywhere.
2. Every Camera_XXX_synchronized.mp4 becomes a camera. If the folder also has the …_timestamps_human_readable.csv files, the cameras are kept frame-synced and the fps is set from them (shown on the right, under the controls: synced · 3 cam · … fps).
3. Session and Recording are filled from the folder path.
4. Switch camera with the Cam 1 Cam 2… tabs or the keys 1–9. ▦ all cameras shows them all side by side; click one to make it the main camera.

Recording details

Fill in the Recording metadata panel before exporting. Subject, Condition and Run give the file its name, shown under the panel (Files export as subject_96_cond_09_run_01_events.yaml + _Info.yaml).

Rec. start and Offset

The annotator turns every marked frame into a real date and time, so that the movements can be found in the sensor data:

event time = Rec. start + time in the video − Offset

• Rec. start is the date and time of the video's first frame, written YYYY-MM-DD HH:MM:SS (e.g. 2026-10-01 10:15:30). It is read from the video file when it contains it, and the message under the panel says where it came from. iPhone videos store the real capture time.
• If the message says (UTC — verify), the time came from a generic field that is often in UTC or is the time the file was copied. Check it and correct it if needed.
• If it says No embedded recording time found, type Rec. start yourself.
• Offset fine-tunes the time, in seconds. Use it when the video clock is slightly off, for example if a movement that is clearly visible in the sensor data appears 0.4 s later in the video: set Offset to 0.4. A positive Offset moves every event earlier. Leave it at 0 if you don't know.

Moving through the video

Click or drag on the timeline (the bar under the video) to jump; hovering shows the time under the mouse. The large clock shows the current time in the video and the frame number.

Use the speed buttons (0.25× 0.5× 1× 2×) to slow the video down around a movement. Keyboard shortcuts do nothing while the cursor is in a text box: click on the video area first.

Marking a segment

A segment is one movement: a label with a start and an end. There are two ways to mark one.

Frame by frame (most precise)

1. Choose the Label (or type a custom label, which overrides the list).
2. Go to the first frame of the movement and press i (Mark start (i)).
3. Go to the last frame and press o (Mark end (o)).
4. Check the box under the buttons: label, start, end and duration. Use the small ‹ › buttons to move the start or end by one frame. The marked part is also highlighted on the timeline.
5. Optionally fill Trial / rep (e.g. 2 for the second repetition) and Notes.
6. Press Enter or Add segment ⏎.

Live, while the video plays

1. Choose the label and play the video (slow it down if it helps).
2. Click ● Live mark: start when the movement starts. The button turns red.
3. Click ■ Live mark: end when it ends. The segment is added straight away.

Live marking is quick but less precise; you can adjust the segment afterwards (see Checking and fixing).

Labels for each test

The labels offered depend on the Condition. If you need a label that is not in the list, type it in …or custom label.

Checking and fixing segments

Every segment appears in the Segments table (sorted by start time) and as a coloured bar on the timeline.

• Jump to a segment: click its start time in the table, or click its bar on the timeline.
• ▷ plays just that segment and stops at its end.
• ✎ edits it: the segment is loaded back into Mark a segment (its row turns grey). Change the label, start, end, trial or notes, then press Add segment ⏎ to save the change.
• ✕ deletes it.
• Clear deletes all segments (it asks first).

Saving: Export and Load YAML

YAML is the only file format the annotator uses. Each video is saved as two files that belong together:

Export YAML

1. Check Subject, Run, Condition and Rec. start.
2. Click Export YAML. The browser downloads both files. The first time, Chrome or Edge may ask whether the site can download multiple files: click Allow.
3. Move both files to Data/VIDEOS/SUBJECT_XX/annotations/, where the analysis notebook (video_annotations_to_runs.ipynb) looks for them.

Load YAML (continue or check later)

1. Open the same video (or folder) as before.
2. Click Load YAML and select both files of the video together (Ctrl+click). Subject, Condition and Run come back from the file name, Rec. start and fps from the _Info.yaml, and all segments, with their trial and notes, from the _events.yaml. If segments are already marked, it asks before replacing them.
3. Make your changes and Export YAML again (it overwrites the old files if you save them with the same names).

If you select only the _events.yaml, the annotator uses the Rec. start already in the panel (for example, read from the video). You can also load the files before the video; the segments are placed using the fps, and appear on the timeline once the video is open.

Old JSON files

Earlier versions of the annotator saved .json files. To turn one into YAML, click Convert it to YAML under the Segments table and choose the JSON. The annotator loads it and immediately downloads the _events.yaml and _Info.yaml files.

• If the JSON has no Rec. start, the segments are loaded but not exported: type Rec. start (or open the video) and click Export YAML.
• If the JSON was made from a multi-camera folder, open that folder first and then convert, so the times use the cameras' timestamp files.

The YAML files

You do not need to edit them, but this is what they contain.

_events.yaml

Under movements each label has its start and end times (in order, so the first start goes with the first end), plus the trial and notes of each occurrence. All times are clock times on the sensor clock.

movements:
  'pour water':
    movement_start:
    - '2026-10-01 10:15:35.000'
    movement_end:
    - '2026-10-01 10:15:39.000'
    trial:
    - '1'
    notes:
    - ''

The analysis notebook reads movement_start and movement_end; trial and notes are kept so that Load YAML restores everything.

_Info.yaml

rec_start: '2026-10-01 10:15:30.000'
rec_fps: 30

• rec_start: clock time of the video's first frame, with the Offset already applied (Rec. start − Offset). So when the file is loaded again, Offset is set back to 0, and the times are the same.
• rec_fps: frame rate of the video (the fps box).

Subject, condition and run are not repeated inside the files: they are in the file names.

Checklist

• Video plays (no warning on the video).
• Condition picked first; Subject and Run filled.
• Rec. start present and checked (especially if marked UTC — verify).
• Every movement marked; overlap warnings checked.
• Segments reviewed with ▷.
• Export YAML clicked and both files (_events.yaml, _Info.yaml) moved to Data/VIDEOS/SUBJECT_XX/annotations/.

Troubleshooting

Clinical Log users: see the Clinical Log guide.