What HaptiQ looks at in your video
HaptiQ turns a video into a script with computer-vision motion tracking: it measures how things move between frames and turns the up-and-down movement into positions from 0 to 100. It does not recognise people or objects.
The video is the “How HaptiQ sees it” view of a demo sample. Everything drawn on it was computed by the same pipeline run that produced the sample’s script. You can switch any demo sample to this view on the Demo page.
1. Frames, not footage
First the pipeline reads the video’s size, frame rate and length. Then it samples a frame about every 40 milliseconds, turns it gray and shrinks it to 160×120 pixels: the small picture in the corner of the video. For side-by-side or over-under VR video it analyses one eye, and it trims black bars.
2. Optical flow: how each pixel moved
Between each pair of frames, dense optical flow (the Farneback method) estimates how every pixel moved. Movement of the whole picture, such as a camera pan, shake or zoom, is fitted and removed, so what remains is the subject. Green arrows mark pixels clearly moving up or down; the amber box surrounds the pixels whose vertical movement is averaged into that frame’s motion signal.
3. From motion to position
The motion signal (the blue bars) is added up into a position, and each passage of a few seconds is scaled to use the full 0–100 range. Stretches without rhythmic movement, like the pause in this clip, hold still (grey bars), and a scene cut starts tracking afresh.
4. Cleaning up the script
Post-processing keeps the turning points and drops points that add nothing, limits speed by shortening strokes rather than delaying them, and keeps every position in range. The green line with dots is the final script, the same file you can download from the demo.
What it cannot see
It follows up-and-down movement of whatever moves most in the frame. Mostly sideways motion, movement hidden behind something, or several things moving in different directions can produce a weaker script. Review the result and edit a short section if it needs it.
Related
Videos and guides · All about funscripts · Create a free HaptiQ account