How to Search a Video by Face — A Step-by-Step Guide
Searching a video by face means sampling frames, encoding the faces in each frame, and comparing them against a reference photo. Here is how the process works — and how to tune it for your footage.
Published 2026-10-08 6 min read
- video search
- how it works
- face matching
Searching a video by face is not the same as searching text. There is no transcript to grep, no metadata to filter on. The only reliable signal is the face itself — and turning video into something you can search by face requires a specific pipeline: sample frames, detect faces, encode them, compare against a reference.
This guide walks through how that pipeline works in practice, what each step does, and which settings you can tune when the default does not fit your footage.
What “searching a video by face” actually means
At its core, searching a video by face is the same problem as searching a photo library by face — the only difference is that video gives you thousands of images to look at instead of one. The practical approach is to reduce that stream to a manageable set of moments:
- Sample frames at a regular interval (roughly one per second by default).
- Detect faces in each sampled frame.
- Encode each face into a numeric representation (an embedding).
- Compare each embedding against the embedding of your reference photo.
- Merge consecutive hits into time ranges so you get “02:14–02:34” instead of 20 separate frame-level results.
That last step is what separates a usable video search from a firehose of individual frame matches. Without merging, a person who stays in shot for 30 seconds produces 30 results. With merging, it is one.
How frame sampling works
Video is a sequence of images shown fast enough to look continuous. To search it by face, you do not need to look at every single frame — 24 or 30 per second is far more than face search requires.
SnapByFace lets you set the sampling interval: how often it pauses to examine a frame.
- 0.1 seconds — examines roughly every frame. Use this for short clips where a split-second appearance matters.
- 1 second (default) — the sweet spot for most footage. Dense enough to catch someone walking through a scene, sparse enough that a two-hour video finishes in minutes.
- Up to 30 seconds — for very long recordings where you only need to know whether someone appears at all, not exactly when.
A tighter interval means more frames examined, which means more time spent. The trade-off is precision versus speed, and the right answer depends on your footage.
Face detection and encoding
Once a frame is sampled, two things happen in sequence:
Detection finds where in the frame a face is. This is the step that says “there is a face here, at these coordinates.” Modern detectors are fast and accurate for faces that are reasonably sized and not severely occluded.
Encoding takes the detected face and turns it into a fixed-length numeric vector — an embedding. Two photos of the same person produce similar vectors; two photos of different people produce dissimilar ones. The encoding is what makes comparison possible: instead of comparing pixels, SnapByFace compares vectors.
Both steps use models that ship inside the app. Nothing is downloaded during a search, and nothing is sent to a server.
Matching faces against a reference
When you add a reference photo of a person, SnapByFace encodes that face the same way it encodes faces found in video. Every face found in the footage is then compared against that reference embedding.
The result of each comparison is a similarity score — a number between 0 and 1 that says how close the two faces are. Higher means more similar.
SnapByFace then filters results by a similarity threshold (default 0.45, adjustable from 0.30 to 0.80):
- Lower threshold (0.30–0.40) — more results, including people who look somewhat similar. Useful when you are not sure what the person looks like in the footage, or when appearance has changed significantly.
- Default (0.45) — works well for most cases where the reference photo is a reasonable match.
- Higher threshold (0.55–0.80) — fewer results, but each one is a stronger match. Use this when you want high confidence and can afford to miss borderline cases.
Tuning the search for your footage
Two knobs control the search: sampling interval and similarity threshold. They are independent and you set both per scan.
| Scenario | Sampling interval | Similarity threshold |
|---|---|---|
| Short clip, brief appearance | 0.1–0.5 s | 0.40 (default-ish) |
| General-purpose search | 1 s (default) | 0.45 (default) |
| Hours of footage, just checking | 5–30 s | 0.45 (default) |
| Person looks very different from reference | 1 s | 0.30–0.35 |
| High-confidence results only | 1 s | 0.55–0.65 |
If the first scan does not give you what you expect, adjust one knob at a time and re-run. Incremental re-indexing means only new or changed files are re-analyzed — re-running a search with a different threshold is instant because the frame-level data is already indexed.
Supported video formats
SnapByFace reads the most common container formats:
- MP4 — the default for most cameras, phones and screen recordings
- MOV — Apple’s container, common in iPhone footage and professional cameras
- AVI — older Windows format, still shows up in legacy archives
- MKV — common for high-quality encodes and multi-audio tracks
- M4V — iTunes / Apple variant of MP4
- WebM — web-native format, common for browser recordings
- TS — MPEG transport stream, common for broadcast recordings
Photos (JPG, PNG, BMP, TIFF, WebP, HEIC/HEIF) and videos share the same index. One reference photo finds a person across both media types in a single search.
Camera RAW files (CR2, NEF, ARW and similar) are not supported.
Why offline matters for video search
Video files are large. A few hours of footage can be tens of gigabytes. Uploading that to a cloud service for processing is slow, expensive, and — for anything sensitive — a privacy problem.
SnapByFace runs entirely on your machine. The face models are bundled inside the app, so recognition works with the network unplugged. No video, no frame, and no face data leaves your computer.
The only outbound data is an optional device statistics ping (machine code, app version, OS, language, activation state — no filenames, no footage, no search activity). It is on by default and can be turned off in Settings.
What stays on your machine
SnapByFace reads your media and never writes to it — no renaming, no moving, no re-encoding. The index it builds lives in ~/.snapbyface/ and contains only the numeric face embeddings, not the images themselves.
You can delete the index at any time and rebuild it from scratch, or let incremental indexing handle new files as they appear.
When the results will disappoint
Worth knowing before you start:
- Fast cuts and heavy motion blur produce frames where the face is not clear enough to encode reliably.
- Very small faces in frame (a wide shot of a crowd) are the hard case for any face search system.
- Heavy appearance change — or a reference photo that looks nothing like the person in the footage — lowers the similarity score below the threshold. Try adding a second reference photo or lowering the threshold.
Try it on your footage
The trial runs the full local search and shows up to 5 matches per person and 20 per search, which is enough to tell you whether it finds what you are looking for. During the trial you can also start up to 100 analyses in total.