.

FaceSort: locally sorting thousands of photos by face recognition on an Apple Silicon Mac

This guide explains how to set up FaceSort, a Python tool that finds known people in a large photo collection and copies matching images into folders named after those people.

All processing takes place on the Mac. Photos and facial embeddings are not sent to an online service. The project described here was designed and tested on a Mac mini M4 with 16 GB of memory, although it can also run on other Apple Silicon Macs.

FaceSort can:

Important — privacy and licensing: a facial embedding is sensitive biometric data. Only use this system with images you are legally allowed to process and in accordance with applicable law. Pretrained LVFace and InsightFace/SCRFD weights may have usage restrictions, including non-commercial limitations. Always review the official licenses before distributing or operating the system.

1. How it works

The pipeline has two distinct stages:

  1. SCRFD detects every face in a photograph and locates its facial landmarks.
  2. LVFace-B converts each aligned face into a numerical vector, referred to here as a facial embedding.

For each known person, FaceSort computes an embedding for every valid reference portrait, averages those vectors, and normalizes the result. Every face found under Input is then compared with those identities using cosine similarity.

In our configuration, a similarity of 0.45 or higher is treated as a confident match. Scores from 0.30 up to 0.45 can be sent to Audit mode for human inspection. These are not universal thresholds; they must be validated against your own photographs.

2. Hardware and software requirements

The configuration used for this project was:

Check Python in Terminal:

python3.11 --version

If Python 3.11 is not installed:

brew install python@3.11

Install ExifTool as well:

brew install exiftool

ExifTool is not required for recognition. Without it, photos will still be sorted, but names will not be written into their metadata.

3. Create the project folder

The project may live anywhere on the disk. FaceSort calculates all paths relative to the script’s own location.

Create the following structure:

FaceSort/
├── Models/
├── Model_weights/
├── Input/
├── Output/
├── Review/
├── FaceSort.py
├── FaceSort.command
├── FaceSort Audit.command
├── Installer.command
└── requirements.txt

Each item has a specific role:

4. Install the Python environment

The requirements.txt file contains:

numpy>=1.26,<3
onnxruntime>=1.20,<2
opencv-python-headless>=4.10,<5
insightface>=0.7.3,<0.8
Pillow>=10,<12
pillow-heif>=0.18,<2

In Terminal, move into the FaceSort directory:

cd "/path/to/FaceSort"

Create an isolated environment and install the dependencies:

python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

The .venv environment remains inside the project directory and does not alter the Mac’s system-wide Python installation.

5. Download the models

FaceSort expects these exact files:

Model_weights/scrfd_10g.onnx
Model_weights/lvface_b.onnx

SCRFD detector

SCRFD-10G is the face detector. The official project and model catalogue are in the InsightFace repository. Download the SCRFD-10G ONNX model with five-point landmarks and rename it to scrfd_10g.onnx if necessary.

LVFace-B recognizer

Download the ONNX version of LVFace-B trained on Glint360K from ByteDance’s official LVFace repository or its linked model page, then store it as lvface_b.onnx.

LVFace-B was selected as a practical balance between accuracy, processing time, and memory use on a 16 GB machine. Larger models do not necessarily provide a useful improvement on a real photo library and can substantially slow the job.

Do not download weights from an unknown source. Check the filename, size, and any checksum supplied by the publisher.

6. Prepare known identities

Create one directory per identity under Models:

Models/
├── Person_A/
│   ├── portrait_01.jpg
│   ├── portrait_02.jpg
│   └── portrait_03.jpg
└── Person_B/
    ├── reference_01.jpg
    └── reference_02.jpg

Each reference photo must contain exactly one detectable face. A reference containing zero or several faces is automatically rejected while the identity is being built.

For a more representative identity embedding:

FaceSort recalculates every identity at each launch, so a newly added reference is used immediately. A technical identity_cache.npz file records the latest calculation but is not used to skip rebuilding the identities.

7. Add the photos to scan

Copy photographs into Input, using as many nested folders as required:

Input/
├── Trip_01/
│   ├── photo_001.jpg
│   └── photo_002.heic
└── Archives/
    └── 2019/
        └── group.png

Supported formats are JPEG, PNG, HEIC, and HEIF.

There is no need to move a large existing library. Input also accepts:

Traversal is recursive. Loop protection prevents circular links, and real-path deduplication ensures that one file is processed only once even when several links point to it.

8. Run FaceSort for the first time

Activate the environment and start the program:

cd "/path/to/FaceSort"
source .venv/bin/activate
python FaceSort.py

Alternatively, double-click FaceSort.command if that launcher is included with your copy of the project.

At startup, FaceSort:

  1. loads SCRFD and LVFace-B;
  2. reads every reference portrait again;
  3. rejects references that do not contain exactly one face;
  4. scans Input recursively;
  5. detects all faces in each photograph;
  6. compares each face with the known identities;
  7. copies confident matches into Output;
  8. writes metadata to the copies.

Example output:

Output/
├── Person_A/
│   ├── photo_001.jpg
│   └── group.png
└── Person_B/
    └── group.png

If group.png contains both people, it is copied into both directories. If a photograph contains no known identity, it is neither copied nor modified.

9. Metadata written to each copy

FaceSort stores recognized identities in three fields that are widely understood by photo-management software:

Only copies under Output are modified. Original source files remain untouched.

To inspect a copy:

exiftool -XMP-dc:Subject -IPTC:Keywords -EXIF:UserComment "Output/Person_A/photo_001.jpg"

10. Use Audit mode

Normal mode uses a 0.45 threshold. It favors confident results and ignores everything else. To look for false negatives that came close to the threshold, run:

python FaceSort.py --audit

or open FaceSort Audit.command.

Audit mode behaves as follows:

Review/candidates.csv records the source path, proposed identity, and score. It is finalized only after a normal end to the run. If the program is interrupted, some images may already exist under Review while the CSV remains incomplete or absent.

Do not automatically promote every Audit candidate into Models. Inspect it first: one wrongly labeled reference can damage the averaged identity embedding.

11. Tune the thresholds

To change the confident-match threshold:

python FaceSort.py --threshold 0.50

A higher threshold generally reduces false positives but misses more genuine appearances. A lower threshold finds more candidates at the cost of possible mistakes.

A cautious calibration process is:

  1. begin at 0.45;
  2. manually inspect every file copied to Output;
  3. run Audit over the 0.30–0.45 band;
  4. record the scores of genuine matches and mistakes;
  5. adjust the threshold using those observations rather than a public benchmark alone.

12. CPU, Core ML, and M4 performance

FaceSort provides three execution modes:

python FaceSort.py --backend hybrid
python FaceSort.py --backend cpu
python FaceSort.py --backend coreml

The default hybrid mode runs SCRFD through ONNX Runtime’s Core ML provider and LVFace-B on the CPU. On our Mac mini M4, a 200-photo test measured approximately:

This is less surprising than it first appears. Using ONNX Runtime’s Core ML provider does not guarantee that every operation will run efficiently on the Neural Engine. LVFace-B may be divided into several execution segments, with costly transfers between providers. Hybrid mode avoided that overhead.

This is not yet a fully optimized native Core ML implementation. A future version could convert both networks to FP16 MLProgram packages, inspect operator placement, and benchmark CPU, GPU, and Neural Engine execution on a fixed test set.

13. Results from a large real-world library

In one anonymized production run with three identities:

This result mainly demonstrates the value of a conservative threshold: FaceSort can locate a small number of appearances among tens of thousands of images without uploading the library to the cloud. It does not prove that there were no false negatives. Audit mode exists for exactly that reason.

14. Understand image warnings

Old photo collections commonly contain imperfect files.

PNG data with a JPG extension

If a file contains PNG data but is named .jpg, FaceSort detects its signature and gives the output copy a .png extension. The reverse correction is also made for JPEG data incorrectly named .png.

Duplicate XMP metadata

ExifTool is run in tolerant mode so that identity tags can still be written when the source contains minor XMP anomalies, such as duplicate properties or inconsistent rdf:about attributes.

“Corrupt JPEG data” or “Invalid SOS parameters”

These messages normally come from the JPEG decoder. If processing continues without a FaceSort ERROR line, the image was decoded and analyzed. If an ERROR is reported, that file is skipped. In either case, FaceSort never attempts to repair or rewrite the original.

15. Make the project portable

FaceSort treats the directory containing FaceSort.py as its default root. To provide the tool to another user:

  1. distribute the project without Input, Output, Review, or personal reference photos;
  2. do not include .venv, which should be rebuilt on the destination Mac;
  3. communicate the ONNX weight licenses separately and clearly;
  4. have the recipient run Installer.command on their machine;
  5. ask them to create their own identities and calibrate their own thresholds.

macOS may request permission the first time a Finder alias points to an external folder. Grant access only to folders that the user intends to scan.

16. Current limitations and possible improvements

The current version represents each person with the normalized average of all valid reference embeddings. This approach is simple and effective, but a more robust gallery could:

Reference quality and human review remain the priorities. Adding more portraits does not necessarily improve an identity if those portraits are poor, redundant, or mislabeled.

Conclusion

FaceSort provides a local, portable, and inspectable way to find known people across thousands of photographs. SCRFD, LVFace-B, ONNX Runtime, and ExifTool together produce a useful folder-based classification system while leaving all source files untouched.

The main Apple Silicon lesson is to measure instead of assuming. In our tests, using Core ML for detection and the CPU for recognition was slightly faster than CPU-only operation and substantially better than sending the complete pipeline through Core ML. The other lesson is just as important: no automatic score replaces human review, especially when biometric data is involved.

Project sources

· macOS, Apple Silicon, Python, Core ML, face recognition, photography