Palm Line Reader

A small neural network that finds the three big creases in a photo of your palm (the heart, head, and life lines) and traces them. It runs entirely in the browser. No server, no upload, your photo never leaves your phone.

Try the live demo →

What this is

Point a camera at your open palm and the model paints three lines over it:

Line	Colour
Heart line	🔴 red
Head line	🔵 blue
Life line	🟢 green

The whole thing is 5.5M parameters and ships as an 11 MB ONNX file, so it loads and runs on a laptop or a mid-range phone with nothing round-tripping to a server.

Examples

Raw crop on the left, model output on the right. These are held-out palms the model never saw during training, in the kind of messy lighting and backgrounds you actually get from a webcam.

Input	Traced

How it works

The interesting part is where the training labels come from. Nobody hand-draws palm lines at scale, so I bootstrapped them.

1. Teacher. I already had a big segmentation model (a mit_b5 UNet, 324 MB, trained on near-infrared hand scans) that traces palm lines well but only after a heavy post-processing chain: close the mask, keep the largest blob, skeletonize, prune branches, dilate. It is far too slow and too big for a browser.

2. Pseudo-labels. I scraped a few thousand palm photos from r/PalmReading, ran each one through the teacher plus that full post-processing chain, and saved the cleaned-up result as a training label. The key trick: because the labels are already the cleaned output, the student learns to produce clean single lines directly, and none of that post-processing has to be reimplemented in JavaScript.

3. Review. The teacher is not perfect on ordinary RGB photos, so I built a little browser paint tool to page through the pseudo-labels and fix or reject the bad ones. Human verdicts override the automatic filter.

4. Student. A mit_b0 UNet (same family as the teacher, much smaller) trained on those labels at 512×512, RGB. The loss is more than plain pixel overlap: it adds clDice to reward an unbroken centre line, a distance term to keep each line in roughly the right place, and a connectivity penalty so each line comes out as exactly one piece instead of scattered fragments.

5. Browser. Export to ONNX, cast the weights to fp16, and run it with onnxruntime-web (WebGPU where available, WASM everywhere else). MediaPipe's hand landmarker finds and crops the palm on the live camera feed, right hands get mirrored so the model always sees a left palm, and one inference runs per captured frame.

The full design write-up is in plan.md.

Repo layout

palm-line-reader/
├─ web/          browser demo: ONNX Runtime Web + MediaPipe, no build step
├─ models/       exported ONNX (fp16 to ship, int8 smaller, fp32 reference)
├─ training/     the student: model, dataset, losses, metrics, train + export
├─ pipeline/     how the labels were made: scrape, teacher inference, paint tool
├─ examples/     a sample crop for smoke-testing
├─ docs/         the images in this README
└─ plan.md       the original design doc

Run the web demo locally

ES modules and fetch need a real server, so a plain file:// open will not work.

cd web
python3 -m http.server 8000
# open http://localhost:8000/index.html

Click "use bundled sample" to run it against the example palm without a camera. More detail, including how to drop the model into your own site, is in web/README.md.

Train it yourself

The training/ folder is self-contained once you have a labelled data/ directory. It does not need the teacher or MediaPipe, just images and masks.

cd training
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements.txt

python train.py --data_dir data --out_dir runs/exp1 --epochs 150 --batch_size 16
python export_onnx.py --checkpoint runs/exp1/checkpoints/best.pt --out exported/student.onnx

Full flag reference and a fast "does my environment work" sanity run are in training/README.md. The version shipped here hit 0.81 foreground Dice on held-out palms at epoch 102.

Model details

• Architecture: segmentation_models_pytorch UNet, mit_b0 encoder, 5.55M params
• Input: RGB, 512×512, ImageNet-normalised, NCHW
• Output: 4-class logits (background, heart, head, life); argmax for the mask
• Files: student_fp16.onnx (11 MB, ship this), student_int8.onnx (5.7 MB, lossier on thin lines), student_fp32.onnx (22 MB, reference)

The machine-readable contract is in models/model_meta.json.

Not included

The scraped source photos and the 324 MB teacher checkpoint are not in this repo, for size and provenance reasons. Everything needed to run the model, and all the code to reproduce the pipeline, is here.

A note on the fortunes

The reading the demo gives you is nonsense, obviously. Palmistry is not real. The segmentation is, though, and getting one clean unbroken line per crease out of a 5 MB model on a phone is a genuinely fun problem.

License

MIT. See LICENSE.