When you upload a photo to WhatZoo, the result appears in a second or two — but a surprising amount happens in that moment. This guide walks through the entire pipeline, from the raw image to your final animal match, so you can understand exactly what the app is (and isn't) doing.
Step 1: Finding your face
Before anything can be measured, the app has to locate a face in your photo. WhatZoo uses MediaPipe Face Mesh, an on-device machine learning model originally built by Google for augmented reality. It scans the image and answers two questions: is there a face here, and where is it?
This step runs entirely in your browser. Your photo is never uploaded to a server — the model files are downloaded to your device once, and all of the heavy lifting happens locally using your device's GPU. That is why analysis works even on a slow connection, and why your images stay private.
Step 2: Detecting 478 landmarks
Once a face is found, the model places 478 landmark points across it. These points trace the outline of your eyes, eyebrows, nose, lips, jaw, and the overall contour of your face. Each landmark is a coordinate in 3D space, so the model captures not just where your features are, but their relative depth and proportion.
478 points is a lot of detail. For comparison, many older face-detection systems used 68 points or fewer. The density matters because the differences between, say, a "fox" face and a "cat" face come down to subtle ratios — the slant of the eyes, the width of the nose bridge, the curve of the jaw.
Step 3: Turning landmarks into metrics
Raw coordinates are not very meaningful on their own — they change with photo size, distance, and angle. So the next step converts them into ratios and angles that stay consistent regardless of how the photo was taken. WhatZoo computes around seven core metrics, including:
- Eye slant — the upward or downward angle of the eyes
- Face aspect ratio — how long the face is relative to its width
- Nose width ratio — the width of the nose relative to the face
- Mouth width ratio — the width of the mouth relative to the face
- and several others covering eye size, jaw shape, and feature spacing
Because these are relative measurements, a selfie and a professional portrait of the same person should produce similar numbers. That is the whole point: the app is reading proportions, not pixels.
Step 4: Matching to an animal profile
Every animal type in WhatZoo has a profile — a set of expected metric ranges that define what that animal "looks like." A fox profile, for example, expects sharply slanted eyes and a narrow face; a bear profile expects a rounder, wider face.
The matching engine compares your metrics against every profile in two ways:
- A gate check (the critical trait). Each animal has one defining feature it cannot do without. If your face doesn't clear that gate, the animal is ruled out early.
- A weighted score. For the remaining candidates, each metric contributes to a score based on how important it is for that animal. The closer your proportions are to the profile, the higher the score.
The animal with the best overall score becomes your result, and the score itself becomes the "similarity" percentage you see on screen.
Why your result can change
People sometimes get a slightly different animal in two different photos. That is expected, and it usually comes down to the input. A tilted head changes your eye-slant reading; harsh side lighting can hide your jawline; a big smile widens your mouth ratio. The model is faithfully measuring what it sees — so a cleaner photo gives a cleaner result. (We wrote a separate guide on taking the perfect photo if you want the highest accuracy.)
An important caveat
This is the part we want to be completely honest about: animal face analysis is for entertainment. The pipeline above is real computer vision, and the measurements are genuine. But the meaning attached to them — the idea that a "lion face" implies a leader's personality — is playful interpretation, not science. There is no peer-reviewed evidence that facial proportions determine character.
So enjoy your result, share it with friends, and take the personality descriptions in the spirit they're intended: a fun mirror, not a diagnosis.
In short
- MediaPipe finds your face, on your device.
- It places 478 landmark points.
- Those points become consistent ratios and angles.
- The ratios are matched against animal profiles to find your closest type.
That's the whole journey — from a single photo to your spirit animal, in about a second.