Stop Inferencing the Same Image Twice: How pHash and dHash Cut GPU Compute by Caching What's Already Known

The Holmes AI pipeline processes 5,000+ support ticket images a day. Field engineers close tickets by submitting evidence — photos of installed hardware, filled checklists, device screens. Every image goes through a QC chain: preprocessing, YOLOv9 checkbox detection, and Qwen3-VL 32B semantic validation.

Each full inference pass costs real GPU time. Qwen3-VL 32B at 8-bit quantization is not cheap to call. We had the throughput under control — but the compute bill was higher than it should have been. When I dug into the incoming image stream, the reason became obvious.

A lot of images were the same image.

The Pattern

Field engineers work across many tickets in a day. For certain device types or installation categories, the checklist form is identical — same template, same layout, same printed text. Engineers photograph the same form, filled in the same way, for ticket after ticket. The images aren't byte-for-byte identical (different lighting, slight angle shifts, compression artifacts from different phone cameras), but visually, structurally, semantically — they're the same.

Every one of those near-identical images was going through the full pipeline independently. 5,000 images. Meaningful redundancy. Compute burned on work that had already been done.

The fix wasn't complicated, but it required choosing the right tool for the problem: perceptual hashing.

Why Not MD5 or SHA256?

The first instinct for deduplication is cryptographic hashing — MD5 or SHA256. Compute a hash of the file, store it, check if you've seen this hash before. It works perfectly for exact duplicates.

But exact duplicates are rare in real-world image pipelines. A photo taken of the same form from a slightly different angle, or compressed by a different phone codec, or resized before upload — those files will have completely different SHA256 hashes. Cryptographic hashes are collision-resistant by design. A single changed bit produces a completely different hash. That property is a feature for security; it's a bug for image deduplication.

What we needed were hashes that change gradually as images change — where similar images produce similar hashes, and "similar" is measurable. That's perceptual hashing.

pHash: The DCT Approach

pHash (perceptual hash) works on the frequency domain of an image. The algorithm is deterministic and fast^[1]:

1. Resize the image to a fixed small size (typically 32×32).
2. Convert to grayscale.
3. Apply a Discrete Cosine Transform (DCT) — the same transform used in JPEG compression.
4. Take the top-left 8×8 block of DCT coefficients, which captures the low-frequency structure (the overall shape and pattern, not fine noise).
5. Compute the mean of those 64 values.
6. Produce a 64-bit hash: bit i is 1 if coefficient i is above the mean, 0 otherwise.

The result is a 64-bit integer that encodes the perceptual structure of the image. Images that look similar produce hashes with similar bit patterns. The distance between two pHashes — measured in Hamming distance (number of bit positions that differ) — is a reliable proxy for visual similarity.

A Hamming distance of 0 means identical images. A distance below 10 generally indicates near-duplicates. A distance above 20 indicates meaningfully different images. These thresholds are empirical and tunable per domain^[2].

pHash is robust to: JPEG compression artifacts, minor brightness and contrast adjustments, small rotations, resizing, and watermarks in low-frequency regions. It is sensitive to: cropping, significant structural edits, and high-frequency content changes (which is a feature — if the checklist structure changes, we want to detect that).

dHash: The Gradient Approach

dHash (difference hash) is a simpler, faster algorithm that captures edge structure rather than frequency content^[3]:

1. Resize the image to 9×8 pixels.
2. Convert to grayscale.
3. For each row, compare adjacent pixel pairs: if the left pixel is brighter than the right, the bit is 1; otherwise 0.
4. This produces 64 bits (8 rows × 8 comparisons).

dHash encodes horizontal gradient structure — the pattern of brightness changes across the image. It's extremely fast (no DCT required) and good at capturing structural similarity: images with the same layout and composition produce similar dHashes even under lighting variation.

Where pHash and dHash differ: pHash is better at overall perceptual similarity; dHash is better at structural layout similarity. Used together, they cover different failure modes — pHash catching frequency-level similarity, dHash catching edge-pattern similarity.

The Filtering Layer

The implementation sits between the image ingestion step and the inference pipeline. Before any model is called:

1. Compute both hashes. For the incoming image, compute its pHash and dHash. This takes microseconds using the imagehash library^[4] — orders of magnitude faster than any model call.
2. Check the cache. Look up both hashes in a Redis store keyed by hash value. We store a tuple: (phash, dhash) as the key, the serialized inference result JSON as the value.
3. Similarity match. For each candidate in the cache (bounded lookup using hash prefix bucketing), compute Hamming distance on both hashes. If pHash distance ≤ 8 and dHash distance ≤ 6, it's a near-duplicate. Return the cached result directly.
4. Cache miss → inference. If no near-duplicate is found, run the full pipeline. Store the result in the cache with both hashes before returning.

The dual-hash requirement (both pHash and dHash must be within threshold) reduces false positives significantly. A form photographed at a different fill level may have similar frequency content (pHash match) but different structural edges (dHash miss). Requiring both to match ensures we're only serving cached results when the image is genuinely the same document in the same state.

The Output JSON Structure

What gets cached is the full inference output — a structured JSON document containing:

• YOLOv9 checkbox detection results: bounding boxes, confidence scores, boolean state per detected checkbox
• Qwen3-VL semantic validation output: per-field extracted values, anomaly flags, confidence per field
• Pipeline metadata: processing timestamp, model versions used, image dimensions

When a near-duplicate is detected, this JSON is returned directly to the caller. From the caller's perspective, the response is identical to a fresh inference result. The cache is invisible. Only the latency is different — milliseconds instead of seconds.

One important design decision: we cache at the inference output level, not the intermediate level. Caching raw model outputs and re-running downstream logic on them would create subtle bugs if the downstream logic changed. Caching the final structured result and returning it as-is means the cache is a pure semantic cache — it says "for this image, the answer is X", not "here are some numbers to feed into your pipeline."

Why Redis for the Hash Store

The hash store needed to be fast (lookup on the hot path before every inference call), persistent (survive pipeline restarts), and expirable (old cache entries should expire — a form template might change). Redis with a TTL on each key was the natural fit.

The lookup is a single Redis GET per hash pair. The write is a single Redis SET with a TTL. Total added latency on the hot path: under 2ms. Total added latency on a cache hit: under 5ms including deserialization. Compare that to a Qwen3-VL 32B inference pass which runs in the seconds range.

Hash collision in Redis (two different images mapping to the same hash key) is theoretically possible but practically negligible for 64-bit perceptual hashes in a domain with bounded visual diversity. We log hash collisions for monitoring but haven't seen one cause an issue in production.

The Numbers

After deploying the filtering layer, we measured the duplicate rate across a week of production traffic. It settled at around 15% of incoming images being near-duplicates that the pipeline had already processed — served from cache, no GPU touched.

15% might sound modest, but on a pipeline processing 5,000+ images a day, that's 750+ inference calls eliminated daily. Inference on Qwen3-VL 32B runs in the seconds range per image. At scale, that's real GPU time reclaimed — time that went back to the queue for the 85% of images that genuinely needed processing. GPU utilization dropped noticeably during peak hours, and P99 latency improved because the queue depth was lower.

More importantly: the cache hits were correct. We ran validation on a sample of cached responses by also running them through the full pipeline and comparing outputs. Agreement rate was 98%+. The 2% that differed were edge cases where images looked visually similar but had different fill states — which the dual-hash threshold is specifically designed to catch. In those cases, the thresholds correctly produced cache misses and routed to full inference.

Threshold Tuning

The right Hamming distance threshold depends on your domain. For our checklist form images:

• pHash threshold of 8 (out of 64 bits): captures compression artifacts, lighting differences, minor angle shifts without over-matching forms with different content.
• dHash threshold of 6: captures structural layout matches without over-matching forms with different filled fields.

These were tuned empirically on a labelled sample of image pairs. The right process: collect pairs of images you consider duplicates and pairs you consider distinct, sweep the threshold, measure precision and recall, pick the threshold that sits at the knee of the precision-recall curve^[5].

For a different domain — natural scene photos, medical images, product photographs — these numbers would be wrong. Perceptual hash thresholds are not universal. They're calibrated to the statistical distribution of your specific image corpus.

What This Pattern Generalizes To

The pHash/dHash caching layer is an instance of a broader pattern: compute-expensive operations on semantically stable inputs benefit from perceptual deduplication, not exact-match deduplication.

The same approach applies anywhere you have:

• An expensive operation (GPU inference, API call, complex transformation)
• Inputs that are semantically equivalent but not byte-identical
• A meaningful repeat rate in your input stream

Document OCR, image embedding generation, video frame analysis, audio transcription — all of these can benefit from perceptual deduplication upstream. The hash computation cost is negligible. The cache hit savings compound with traffic volume.

The insight is simple: the most expensive computation you can do is one you've already done once. If you can recognize that you've already done it, you don't have to do it again.