Compare commits
3 Commits
512f678310
...
master
| Author | SHA1 | Date | |
|---|---|---|---|
| f2ae80c9e5 | |||
| 8b67b50d19 | |||
| 5ce6265a90 |
27
README.md
27
README.md
@ -2,6 +2,33 @@
|
||||
|
||||
A testing framework for evaluating logo detection accuracy using DETR (DEtection TRansformer) and CLIP (Contrastive Language-Image Pre-training) models.
|
||||
|
||||
## Burnley Test: Averaged Embeddings with DINOv2
|
||||
|
||||
A targeted test using `DetectLogosEmbeddings` to detect two specific logos (barnfield and vertu) in 516 Burnley match images. Reference embeddings are averaged across all images in each reference directory, and matching uses margin-based comparison (margin=0.05).
|
||||
|
||||
**Test command:**
|
||||
```bash
|
||||
uv run python test_burnley_detection.py -e dinov2 -t 0.7 --margin 0.05 --output-file results_average_embeddings.txt
|
||||
```
|
||||
|
||||
**Results (DINOv2, threshold 0.70, margin 0.05):**
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| True Positives | 28 |
|
||||
| False Positives | 36 |
|
||||
| False Negatives | 125 |
|
||||
| Total Expected | 146 |
|
||||
| **Precision** | **43.8%** |
|
||||
| **Recall** | **19.2%** |
|
||||
| **F1 Score** | **26.7%** |
|
||||
|
||||
Ground truth is derived from filename prefixes: `vertu_` (vertu logo), `barnfield_` (barnfield logo), `barnfield+vertu_` (both logos). Images without these prefixes are treated as negatives.
|
||||
|
||||
Low recall suggests many logos go undetected by DETR or fall below the similarity threshold. The relatively low precision indicates DINOv2 averaged embeddings struggle to discriminate between the two logos in this domain. Further tuning of thresholds, margin, and embedding model (e.g. CLIP or SigLIP) may improve results.
|
||||
|
||||
---
|
||||
|
||||
## Recommended Settings
|
||||
|
||||
Based on extensive testing with the LogoDet-3K dataset, these are the optimal settings:
|
||||
|
||||
52
results_average_embeddings.txt
Normal file
52
results_average_embeddings.txt
Normal file
@ -0,0 +1,52 @@
|
||||
======================================================================
|
||||
BURNLEY LOGO DETECTION TEST
|
||||
Model: dinov2
|
||||
Method: Margin-based (margin=0.05)
|
||||
======================================================================
|
||||
Date: 2026-03-31 11:45:03
|
||||
|
||||
Configuration:
|
||||
Embedding model: dinov2
|
||||
Similarity threshold: 0.7
|
||||
DETR threshold: 0.5
|
||||
Matching margin: 0.05
|
||||
Test images processed: 516
|
||||
Reference logos: barnfield, vertu
|
||||
|
||||
Results:
|
||||
True Positives: 28
|
||||
False Positives: 36
|
||||
False Negatives: 125
|
||||
Total Expected: 146
|
||||
|
||||
Scores:
|
||||
Precision: 0.4375 (43.8%)
|
||||
Recall: 0.1918 (19.2%)
|
||||
F1 Score: 0.2667 (26.7%)
|
||||
|
||||
======================================================================
|
||||
BURNLEY LOGO DETECTION TEST
|
||||
Model: dinov2
|
||||
Method: Margin-based (margin=0.05)
|
||||
======================================================================
|
||||
Date: 2026-03-31 12:29:32
|
||||
|
||||
Configuration:
|
||||
Embedding model: dinov2
|
||||
Similarity threshold: 0.7
|
||||
DETR threshold: 0.5
|
||||
Matching margin: 0.05
|
||||
Test images processed: 516
|
||||
Reference logos: barnfield, vertu
|
||||
|
||||
Results:
|
||||
True Positives: 28
|
||||
False Positives: 36
|
||||
False Negatives: 125
|
||||
Total Expected: 146
|
||||
|
||||
Scores:
|
||||
Precision: 0.4375 (43.8%)
|
||||
Recall: 0.1918 (19.2%)
|
||||
F1 Score: 0.2667 (26.7%)
|
||||
|
||||
Reference in New Issue
Block a user