diff --git a/README.md b/README.md index 9ab8326..7add27b 100644 --- a/README.md +++ b/README.md @@ -32,13 +32,33 @@ pip install -r requirements.txt ### Prepare Test Data -First, prepare the test database with logo mappings: +The test framework requires the **LogoDet-3K** dataset. Download it and place it in the project directory: + +``` +logo_test/ +├── LogoDet-3K/ # Dataset directory (required) +│ ├── Clothes/ # Category directories +│ │ ├── Adidas/ # Brand directories with images + XML annotations +│ │ ├── Nike/ +│ │ └── ... +│ ├── Electronic/ +│ ├── Food/ +│ └── ... +``` + +The dataset should contain images with corresponding Pascal VOC format XML annotation files that define logo bounding boxes. + +Then run the preparation script: ```bash uv run python prepare_test_data.py ``` -This creates `test_data_mapping.db` with ground truth mappings between test images and logos. +This script: +1. Scans `LogoDet-3K/` for images and XML annotation files +2. Extracts cropped logo regions using bounding box data → saves to `reference_logos/` +3. Copies full images → saves to `test_images/` +4. Creates `test_data_mapping.db` SQLite database with ground truth mappings ### Run Detection Tests @@ -64,14 +84,33 @@ uv run python test_logo_detection.py -n 50 --seed 42 | `-n, --num-logos` | 10 | Number of reference logos to sample | | `-t, --threshold` | 0.7 | CLIP similarity threshold | | `-d, --detr-threshold` | 0.5 | DETR detection confidence threshold | -| `--matching-method` | margin | Matching method: `margin` or `multi-ref` | -| `--margin` | 0.05 | Margin over second-best match (margin method) | -| `--min-matching-refs` | 1 | Min refs that must match (multi-ref method) | +| `--matching-method` | margin | Matching method: `simple`, `margin`, or `multi-ref` | +| `--margin` | 0.05 | Margin over second-best match (margin/multi-ref) | | `--refs-per-logo` | 3 | Reference images per logo | +| `--min-matching-refs` | 1 | Min refs that must match (multi-ref only) | +| `--use-max-similarity` | False | Use max instead of mean similarity (multi-ref only) | +| `--positive-samples` | 5 | Positive test images per logo | +| `--negative-samples` | 20 | Negative test images per logo | | `-s, --seed` | None | Random seed for reproducibility | +| `--output-file` | None | Append results summary to file (clean output) | + +**Matching Methods:** +- `simple` - Returns all logos above threshold (baseline, most permissive) +- `margin` - Requires margin over second-best match (reduces false positives) +- `multi-ref` - Aggregates scores across multiple reference images per logo See `--help` for all options. +### Run Comparison Tests + +To compare all matching methods with consistent parameters: + +```bash +./run_comparison_tests.sh +``` + +This runs all four matching configurations (simple, margin, multi-ref mean, multi-ref max) and saves clean results to `comparison_results.txt`. + ## Project Structure ``` @@ -79,9 +118,11 @@ logo_test/ ├── logo_detection_detr.py # Core detection library (DetectLogosDETR class) ├── test_logo_detection.py # Test script for accuracy evaluation ├── prepare_test_data.py # Script to prepare test database +├── run_comparison_tests.sh # Script to run all matching methods ├── test_data_mapping.db # SQLite database with ground truth ├── reference_logos/ # Reference logo images (not in git) ├── test_images/ # Test images (not in git) +├── LogoDet-3K/ # Source dataset (not in git) ├── logo_detection_detr_usage.md # API usage guide └── logo_detection_test_methodology.md # Test methodology documentation ```