Update README with new test parameters and dataset setup

- Add detailed instructions for LogoDet-3K dataset placement
- Document all test script parameters including new options:
  - simple matching method
  - --output-file for clean results output
  - --use-max-similarity, --positive-samples, --negative-samples
- Add section on running comparison tests with shell script
- Update project structure to include run_comparison_tests.sh
This commit is contained in:
Rick McEwen
2025-12-31 17:49:56 -05:00
parent 41c75356d9
commit ea589a50a4

View File

@ -32,13 +32,33 @@ pip install -r requirements.txt
### Prepare Test Data ### Prepare Test Data
First, prepare the test database with logo mappings: The test framework requires the **LogoDet-3K** dataset. Download it and place it in the project directory:
```
logo_test/
├── LogoDet-3K/ # Dataset directory (required)
│ ├── Clothes/ # Category directories
│ │ ├── Adidas/ # Brand directories with images + XML annotations
│ │ ├── Nike/
│ │ └── ...
│ ├── Electronic/
│ ├── Food/
│ └── ...
```
The dataset should contain images with corresponding Pascal VOC format XML annotation files that define logo bounding boxes.
Then run the preparation script:
```bash ```bash
uv run python prepare_test_data.py uv run python prepare_test_data.py
``` ```
This creates `test_data_mapping.db` with ground truth mappings between test images and logos. This script:
1. Scans `LogoDet-3K/` for images and XML annotation files
2. Extracts cropped logo regions using bounding box data → saves to `reference_logos/`
3. Copies full images → saves to `test_images/`
4. Creates `test_data_mapping.db` SQLite database with ground truth mappings
### Run Detection Tests ### Run Detection Tests
@ -64,14 +84,33 @@ uv run python test_logo_detection.py -n 50 --seed 42
| `-n, --num-logos` | 10 | Number of reference logos to sample | | `-n, --num-logos` | 10 | Number of reference logos to sample |
| `-t, --threshold` | 0.7 | CLIP similarity threshold | | `-t, --threshold` | 0.7 | CLIP similarity threshold |
| `-d, --detr-threshold` | 0.5 | DETR detection confidence threshold | | `-d, --detr-threshold` | 0.5 | DETR detection confidence threshold |
| `--matching-method` | margin | Matching method: `margin` or `multi-ref` | | `--matching-method` | margin | Matching method: `simple`, `margin`, or `multi-ref` |
| `--margin` | 0.05 | Margin over second-best match (margin method) | | `--margin` | 0.05 | Margin over second-best match (margin/multi-ref) |
| `--min-matching-refs` | 1 | Min refs that must match (multi-ref method) |
| `--refs-per-logo` | 3 | Reference images per logo | | `--refs-per-logo` | 3 | Reference images per logo |
| `--min-matching-refs` | 1 | Min refs that must match (multi-ref only) |
| `--use-max-similarity` | False | Use max instead of mean similarity (multi-ref only) |
| `--positive-samples` | 5 | Positive test images per logo |
| `--negative-samples` | 20 | Negative test images per logo |
| `-s, --seed` | None | Random seed for reproducibility | | `-s, --seed` | None | Random seed for reproducibility |
| `--output-file` | None | Append results summary to file (clean output) |
**Matching Methods:**
- `simple` - Returns all logos above threshold (baseline, most permissive)
- `margin` - Requires margin over second-best match (reduces false positives)
- `multi-ref` - Aggregates scores across multiple reference images per logo
See `--help` for all options. See `--help` for all options.
### Run Comparison Tests
To compare all matching methods with consistent parameters:
```bash
./run_comparison_tests.sh
```
This runs all four matching configurations (simple, margin, multi-ref mean, multi-ref max) and saves clean results to `comparison_results.txt`.
## Project Structure ## Project Structure
``` ```
@ -79,9 +118,11 @@ logo_test/
├── logo_detection_detr.py # Core detection library (DetectLogosDETR class) ├── logo_detection_detr.py # Core detection library (DetectLogosDETR class)
├── test_logo_detection.py # Test script for accuracy evaluation ├── test_logo_detection.py # Test script for accuracy evaluation
├── prepare_test_data.py # Script to prepare test database ├── prepare_test_data.py # Script to prepare test database
├── run_comparison_tests.sh # Script to run all matching methods
├── test_data_mapping.db # SQLite database with ground truth ├── test_data_mapping.db # SQLite database with ground truth
├── reference_logos/ # Reference logo images (not in git) ├── reference_logos/ # Reference logo images (not in git)
├── test_images/ # Test images (not in git) ├── test_images/ # Test images (not in git)
├── LogoDet-3K/ # Source dataset (not in git)
├── logo_detection_detr_usage.md # API usage guide ├── logo_detection_detr_usage.md # API usage guide
└── logo_detection_test_methodology.md # Test methodology documentation └── logo_detection_test_methodology.md # Test methodology documentation
``` ```