Update README with new test parameters and dataset setup
- Add detailed instructions for LogoDet-3K dataset placement - Document all test script parameters including new options: - simple matching method - --output-file for clean results output - --use-max-similarity, --positive-samples, --negative-samples - Add section on running comparison tests with shell script - Update project structure to include run_comparison_tests.sh
This commit is contained in:
51
README.md
51
README.md
@ -32,13 +32,33 @@ pip install -r requirements.txt
|
|||||||
|
|
||||||
### Prepare Test Data
|
### Prepare Test Data
|
||||||
|
|
||||||
First, prepare the test database with logo mappings:
|
The test framework requires the **LogoDet-3K** dataset. Download it and place it in the project directory:
|
||||||
|
|
||||||
|
```
|
||||||
|
logo_test/
|
||||||
|
├── LogoDet-3K/ # Dataset directory (required)
|
||||||
|
│ ├── Clothes/ # Category directories
|
||||||
|
│ │ ├── Adidas/ # Brand directories with images + XML annotations
|
||||||
|
│ │ ├── Nike/
|
||||||
|
│ │ └── ...
|
||||||
|
│ ├── Electronic/
|
||||||
|
│ ├── Food/
|
||||||
|
│ └── ...
|
||||||
|
```
|
||||||
|
|
||||||
|
The dataset should contain images with corresponding Pascal VOC format XML annotation files that define logo bounding boxes.
|
||||||
|
|
||||||
|
Then run the preparation script:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv run python prepare_test_data.py
|
uv run python prepare_test_data.py
|
||||||
```
|
```
|
||||||
|
|
||||||
This creates `test_data_mapping.db` with ground truth mappings between test images and logos.
|
This script:
|
||||||
|
1. Scans `LogoDet-3K/` for images and XML annotation files
|
||||||
|
2. Extracts cropped logo regions using bounding box data → saves to `reference_logos/`
|
||||||
|
3. Copies full images → saves to `test_images/`
|
||||||
|
4. Creates `test_data_mapping.db` SQLite database with ground truth mappings
|
||||||
|
|
||||||
### Run Detection Tests
|
### Run Detection Tests
|
||||||
|
|
||||||
@ -64,14 +84,33 @@ uv run python test_logo_detection.py -n 50 --seed 42
|
|||||||
| `-n, --num-logos` | 10 | Number of reference logos to sample |
|
| `-n, --num-logos` | 10 | Number of reference logos to sample |
|
||||||
| `-t, --threshold` | 0.7 | CLIP similarity threshold |
|
| `-t, --threshold` | 0.7 | CLIP similarity threshold |
|
||||||
| `-d, --detr-threshold` | 0.5 | DETR detection confidence threshold |
|
| `-d, --detr-threshold` | 0.5 | DETR detection confidence threshold |
|
||||||
| `--matching-method` | margin | Matching method: `margin` or `multi-ref` |
|
| `--matching-method` | margin | Matching method: `simple`, `margin`, or `multi-ref` |
|
||||||
| `--margin` | 0.05 | Margin over second-best match (margin method) |
|
| `--margin` | 0.05 | Margin over second-best match (margin/multi-ref) |
|
||||||
| `--min-matching-refs` | 1 | Min refs that must match (multi-ref method) |
|
|
||||||
| `--refs-per-logo` | 3 | Reference images per logo |
|
| `--refs-per-logo` | 3 | Reference images per logo |
|
||||||
|
| `--min-matching-refs` | 1 | Min refs that must match (multi-ref only) |
|
||||||
|
| `--use-max-similarity` | False | Use max instead of mean similarity (multi-ref only) |
|
||||||
|
| `--positive-samples` | 5 | Positive test images per logo |
|
||||||
|
| `--negative-samples` | 20 | Negative test images per logo |
|
||||||
| `-s, --seed` | None | Random seed for reproducibility |
|
| `-s, --seed` | None | Random seed for reproducibility |
|
||||||
|
| `--output-file` | None | Append results summary to file (clean output) |
|
||||||
|
|
||||||
|
**Matching Methods:**
|
||||||
|
- `simple` - Returns all logos above threshold (baseline, most permissive)
|
||||||
|
- `margin` - Requires margin over second-best match (reduces false positives)
|
||||||
|
- `multi-ref` - Aggregates scores across multiple reference images per logo
|
||||||
|
|
||||||
See `--help` for all options.
|
See `--help` for all options.
|
||||||
|
|
||||||
|
### Run Comparison Tests
|
||||||
|
|
||||||
|
To compare all matching methods with consistent parameters:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./run_comparison_tests.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
This runs all four matching configurations (simple, margin, multi-ref mean, multi-ref max) and saves clean results to `comparison_results.txt`.
|
||||||
|
|
||||||
## Project Structure
|
## Project Structure
|
||||||
|
|
||||||
```
|
```
|
||||||
@ -79,9 +118,11 @@ logo_test/
|
|||||||
├── logo_detection_detr.py # Core detection library (DetectLogosDETR class)
|
├── logo_detection_detr.py # Core detection library (DetectLogosDETR class)
|
||||||
├── test_logo_detection.py # Test script for accuracy evaluation
|
├── test_logo_detection.py # Test script for accuracy evaluation
|
||||||
├── prepare_test_data.py # Script to prepare test database
|
├── prepare_test_data.py # Script to prepare test database
|
||||||
|
├── run_comparison_tests.sh # Script to run all matching methods
|
||||||
├── test_data_mapping.db # SQLite database with ground truth
|
├── test_data_mapping.db # SQLite database with ground truth
|
||||||
├── reference_logos/ # Reference logo images (not in git)
|
├── reference_logos/ # Reference logo images (not in git)
|
||||||
├── test_images/ # Test images (not in git)
|
├── test_images/ # Test images (not in git)
|
||||||
|
├── LogoDet-3K/ # Source dataset (not in git)
|
||||||
├── logo_detection_detr_usage.md # API usage guide
|
├── logo_detection_detr_usage.md # API usage guide
|
||||||
└── logo_detection_test_methodology.md # Test methodology documentation
|
└── logo_detection_test_methodology.md # Test methodology documentation
|
||||||
```
|
```
|
||||||
|
|||||||
Reference in New Issue
Block a user