refactor YOLO crop model

This commit is contained in:
Nguyễn Phước Thành
2025-08-05 20:53:47 +07:00
parent a4e7573dca
commit 3fd270c8bd
24 changed files with 136 additions and 479 deletions

View File

@@ -1,56 +0,0 @@
# ID Card Cropper
Script đơn giản để cắt ID cards từ ảnh sử dụng Roboflow API.
## Cách sử dụng
```bash
python id_card_cropper.py input_folder output_folder
```
### Ví dụ:
```bash
# Sử dụng API key mặc định
python id_card_cropper.py data/IDcards/Archive output/cropped_cards
# Sử dụng API key tùy chỉnh
python id_card_cropper.py data/IDcards/Archive output/cropped_cards --api-key YOUR_API_KEY
```
## Tham số
- `input_folder`: Thư mục chứa ảnh cần xử lý
- `output_folder`: Thư mục lưu ID cards đã cắt
- `--api-key`: API key Roboflow (mặc định: demo key)
## Hỗ trợ định dạng ảnh
- JPG/JPEG
- PNG
- BMP
- TIFF
## Kết quả
Script sẽ:
1. Tìm tất cả ảnh trong thư mục input
2. Phát hiện ID cards trong mỗi ảnh
3. Cắt và lưu ID cards vào thư mục output
4. Đặt tên file theo format: `{tên_ảnh_gốc}_card_{số}.jpg`
## Ví dụ kết quả
```
output/cropped_cards/
├── im1__card_1.jpg
├── im5_card_1.jpg
├── im11_card_1.jpg
└── im11_card_2.jpg
```
## Lưu ý
- Cần kết nối internet để sử dụng Roboflow API
- Có delay 1 giây giữa các request để tránh rate limiting
- Chỉ lưu ID cards đã cắt, không lưu ảnh gốc với bounding boxes

View File

@@ -3,7 +3,7 @@
# Paths configuration
paths:
input_dir: "data/Archive"
input_dir: "data/IDcards/processed"
output_dir: "out"
log_file: "logs/data_augmentation.log"
@@ -17,7 +17,7 @@ augmentation:
# Processing configuration
processing:
target_size: [224, 224] # [width, height]
target_size: [640, 640] # [width, height] - Increased for better coverage
batch_size: 32
num_augmentations: 3 # number of augmented versions per image
save_format: "jpg"

View File

@@ -1,222 +0,0 @@
2025-08-05 18:53:06,981 - src.model.yolo_detector - INFO - Using pre-trained YOLOv8n model
2025-08-05 18:53:07,004 - src.model.yolo_detector - INFO - Using device: cuda
2025-08-05 18:53:07,038 - src.model.yolo_detector - INFO - Using pre-trained YOLOv8n model
2025-08-05 18:53:07,038 - src.model.yolo_detector - INFO - Using device: cuda
2025-08-05 18:53:07,361 - src.model.yolo_detector - INFO - Using pre-trained YOLOv8n model
2025-08-05 18:53:07,362 - src.model.yolo_detector - INFO - Using device: cuda
2025-08-05 18:53:07,363 - src.model.id_card_processor - INFO - Detecting and cropping ID cards...
2025-08-05 18:53:07,363 - src.model.yolo_detector - ERROR - No images found in data\IDcards
2025-08-05 18:53:07,364 - src.model.id_card_processor - INFO - Processing cropped ID cards...
2025-08-05 18:53:07,364 - src.model.id_card_processor - ERROR - No images found in data\test_output\cropped
2025-08-05 19:04:14,903 - src.model.yolo_detector - INFO - Using pre-trained YOLOv8n model
2025-08-05 19:04:14,995 - src.model.yolo_detector - INFO - Using device: cuda
2025-08-05 19:04:14,996 - src.model.id_card_processor - INFO - Detecting and cropping ID cards...
2025-08-05 19:04:14,997 - src.model.yolo_detector - INFO - Processing 29 images from data\IDcards and subdirectories
2025-08-05 19:04:14,998 - src.model.yolo_detector - INFO - Processing 1/29: im10.png
2025-08-05 19:04:19,785 - src.model.yolo_detector - INFO - Found 1 detections in im10.png
2025-08-05 19:04:19,813 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im10_card_1.jpg
2025-08-05 19:04:19,813 - src.model.yolo_detector - INFO - Processed im10.png: 1 cards cropped
2025-08-05 19:04:19,814 - src.model.yolo_detector - INFO - Processing 2/29: im11.png
2025-08-05 19:04:19,926 - src.model.yolo_detector - INFO - Found 2 detections in im11.png
2025-08-05 19:04:19,937 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im11_card_1.jpg
2025-08-05 19:04:19,946 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im11_card_2.jpg
2025-08-05 19:04:19,946 - src.model.yolo_detector - INFO - Processed im11.png: 2 cards cropped
2025-08-05 19:04:19,946 - src.model.yolo_detector - INFO - Processing 3/29: im12.png
2025-08-05 19:04:20,056 - src.model.yolo_detector - INFO - Found 2 detections in im12.png
2025-08-05 19:04:20,069 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im12_card_1.jpg
2025-08-05 19:04:20,082 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im12_card_2.jpg
2025-08-05 19:04:20,083 - src.model.yolo_detector - INFO - Processed im12.png: 2 cards cropped
2025-08-05 19:04:20,083 - src.model.yolo_detector - INFO - Processing 4/29: im13.png
2025-08-05 19:04:20,116 - src.model.yolo_detector - INFO - Found 0 detections in im13.png
2025-08-05 19:04:20,117 - src.model.yolo_detector - WARNING - No ID cards detected in im13.png
2025-08-05 19:04:20,117 - src.model.yolo_detector - INFO - Processing 5/29: im14.png
2025-08-05 19:04:20,156 - src.model.yolo_detector - INFO - Found 1 detections in im14.png
2025-08-05 19:04:20,172 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im14_card_1.jpg
2025-08-05 19:04:20,173 - src.model.yolo_detector - INFO - Processed im14.png: 1 cards cropped
2025-08-05 19:04:20,174 - src.model.yolo_detector - INFO - Processing 6/29: im15.png
2025-08-05 19:04:20,208 - src.model.yolo_detector - INFO - Found 1 detections in im15.png
2025-08-05 19:04:20,222 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im15_card_1.jpg
2025-08-05 19:04:20,222 - src.model.yolo_detector - INFO - Processed im15.png: 1 cards cropped
2025-08-05 19:04:20,223 - src.model.yolo_detector - INFO - Processing 7/29: im1_.png
2025-08-05 19:04:20,466 - src.model.yolo_detector - INFO - Found 0 detections in im1_.png
2025-08-05 19:04:20,466 - src.model.yolo_detector - WARNING - No ID cards detected in im1_.png
2025-08-05 19:04:20,466 - src.model.yolo_detector - INFO - Processing 8/29: im2.png
2025-08-05 19:04:20,534 - src.model.yolo_detector - INFO - Found 2 detections in im2.png
2025-08-05 19:04:20,564 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im2_card_1.jpg
2025-08-05 19:04:20,594 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im2_card_2.jpg
2025-08-05 19:04:20,594 - src.model.yolo_detector - INFO - Processed im2.png: 2 cards cropped
2025-08-05 19:04:20,595 - src.model.yolo_detector - INFO - Processing 9/29: im3.png
2025-08-05 19:04:20,648 - src.model.yolo_detector - INFO - Found 1 detections in im3.png
2025-08-05 19:04:20,671 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im3_card_1.jpg
2025-08-05 19:04:20,671 - src.model.yolo_detector - INFO - Processed im3.png: 1 cards cropped
2025-08-05 19:04:20,672 - src.model.yolo_detector - INFO - Processing 10/29: im4.png
2025-08-05 19:04:20,724 - src.model.yolo_detector - INFO - Found 1 detections in im4.png
2025-08-05 19:04:20,753 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im4_card_1.jpg
2025-08-05 19:04:20,754 - src.model.yolo_detector - INFO - Processed im4.png: 1 cards cropped
2025-08-05 19:04:20,754 - src.model.yolo_detector - INFO - Processing 11/29: im5.png
2025-08-05 19:04:20,798 - src.model.yolo_detector - INFO - Found 2 detections in im5.png
2025-08-05 19:04:20,816 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im5_card_1.jpg
2025-08-05 19:04:20,835 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im5_card_2.jpg
2025-08-05 19:04:20,836 - src.model.yolo_detector - INFO - Processed im5.png: 2 cards cropped
2025-08-05 19:04:20,837 - src.model.yolo_detector - INFO - Processing 12/29: im6.png
2025-08-05 19:04:20,994 - src.model.yolo_detector - INFO - Found 2 detections in im6.png
2025-08-05 19:04:21,052 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im6_card_1.jpg
2025-08-05 19:04:21,118 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im6_card_2.jpg
2025-08-05 19:04:21,119 - src.model.yolo_detector - INFO - Processed im6.png: 2 cards cropped
2025-08-05 19:04:21,120 - src.model.yolo_detector - INFO - Processing 13/29: im7.png
2025-08-05 19:04:21,159 - src.model.yolo_detector - INFO - Found 3 detections in im7.png
2025-08-05 19:04:21,168 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im7_card_1.jpg
2025-08-05 19:04:21,176 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im7_card_2.jpg
2025-08-05 19:04:21,184 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im7_card_3.jpg
2025-08-05 19:04:21,184 - src.model.yolo_detector - INFO - Processed im7.png: 3 cards cropped
2025-08-05 19:04:21,185 - src.model.yolo_detector - INFO - Processing 14/29: im8.png
2025-08-05 19:04:21,353 - src.model.yolo_detector - INFO - Found 2 detections in im8.png
2025-08-05 19:04:21,387 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im8_card_1.jpg
2025-08-05 19:04:21,423 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im8_card_2.jpg
2025-08-05 19:04:21,424 - src.model.yolo_detector - INFO - Processed im8.png: 2 cards cropped
2025-08-05 19:04:21,425 - src.model.yolo_detector - INFO - Processing 15/29: im9.png
2025-08-05 19:04:21,522 - src.model.yolo_detector - INFO - Found 1 detections in im9.png
2025-08-05 19:04:21,532 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\Archive\im9_card_1.jpg
2025-08-05 19:04:21,532 - src.model.yolo_detector - INFO - Processed im9.png: 1 cards cropped
2025-08-05 19:04:21,532 - src.model.yolo_detector - INFO - Processing 16/29: im10.png
2025-08-05 19:04:21,585 - src.model.yolo_detector - INFO - Found 3 detections in im10.png
2025-08-05 19:04:21,601 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im10_card_1.jpg
2025-08-05 19:04:21,618 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im10_card_2.jpg
2025-08-05 19:04:21,636 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im10_card_3.jpg
2025-08-05 19:04:21,636 - src.model.yolo_detector - INFO - Processed im10.png: 3 cards cropped
2025-08-05 19:04:21,638 - src.model.yolo_detector - INFO - Processing 17/29: im11.png
2025-08-05 19:04:21,679 - src.model.yolo_detector - INFO - Found 2 detections in im11.png
2025-08-05 19:04:21,696 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im11_card_1.jpg
2025-08-05 19:04:21,712 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im11_card_2.jpg
2025-08-05 19:04:21,713 - src.model.yolo_detector - INFO - Processed im11.png: 2 cards cropped
2025-08-05 19:04:21,713 - src.model.yolo_detector - INFO - Processing 18/29: im12.png
2025-08-05 19:04:21,755 - src.model.yolo_detector - INFO - Found 0 detections in im12.png
2025-08-05 19:04:21,756 - src.model.yolo_detector - WARNING - No ID cards detected in im12.png
2025-08-05 19:04:21,756 - src.model.yolo_detector - INFO - Processing 19/29: im13.png
2025-08-05 19:04:21,793 - src.model.yolo_detector - INFO - Found 1 detections in im13.png
2025-08-05 19:04:21,806 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im13_card_1.jpg
2025-08-05 19:04:21,806 - src.model.yolo_detector - INFO - Processed im13.png: 1 cards cropped
2025-08-05 19:04:21,806 - src.model.yolo_detector - INFO - Processing 20/29: im14.png
2025-08-05 19:04:21,846 - src.model.yolo_detector - INFO - Found 2 detections in im14.png
2025-08-05 19:04:21,862 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im14_card_1.jpg
2025-08-05 19:04:21,877 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im14_card_2.jpg
2025-08-05 19:04:21,877 - src.model.yolo_detector - INFO - Processed im14.png: 2 cards cropped
2025-08-05 19:04:21,878 - src.model.yolo_detector - INFO - Processing 21/29: im15.png
2025-08-05 19:04:21,914 - src.model.yolo_detector - INFO - Found 0 detections in im15.png
2025-08-05 19:04:21,914 - src.model.yolo_detector - WARNING - No ID cards detected in im15.png
2025-08-05 19:04:21,914 - src.model.yolo_detector - INFO - Processing 22/29: im1_.png
2025-08-05 19:04:21,959 - src.model.yolo_detector - INFO - Found 3 detections in im1_.png
2025-08-05 19:04:21,971 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im1__card_1.jpg
2025-08-05 19:04:21,983 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im1__card_2.jpg
2025-08-05 19:04:21,996 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im1__card_3.jpg
2025-08-05 19:04:21,997 - src.model.yolo_detector - INFO - Processed im1_.png: 3 cards cropped
2025-08-05 19:04:21,997 - src.model.yolo_detector - INFO - Processing 23/29: im2.png
2025-08-05 19:04:22,101 - src.model.yolo_detector - INFO - Found 1 detections in im2.png
2025-08-05 19:04:22,174 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im2_card_1.jpg
2025-08-05 19:04:22,174 - src.model.yolo_detector - INFO - Processed im2.png: 1 cards cropped
2025-08-05 19:04:22,176 - src.model.yolo_detector - INFO - Processing 24/29: im3.png
2025-08-05 19:04:22,220 - src.model.yolo_detector - INFO - Found 2 detections in im3.png
2025-08-05 19:04:22,235 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im3_card_1.jpg
2025-08-05 19:04:22,251 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im3_card_2.jpg
2025-08-05 19:04:22,252 - src.model.yolo_detector - INFO - Processed im3.png: 2 cards cropped
2025-08-05 19:04:22,252 - src.model.yolo_detector - INFO - Processing 25/29: im5.png
2025-08-05 19:04:22,307 - src.model.yolo_detector - INFO - Found 1 detections in im5.png
2025-08-05 19:04:22,316 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im5_card_1.jpg
2025-08-05 19:04:22,316 - src.model.yolo_detector - INFO - Processed im5.png: 1 cards cropped
2025-08-05 19:04:22,317 - src.model.yolo_detector - INFO - Processing 26/29: im6.png
2025-08-05 19:04:22,375 - src.model.yolo_detector - INFO - Found 2 detections in im6.png
2025-08-05 19:04:22,387 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im6_card_1.jpg
2025-08-05 19:04:22,397 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im6_card_2.jpg
2025-08-05 19:04:22,398 - src.model.yolo_detector - INFO - Processed im6.png: 2 cards cropped
2025-08-05 19:04:22,399 - src.model.yolo_detector - INFO - Processing 27/29: im7.png
2025-08-05 19:04:22,441 - src.model.yolo_detector - INFO - Found 1 detections in im7.png
2025-08-05 19:04:22,458 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im7_card_1.jpg
2025-08-05 19:04:22,459 - src.model.yolo_detector - INFO - Processed im7.png: 1 cards cropped
2025-08-05 19:04:22,460 - src.model.yolo_detector - INFO - Processing 28/29: im8.png
2025-08-05 19:04:22,492 - src.model.yolo_detector - INFO - Found 2 detections in im8.png
2025-08-05 19:04:22,502 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im8_card_1.jpg
2025-08-05 19:04:22,509 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im8_card_2.jpg
2025-08-05 19:04:22,510 - src.model.yolo_detector - INFO - Processed im8.png: 2 cards cropped
2025-08-05 19:04:22,510 - src.model.yolo_detector - INFO - Processing 29/29: im9.png
2025-08-05 19:04:22,540 - src.model.yolo_detector - INFO - Found 1 detections in im9.png
2025-08-05 19:04:22,546 - src.model.yolo_detector - INFO - Saved cropped image to data\processed_id_cards\cropped\titre-sejour-fr\im9_card_1.jpg
2025-08-05 19:04:22,546 - src.model.yolo_detector - INFO - Processed im9.png: 1 cards cropped
2025-08-05 19:04:22,546 - src.model.yolo_detector - INFO - Batch processing completed:
2025-08-05 19:04:22,548 - src.model.yolo_detector - INFO - - Total images: 29
2025-08-05 19:04:22,548 - src.model.yolo_detector - INFO - - Processed: 25
2025-08-05 19:04:22,548 - src.model.yolo_detector - INFO - - Total detections: 42
2025-08-05 19:04:22,549 - src.model.yolo_detector - INFO - - Total cropped: 42
2025-08-05 19:04:22,549 - src.model.id_card_processor - INFO - Processing cropped ID cards...
2025-08-05 19:04:22,552 - src.model.id_card_processor - INFO - Processing 42 images from data\processed_id_cards\cropped and subdirectories
2025-08-05 19:04:22,552 - src.model.id_card_processor - INFO - Processing 1/42: im10_card_1.jpg
2025-08-05 19:04:22,564 - src.model.id_card_processor - INFO - Removing background from im10_card_1.jpg
2025-08-05 19:04:22,877 - src.model.id_card_processor - INFO - Enhancing im10_card_1.jpg
2025-08-05 19:04:23,016 - src.model.id_card_processor - INFO - Normalizing im10_card_1.jpg
2025-08-05 19:04:23,023 - src.model.id_card_processor - INFO - Processed im10_card_1.jpg
2025-08-05 19:04:23,023 - src.model.id_card_processor - INFO - Processing 2/42: im11_card_1.jpg
2025-08-05 19:04:23,034 - src.model.id_card_processor - INFO - Removing background from im11_card_1.jpg
2025-08-05 19:04:23,264 - src.model.id_card_processor - INFO - Enhancing im11_card_1.jpg
2025-08-05 19:04:23,265 - src.model.id_card_processor - INFO - Normalizing im11_card_1.jpg
2025-08-05 19:04:23,270 - src.model.id_card_processor - INFO - Processed im11_card_1.jpg
2025-08-05 19:04:23,271 - src.model.id_card_processor - INFO - Processing 3/42: im11_card_2.jpg
2025-08-05 19:04:23,282 - src.model.id_card_processor - INFO - Removing background from im11_card_2.jpg
2025-08-05 19:04:23,312 - src.model.id_card_processor - INFO - Enhancing im11_card_2.jpg
2025-08-05 19:04:23,313 - src.model.id_card_processor - INFO - Normalizing im11_card_2.jpg
2025-08-05 19:04:23,316 - src.model.id_card_processor - INFO - Processed im11_card_2.jpg
2025-08-05 19:04:23,316 - src.model.id_card_processor - INFO - Processing 4/42: im12_card_1.jpg
2025-08-05 19:04:23,328 - src.model.id_card_processor - INFO - Removing background from im12_card_1.jpg
2025-08-05 19:04:23,670 - src.model.id_card_processor - INFO - Enhancing im12_card_1.jpg
2025-08-05 19:04:23,671 - src.model.id_card_processor - INFO - Normalizing im12_card_1.jpg
2025-08-05 19:04:23,675 - src.model.id_card_processor - INFO - Processed im12_card_1.jpg
2025-08-05 19:04:23,676 - src.model.id_card_processor - INFO - Processing 5/42: im12_card_2.jpg
2025-08-05 19:04:23,686 - src.model.id_card_processor - INFO - Removing background from im12_card_2.jpg
2025-08-05 19:04:29,279 - src.model.id_card_processor - INFO - Enhancing im12_card_2.jpg
2025-08-05 19:04:29,284 - src.model.id_card_processor - INFO - Normalizing im12_card_2.jpg
2025-08-05 19:04:29,289 - src.model.id_card_processor - INFO - Processed im12_card_2.jpg
2025-08-05 19:04:29,290 - src.model.id_card_processor - INFO - Processing 6/42: im14_card_1.jpg
2025-08-05 19:04:29,301 - src.model.id_card_processor - INFO - Removing background from im14_card_1.jpg
2025-08-05 19:04:29,774 - src.model.id_card_processor - INFO - Enhancing im14_card_1.jpg
2025-08-05 19:04:29,775 - src.model.id_card_processor - INFO - Normalizing im14_card_1.jpg
2025-08-05 19:04:29,779 - src.model.id_card_processor - INFO - Processed im14_card_1.jpg
2025-08-05 19:04:29,780 - src.model.id_card_processor - INFO - Processing 7/42: im15_card_1.jpg
2025-08-05 19:04:29,791 - src.model.id_card_processor - INFO - Removing background from im15_card_1.jpg
2025-08-05 19:04:30,009 - src.model.id_card_processor - INFO - Enhancing im15_card_1.jpg
2025-08-05 19:04:30,010 - src.model.id_card_processor - INFO - Normalizing im15_card_1.jpg
2025-08-05 19:04:30,015 - src.model.id_card_processor - INFO - Processed im15_card_1.jpg
2025-08-05 19:04:30,015 - src.model.id_card_processor - INFO - Processing 8/42: im2_card_1.jpg
2025-08-05 19:04:30,017 - src.model.id_card_processor - INFO - Removing background from im2_card_1.jpg
2025-08-05 19:04:31,861 - src.model.id_card_processor - INFO - Enhancing im2_card_1.jpg
2025-08-05 19:04:31,863 - src.model.id_card_processor - INFO - Normalizing im2_card_1.jpg
2025-08-05 19:04:31,869 - src.model.id_card_processor - INFO - Processed im2_card_1.jpg
2025-08-05 19:04:31,869 - src.model.id_card_processor - INFO - Processing 9/42: im2_card_2.jpg
2025-08-05 19:04:31,884 - src.model.id_card_processor - INFO - Removing background from im2_card_2.jpg
2025-08-05 19:04:38,985 - src.model.id_card_processor - INFO - Enhancing im2_card_2.jpg
2025-08-05 19:04:38,996 - src.model.id_card_processor - INFO - Normalizing im2_card_2.jpg
2025-08-05 19:04:39,007 - src.model.id_card_processor - INFO - Processed im2_card_2.jpg
2025-08-05 19:04:39,008 - src.model.id_card_processor - INFO - Processing 10/42: im3_card_1.jpg
2025-08-05 19:04:39,009 - src.model.id_card_processor - INFO - Removing background from im3_card_1.jpg
2025-08-05 19:04:39,177 - src.model.id_card_processor - INFO - Enhancing im3_card_1.jpg
2025-08-05 19:04:39,178 - src.model.id_card_processor - INFO - Normalizing im3_card_1.jpg
2025-08-05 19:04:39,182 - src.model.id_card_processor - INFO - Processed im3_card_1.jpg
2025-08-05 19:04:39,182 - src.model.id_card_processor - INFO - Processing 11/42: im4_card_1.jpg
2025-08-05 19:04:39,184 - src.model.id_card_processor - INFO - Removing background from im4_card_1.jpg
2025-08-05 19:04:39,374 - src.model.id_card_processor - INFO - Enhancing im4_card_1.jpg
2025-08-05 19:04:39,375 - src.model.id_card_processor - INFO - Normalizing im4_card_1.jpg
2025-08-05 19:04:39,379 - src.model.id_card_processor - INFO - Processed im4_card_1.jpg
2025-08-05 19:04:39,379 - src.model.id_card_processor - INFO - Processing 12/42: im5_card_1.jpg
2025-08-05 19:04:39,389 - src.model.id_card_processor - INFO - Removing background from im5_card_1.jpg
2025-08-05 19:04:39,842 - src.model.id_card_processor - INFO - Enhancing im5_card_1.jpg
2025-08-05 19:04:39,843 - src.model.id_card_processor - INFO - Normalizing im5_card_1.jpg
2025-08-05 19:04:39,846 - src.model.id_card_processor - INFO - Processed im5_card_1.jpg
2025-08-05 19:04:39,846 - src.model.id_card_processor - INFO - Processing 13/42: im5_card_2.jpg
2025-08-05 19:04:39,859 - src.model.id_card_processor - INFO - Removing background from im5_card_2.jpg
2025-08-05 19:04:42,430 - src.model.id_card_processor - INFO - Enhancing im5_card_2.jpg
2025-08-05 19:04:42,434 - src.model.id_card_processor - INFO - Normalizing im5_card_2.jpg
2025-08-05 19:04:42,438 - src.model.id_card_processor - INFO - Processed im5_card_2.jpg
2025-08-05 19:04:42,439 - src.model.id_card_processor - INFO - Processing 14/42: im6_card_1.jpg
2025-08-05 19:04:42,449 - src.model.id_card_processor - INFO - Removing background from im6_card_1.jpg
2025-08-05 19:04:47,647 - src.model.id_card_processor - INFO - Enhancing im6_card_1.jpg
2025-08-05 19:04:47,652 - src.model.id_card_processor - INFO - Normalizing im6_card_1.jpg
2025-08-05 19:04:47,657 - src.model.id_card_processor - INFO - Processed im6_card_1.jpg
2025-08-05 19:04:47,657 - src.model.id_card_processor - INFO - Processing 15/42: im6_card_2.jpg
2025-08-05 19:04:47,680 - src.model.id_card_processor - INFO - Removing background from im6_card_2.jpg

View File

@@ -1,9 +1,9 @@
"""
Data Augmentation Package
Data Augmentation Package for ID Cards
"""
__version__ = "1.0.0"
__author__ = "OCR Data Augmentation Tool"
__version__ = "2.0.0"
__author__ = "ID Card Data Augmentation Tool"
from .utils import *
from .image_processor import ImageProcessor
@@ -12,12 +12,12 @@ from .config_manager import ConfigManager
__all__ = [
"ImageProcessor",
"DataAugmentation",
"DataAugmentation",
"ConfigManager",
"setup_logging",
"get_image_files",
"load_image",
"save_image",
"validate_image",
"create_augmented_filename",
"print_progress",
]

Binary file not shown.

View File

@@ -1,40 +0,0 @@
"""
Configuration file for data augmentation
"""
import os
from pathlib import Path
# Paths
BASE_DIR = Path(__file__).parent.parent
DATA_DIR = BASE_DIR / "data"
INPUT_IMAGES_DIR = DATA_DIR / "dataset" / "training_data" / "images"
OUTPUT_DIR = DATA_DIR / "augmented_data"
# Data augmentation parameters
AUGMENTATION_CONFIG = {
"rotation_range": 15, # degrees
"width_shift_range": 0.1, # fraction of total width
"height_shift_range": 0.1, # fraction of total height
"brightness_range": [0.8, 1.2], # brightness factor
"zoom_range": [0.9, 1.1], # zoom factor
"horizontal_flip": True,
"vertical_flip": False,
"fill_mode": "nearest",
"cval": 0,
"rescale": 1./255,
}
# Processing parameters
PROCESSING_CONFIG = {
"target_size": (224, 224), # (width, height)
"batch_size": 32,
"num_augmentations": 3, # number of augmented versions per image
"save_format": "jpg",
"quality": 95,
}
# Supported image formats
SUPPORTED_FORMATS = ['.jpg', '.jpeg', '.png', '.bmp', '.tiff']
# Create output directory if it doesn't exist
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)

View File

@@ -1,5 +1,5 @@
"""
Data augmentation class for image augmentation - ONLY ROTATION
Data augmentation class for image augmentation - ONLY ROTATION with quality preservation
"""
import cv2
import numpy as np
@@ -23,32 +23,134 @@ class DataAugmentation:
self.config = config or {}
self.image_processor = ImageProcessor()
def rotate_image(self, image: np.ndarray, angle: float) -> np.ndarray:
def rotate_image_preserve_quality(self, image: np.ndarray, angle: float) -> np.ndarray:
"""
Rotate image by given angle
Rotate image by given angle with white background and crop to preserve quality
Args:
image: Input image
angle: Rotation angle in degrees
Returns:
Rotated image
Rotated and cropped image
"""
height, width = image.shape[:2]
center = (width // 2, height // 2)
# Calculate new dimensions for rotation
angle_rad = math.radians(angle)
cos_val = abs(math.cos(angle_rad))
sin_val = abs(math.sin(angle_rad))
# Calculate new width and height
new_width = int(width * cos_val + height * sin_val)
new_height = int(height * cos_val + width * sin_val)
# Create larger canvas with white background
canvas = np.ones((new_height, new_width, 3), dtype=np.uint8) * 255
# Calculate offset to center the image
offset_x = (new_width - width) // 2
offset_y = (new_height - height) // 2
# Ensure offsets are valid
if offset_x >= 0 and offset_y >= 0 and offset_x + width <= new_width and offset_y + height <= new_height:
# Place original image in center of canvas
canvas[offset_y:offset_y+height, offset_x:offset_x+width] = image
else:
# If calculation is wrong, use a simpler approach
canvas = np.ones((max(height, width) * 2, max(height, width) * 2, 3), dtype=np.uint8) * 255
center_y, center_x = canvas.shape[0] // 2, canvas.shape[1] // 2
start_y = center_y - height // 2
start_x = center_x - width // 2
canvas[start_y:start_y+height, start_x:start_x+width] = image
new_width, new_height = canvas.shape[1], canvas.shape[0]
# Calculate center for rotation
center = (new_width // 2, new_height // 2)
# Create rotation matrix
rotation_matrix = cv2.getRotationMatrix2D(center, angle, 1.0)
# Perform rotation
rotated = cv2.warpAffine(image, rotation_matrix, (width, height),
borderMode=cv2.BORDER_REPLICATE)
rotated = cv2.warpAffine(canvas, rotation_matrix, (new_width, new_height),
borderMode=cv2.BORDER_CONSTANT, borderValue=(255, 255, 255))
# Crop white borders to get the actual image content
rotated = self._crop_white_borders(rotated)
return rotated
def _crop_white_borders(self, image: np.ndarray) -> np.ndarray:
"""
Crop white borders from image to get the actual content
Args:
image: Input image with white borders
Returns:
Cropped image without white borders
"""
# Convert to grayscale for edge detection
gray = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)
# Find non-white pixels (content)
non_white = gray < 250 # Threshold for white pixels
# Find bounding box of content
coords = cv2.findNonZero(non_white.astype(np.uint8))
if coords is not None:
x, y, w, h = cv2.boundingRect(coords)
return image[y:y+h, x:x+w]
return image
def resize_preserve_aspect(self, image: np.ndarray, target_size: Tuple[int, int]) -> np.ndarray:
"""
Resize image preserving aspect ratio with white padding
Args:
image: Input image
target_size: Target size (width, height)
Returns:
Resized image with preserved aspect ratio
"""
target_width, target_height = target_size
img_height, img_width = image.shape[:2]
# Calculate aspect ratios
target_aspect = target_width / target_height
img_aspect = img_width / img_height
if img_aspect > target_aspect:
# Image is wider than target
new_width = target_width
new_height = int(target_width / img_aspect)
else:
# Image is taller than target
new_height = target_height
new_width = int(target_height * img_aspect)
# Resize image
resized = cv2.resize(image, (new_width, new_height), interpolation=cv2.INTER_AREA)
# Create white background
result = np.ones((target_height, target_width, 3), dtype=np.uint8) * 255
# Calculate offset to center the image
offset_x = (target_width - new_width) // 2
offset_y = (target_height - new_height) // 2
# Place resized image in center
result[offset_y:offset_y+new_height, offset_x:offset_x+new_width] = resized
return result
def augment_single_image(self, image: np.ndarray, num_augmentations: int = None) -> List[np.ndarray]:
"""
Apply rotation augmentation to a single image
Apply rotation augmentation to a single image with quality preservation
Args:
image: Input image
@@ -65,12 +167,18 @@ class DataAugmentation:
angles = rotation_config.get("angles", [30, 60, 120, 150, 180, 210, 240, 300, 330])
for i in range(num_augmentations):
# Start with original image
augmented = image.copy()
# Apply rotation with random angle from the specified list
# Apply rotation with quality preservation
if rotation_config.get("enabled", False):
angle = random.choice(angles)
augmented = self.rotate_image(augmented, angle)
augmented = self.rotate_image_preserve_quality(augmented, angle)
# Resize preserving aspect ratio
target_size = self.image_processor.target_size
if target_size:
augmented = self.resize_preserve_aspect(augmented, target_size)
augmented_images.append(augmented)
@@ -78,7 +186,7 @@ class DataAugmentation:
def augment_image_file(self, image_path: Path, output_dir: Path, num_augmentations: int = None) -> List[Path]:
"""
Augment a single image file and save results
Augment a single image file and save results with quality preservation
Args:
image_path: Path to input image
@@ -88,8 +196,8 @@ class DataAugmentation:
Returns:
List of paths to saved augmented images
"""
# Load image
image = load_image(image_path, self.image_processor.target_size)
# Load image without resizing to preserve original quality
image = load_image(image_path, None) # Load original size
if image is None:
return []

View File

@@ -1,14 +1,14 @@
"""
Image processing class for basic image operations
Image processing class for data augmentation
"""
import cv2
import numpy as np
from pathlib import Path
from typing import Tuple, Optional, List
from utils import load_image, save_image, validate_image, get_image_files
from typing import Tuple, Optional
from utils import load_image
class ImageProcessor:
"""Class for basic image processing operations"""
"""Class for image processing operations used in data augmentation"""
def __init__(self, target_size: Tuple[int, int] = None):
"""
@@ -17,67 +17,7 @@ class ImageProcessor:
Args:
target_size: Target size for image resizing (width, height)
"""
self.target_size = target_size or (224, 224) # Default size
def load_and_preprocess(self, image_path: Path) -> Optional[np.ndarray]:
"""
Load and preprocess image
Args:
image_path: Path to image file
Returns:
Preprocessed image as numpy array or None if failed
"""
if not validate_image(image_path):
print(f"Invalid image file: {image_path}")
return None
image = load_image(image_path, self.target_size)
if image is None:
return None
# Normalize pixel values
image = image.astype(np.float32) / 255.0
return image
def resize_image(self, image: np.ndarray, target_size: Tuple[int, int]) -> np.ndarray:
"""
Resize image to target size
Args:
image: Input image as numpy array
target_size: Target size (width, height)
Returns:
Resized image
"""
return cv2.resize(image, target_size, interpolation=cv2.INTER_AREA)
def normalize_image(self, image: np.ndarray) -> np.ndarray:
"""
Normalize image pixel values to [0, 1]
Args:
image: Input image
Returns:
Normalized image
"""
return image.astype(np.float32) / 255.0
def denormalize_image(self, image: np.ndarray) -> np.ndarray:
"""
Denormalize image pixel values to [0, 255]
Args:
image: Input image (normalized)
Returns:
Denormalized image
"""
return (image * 255).astype(np.uint8)
self.target_size = target_size or (640, 640) # Default size for ID cards
def get_image_info(self, image_path: Path) -> dict:
"""
@@ -107,68 +47,4 @@ class ImageProcessor:
}
except Exception as e:
print(f"Error getting image info for {image_path}: {e}")
return {}
def batch_process_images(self, input_dir: Path, output_dir: Path) -> List[Path]:
"""
Process all images in a directory
Args:
input_dir: Input directory containing images
output_dir: Output directory for processed images
Returns:
List of processed image paths
"""
image_files = get_image_files(input_dir)
processed_files = []
print(f"Found {len(image_files)} images to process")
for i, image_path in enumerate(image_files):
print_progress(i + 1, len(image_files), "Processing images")
# Load and preprocess image
image = self.load_and_preprocess(image_path)
if image is None:
continue
# Create output path
output_path = output_dir / image_path.name
# Denormalize for saving
image = self.denormalize_image(image)
# Save processed image
if save_image(image, output_path):
processed_files.append(output_path)
print(f"\nProcessed {len(processed_files)} images successfully")
return processed_files
def create_thumbnail(self, image: np.ndarray, size: Tuple[int, int] = (100, 100)) -> np.ndarray:
"""
Create thumbnail of image
Args:
image: Input image
size: Thumbnail size (width, height)
Returns:
Thumbnail image
"""
return cv2.resize(image, size, interpolation=cv2.INTER_AREA)
def convert_to_grayscale(self, image: np.ndarray) -> np.ndarray:
"""
Convert image to grayscale
Args:
image: Input image (RGB)
Returns:
Grayscale image
"""
if len(image.shape) == 3:
return cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)
return image
return {}

View File

@@ -32,14 +32,7 @@ def get_image_files(directory: Path) -> List[Path]:
image_files.extend(directory.glob(f"*{ext.upper()}"))
return sorted(image_files)
def validate_image(image_path: Path) -> bool:
"""Validate if file is a valid image"""
try:
with Image.open(image_path) as img:
img.verify()
return True
except Exception:
return False
def load_image(image_path: Path, target_size: Tuple[int, int] = None) -> Optional[np.ndarray]:
"""Load and resize image"""
@@ -83,9 +76,7 @@ def create_augmented_filename(original_path: Path, index: int, suffix: str = "au
suffix = f"_{suffix}_{index:02d}"
return original_path.parent / f"{stem}{suffix}{original_path.suffix}"
def get_file_size_mb(file_path: Path) -> float:
"""Get file size in MB"""
return file_path.stat().st_size / (1024 * 1024)
def print_progress(current: int, total: int, prefix: str = "Progress"):
"""Print progress bar"""