Add benchmark on video #18
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Reviewer's Guide by SourceryThis pull request introduces video benchmarking capabilities to the image augmentation benchmark suite. It includes video loading, applying transformations, and measuring performance for Albumentations, TorchVision, and Kornia. The changes involve adding new video implementations for each library, updating transform specifications, and providing scripts for running benchmarks and comparing results. The implementation uses float16 precision for TorchVision and Kornia on GPU to improve performance. Additionally, the PR includes scripts for running benchmarks and generating comparison tables. Sequence diagram for video loading and transformation with KorniasequenceDiagram
participant User
participant VideoBenchmarkRunner
participant KorniaVideoImpl
participant KorniaLibrary
participant GPU
User->>VideoBenchmarkRunner: run()
VideoBenchmarkRunner->>VideoBenchmarkRunner: load_videos()
VideoBenchmarkRunner->>KorniaLibrary: read_video_kornia(path)
activate KorniaLibrary
KorniaLibrary->>GPU: Load video frames and convert to float16
activate GPU
GPU-->>KorniaLibrary: Returns video frames (float16)
deactivate GPU
KorniaLibrary-->>VideoBenchmarkRunner: Returns video tensor (T, C, H, W)
deactivate KorniaLibrary
VideoBenchmarkRunner->>KorniaVideoImpl: __call__(transform, video)
activate KorniaVideoImpl
KorniaVideoImpl->>GPU: Apply transform to video tensor (float16)
activate GPU
GPU-->>KorniaVideoImpl: Returns transformed video tensor (float16)
deactivate GPU
KorniaVideoImpl-->>VideoBenchmarkRunner: Returns transformed video tensor (T, C, H, W)
deactivate KorniaVideoImpl
VideoBenchmarkRunner-->>User: Returns benchmark results
Updated class diagram for AlbumentationsVideoImplclassDiagram
class AlbumentationsVideoImpl {
+Resize(params: dict) BasicTransform
+RandomCrop128(params: dict) BasicTransform
+RandomResizedCrop(params: dict) BasicTransform
+CenterCrop128(params: dict) BasicTransform
+HorizontalFlip(params: dict) BasicTransform
+VerticalFlip(params: dict) BasicTransform
+Pad(params: dict) BasicTransform
+Rotate(params: dict) BasicTransform
+Affine(params: dict) BasicTransform
+Perspective(params: dict) BasicTransform
+Elastic(params: dict) BasicTransform
+ColorJitter(params: dict) BasicTransform
+ChannelShuffle(params: dict) BasicTransform
+Grayscale(params: dict) BasicTransform
+RGBShift(params: dict) BasicTransform
+GaussianBlur(params: dict) BasicTransform
+GaussianNoise(params: dict) BasicTransform
+Invert(params: dict) BasicTransform
+Posterize(params: dict) BasicTransform
+Solarize(params: dict) BasicTransform
+Sharpen(params: dict) BasicTransform
+AutoContrast(params: dict) BasicTransform
+Equalize(params: dict) BasicTransform
+Normalize(params: dict) BasicTransform
+Erasing(params: dict) BasicTransform
+JpegCompression(params: dict) BasicTransform
+RandomGamma(params: dict) BasicTransform
+PlankianJitter(params: dict) BasicTransform
+MedianBlur(params: dict) BasicTransform
+MotionBlur(params: dict) BasicTransform
+CLAHE(params: dict) BasicTransform
+Brightness(params: dict) BasicTransform
+Contrast(params: dict) BasicTransform
+CoarseDropout(params: dict) BasicTransform
+Blur(params: dict) BasicTransform
+HSV(params: dict) BasicTransform
+ChannelDropout(params: dict) BasicTransform
+LinearIllumination(params: dict) BasicTransform
+CornerIllumination(params: dict) BasicTransform
+GaussianIllumination(params: dict) BasicTransform
+Hue(params: dict) BasicTransform
+PlasmaBrightness(params: dict) BasicTransform
+PlasmaContrast(params: dict) BasicTransform
+PlasmaShadow(params: dict) BasicTransform
+Rain(params: dict) BasicTransform
+SaltAndPepper(params: dict) BasicTransform
+Saturation(params: dict) BasicTransform
+Snow(params: dict) BasicTransform
+OpticalDistortion(params: dict) BasicTransform
+Shear(params: dict) BasicTransform
+ThinPlateSpline(params: dict) BasicTransform
+__call__(transform: BasicTransform, video: np.ndarray) np.ndarray
}
note for AlbumentationsVideoImpl "Applies Albumentations augmentations to video arrays, utilizing CPU."
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey @ternaus - I've reviewed your changes - here's some feedback:
Overall Comments:
- Consider adding a script to automatically download sample videos for benchmarking.
- It might be helpful to include a README file explaining how to run the video benchmarks and interpret the results.
Here's what I looked at during the review
- 🟢 General issues: all looks good
- 🟢 Security: all looks good
- 🟢 Testing: all looks good
- 🟡 Complexity: 1 issue found
- 🟢 Documentation: all looks good
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| return torch.tensor(data, device=device, dtype=dtype) | ||
|
|
||
|
|
||
| class KorniaVideoImpl: |
There was a problem hiding this comment.
issue (complexity): Consider using a mapping of transform names to factory lambdas to reduce code duplication and improve maintainability by replacing static methods with calls to a central transform creation function, handling special cases separately if needed .
Consider reducing duplication by using a mapping of transform names to their corresponding factory lambdas. For example:
AUG_MAP: dict[str, Callable[[dict[str, Any]], Kaug.AugmentationBase2D]] = {
"ColorJitter": lambda p: Kaug.ColorJitter(
brightness=p["brightness"],
contrast=p["contrast"],
saturation=p["saturation"],
hue=p["hue"],
p=1,
same_on_batch=True
).to(device),
"AutoContrast": lambda p: Kaug.RandomAutoContrast(p=1, same_on_batch=True).to(device),
# ... add other mappings similarly ...
}
def create_transform(name: str, params: dict[str, Any]) -> Kaug.AugmentationBase2D:
try:
return AUG_MAP[name](params)
except KeyError:
raise ValueError(f"Unknown transform: {name}")You can then replace the many static methods with calls to create_transform. For transforms with special logic, consider overriding or handling those cases separately. This approach preserves functionality while reducing the boilerplate and complexity.
tools/compare_video_results.py
Outdated
Comment on lines
33
to
35
| if std is not None: | ||
| return f"{time_ms:.2f} ± {std:.2f}" | ||
| return f"{time_ms:.2f}" |
There was a problem hiding this comment.
suggestion (code-quality): We've found these issues:
- Lift code into else after jump in control flow (
reintroduce-else) - Replace if statement with if expression (
assign-if-exp)
Suggested change
| if std is not None: | |
| return f"{time_ms:.2f} ± {std:.2f}" | |
| return f"{time_ms:.2f}" | |
| return f"{time_ms:.2f} ± {std:.2f}" if std is not None else f"{time_ms:.2f}" |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Addresses: #10
Summary by Sourcery
Adds video benchmarking capabilities to the project, including video loading, transformation implementations for Albumentations, Torchvision and Kornia, and benchmark running. It also includes scripts to run the benchmarks and compare the results.
New Features:
Tests: