A fast and efficient strategy for true lossless cropping, yes., cropping !

[shodanx2]

Hello,

I would like to suggest a strategy for fast, lossless cropping

Unfortunately, this method would not be codec agnostic, but hopefully the code would me mostly reusable across different codecs

First, UI element, right now we specify start and stop frames.
To specify a crop, the stop and start frame would need to define the corners of a rectangle

How the method works

Video files are made out of quantization blocks that represent parts of the screen.

The strategy is, for each block, detect if they are inside or outside the crop region.
And then zero these blocks out.
Do this for keyframes and and inter frames.

This would preserve the video without a re-encode
And it should very very fast to do.

There would be some defects
When movement from outside the crop zone goes into the crop zone, would smear zeroes (blackness) onto the video.
I don't believe this would be much of a problem as large movement would trigger a new keyframe.

Another strategy would be to define two crop zone, inner and outer.
The outer zone would keep blocks that are going to move into the inner zone in the frame between now and the next keyframe.

This would increase the processing time quite a bit however so don't get hung up on this.

I think just trying to make the crop work by just blacking out region outside the crop zone would be very useful as a fast, no re-compress, rough cropping tool

[mifi]

Hi.
This is an interesting idea for sure, but I don’t know how to practically execute on it. I cannot find any API in ffmpeg to work on quantization blocks.

[shodanx2]

I have a slightly crazy suggestion and that would be to try and overwrite parts of the file directly with Zeroes and see what happens.

You could use a method similar to what oscilloscopes use to decipher an unknown signal with their trigger mechanism.

In electronics on an oscilloscope, you would set the trigger to fire when the waveform reaches a certain voltage, wait a specific delay and then start signal capture.

In the same vein, I would suggest to read the file and look first for the beginning of a new frame, I suspect there is a repeating bit pattern on every new frame. Then I suspect there is also a repeating bit pattern that indicates the end of a frame.

Then maybe for each files within those beginning and end of frames, for every file in the file, at the same rough number of bits from the start, zero out a group of bits. Small enough to likely be all within the same quantization block (I would say 500 bytes in a row, all zero). And say all at 33.5% of the number of bits between beginning and end of frame.

There is a chance this will zero out the header/footer of a quantization block, so moving the exact % might allow to nail exactly in the middle of a quantization block.

Then try to play the file in a damage tolerant mp4 player. Maybe a steady block of blackness (or stillness, for interframes) might appear ?

If that works, then close in on that section of the file with the zeroes in an hex editor and try to discover the start and end structures of the quantization blocks.

I think the whole process would be something like find beginning and end of mp4 frames, discriminates frames by stream type (video, audio, caption text, etc...), Select only video frames, inside of video frames, discover beginning and end of quantization blocks and zero out their insides.

At first zero out the blocks by number of bytes since beginning of mp4 frame, to always hit the same block.
Then make that function something that you can change the block's "byte address" on the fly and pipe the output directly into an mp4 for instant feedback.

If lucky you might be able zero out a specific block at a certain X,Y location on every frame.

From that it shouldn't be to difficult to discover how to map out which block goes to what X,Y location in the image. Hopefully these blocks are arranged in a simple left to right, top to bottom pattern that is easy to figure out.

I suspect the block locations and overall structure is going to be highly dependant on the codec used and not really a feature of the MP4 container format ?

[mifi]

This is a bit beyond me, but if you can provide some code or script that works, then it might be something to look into integrating