feat: Extend Lottie specification with VideoFrame asset support #149
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,6 +10,7 @@ | |
| "assets": [ | ||
| { | ||
| "id": "comp_0", | ||
| "ty": 0, | ||
| "nm": "time_remap_precomp", | ||
| "fr": 60, | ||
| "layers": [ | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,6 +10,7 @@ | |
| "assets": [ | ||
| { | ||
| "id": "comp_0", | ||
| "ty": 0, | ||
| "nm": "swing", | ||
| "fr": 60, | ||
| "layers": [ | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| { | ||
| "$schema": "https://json-schema.org/draft/2020-12/schema", | ||
| "type": "object", | ||
| "title": "Video Frame", | ||
| "description": "Asset representing a frame from a video at a specific timestamp.", | ||
| "allOf": [ | ||
| { | ||
| "$ref": "#/$defs/assets/asset" | ||
| }, | ||
| { | ||
| "$ref": "#/$defs/helpers/slottable-object" | ||
| }, | ||
| { | ||
| "type": "object", | ||
| "properties": { | ||
|
Member
There was a problem hiding this comment. I think we should introduce a |
||
| "ty": { | ||
| "title": "Type", | ||
| "description": "Asset type", | ||
| "type": "integer", | ||
| "const": 3 | ||
| }, | ||
| "vsid": { | ||
|
Member
There was a problem hiding this comment. Naming bike-shed: normally How about
Contributor
Author
There was a problem hiding this comment. Agree, refId makes more sense, updated |
||
| "title": "Video Asset ID", | ||
| "description": "ID of the video asset to extract frame from", | ||
| "type": "string" | ||
| }, | ||
| "t": { | ||
| "title": "Timestamp", | ||
| "description": "Timestamp in seconds where to extract the frame from the video", | ||
|
Member
There was a problem hiding this comment. How does this interact with start-time/time-stretch/general time remapping in referencing layers? Also actual time units seem a bit odd since everything else in Lottie is expressed as frame indices (ip, op, etc).
Contributor
Author
There was a problem hiding this comment. Actually, you are right, we can define time in frames here |
||
| "type": "number", | ||
| "minimum": 0 | ||
| }, | ||
| "w": { | ||
| "title": "Width", | ||
| "description": "Width of the frame", | ||
| "type": "number" | ||
| }, | ||
| "h": { | ||
| "title": "Height", | ||
| "description": "Height of the frame", | ||
| "type": "number" | ||
| } | ||
|
Comment on lines
+33
to
+42
Member
There was a problem hiding this comment. The video asset schema also has
Contributor
Author
There was a problem hiding this comment. @mbasaglia I was thinking about extracting a frame from VideoResource, specifying width, height, x, y from the top left corner. But you are right, as a first iteration we can drop w/h from frame |
||
| }, | ||
| "required": ["ty"], | ||
| "if": { | ||
| "required": ["sid"] | ||
| }, | ||
| "else": { | ||
| "required": ["ty", "vsid", "t"] | ||
| } | ||
| } | ||
| ] | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| { | ||
| "$schema": "https://json-schema.org/draft/2020-12/schema", | ||
| "type": "object", | ||
| "title": "Video", | ||
| "description": "Asset containing a video that can be referenced by layers or video frame assets.", | ||
| "allOf": [ | ||
| { | ||
| "$ref": "#/$defs/assets/asset" | ||
| }, | ||
| { | ||
| "$ref": "#/$defs/helpers/slottable-object" | ||
| }, | ||
| { | ||
| "$ref": "#/$defs/helpers/file-asset" | ||
| }, | ||
| { | ||
| "type": "object", | ||
| "properties": { | ||
| "ty": { | ||
| "title": "Type", | ||
| "description": "Asset type", | ||
| "type": "integer", | ||
| "const": 2 | ||
| }, | ||
| "w": { | ||
| "title": "Width", | ||
| "description": "Width of the video", | ||
| "type": "number" | ||
| }, | ||
| "h": { | ||
| "title": "Height", | ||
| "description": "Height of the video", | ||
| "type": "number" | ||
| }, | ||
| "duration": { | ||
| "title": "Duration", | ||
| "description": "Duration of the video in seconds", | ||
| "type": "number", | ||
| "minimum": 0 | ||
| } | ||
| }, | ||
| "required": ["ty"], | ||
| "if": { | ||
| "required": ["sid"] | ||
| }, | ||
| "else": { | ||
| "required": ["ty", "w", "h", "duration", "p"] | ||
| } | ||
| } | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| { | ||
| "$schema": "https://json-schema.org/draft/2020-12/schema", | ||
| "type": "object", | ||
| "title": "File Asset", | ||
| "description": "Common properties for file-based assets like images and videos", | ||
| "properties": { | ||
| "p": { | ||
| "title": "File Name", | ||
| "description": "Name of the file or a data url", | ||
| "type": "string" | ||
| }, | ||
| "u": { | ||
| "title": "File Path", | ||
| "description": "Path to the file", | ||
| "type": "string" | ||
| }, | ||
| "e": { | ||
| "title": "Embedded", | ||
| "description": "If '1', 'p' is a Data URL", | ||
| "$ref": "#/$defs/values/int-boolean" | ||
| } | ||
| }, | ||
| "allOf": [ | ||
| { | ||
| "if": { | ||
| "properties": { | ||
| "e": {"const": 1} | ||
| }, | ||
| "required": ["e"] | ||
| }, | ||
| "then": { | ||
| "properties": { | ||
| "p": {"$ref": "#/$defs/values/data-url"} | ||
| } | ||
| } | ||
| } | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,6 +10,7 @@ | |
| "assets": [ | ||
| { | ||
| "id": "image_0", | ||
| "ty": 1, | ||
| "w": 1024, | ||
| "h": 1024, | ||
| "u": "images/", | ||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this a SHOULD or a MUST?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This point is up for discussion.
The advantage of using MUST is that Lottie players could extract frames on the fly, which speeds up animation initialization and reduces memory usage before playback starts. At the same time, if players don’t treat the extracted frame as a static image but instead process it in real time, it could impose constraints on the video itself - for example, requiring more keyframes to ensure fast seeking, which would increase file size.
That’s why I would vote for MUST.
In the future, if we want to support optimization for initialization we could extend videoFrame with initialization: static | dynamic param to achieve that.