[gsoc 26]: Reaching out to mentors of project#4 - "OpenVINO GenAI: Add Image-to-Video Support to LTX Video Generation Pipeline"

[FAbdullah17]

Hi @likholat @sgonorov and Everyone! hope you're doing well.
I'm Fahad a 4th year BS Artificial Intelligence student with expertise in AI & ML and AI/ML Engineer with 2+ years of hands-on experience building and deploying generative AI systems, LLM pipelines, and scalable inference services using PyTorch, Hugging Face, FastAPI, Docker, and Kubernetes. some of my work is:

• Virtual-TryOn-System
• GRS-Gemini
• Barren-Plateau-Mitigation-QNN

I’m particularly interested in Project#4 (OpenVINO GenAI Image-to-Video pipeline extension) as it aligns closely with my experience in generative pipelines, model optimization, and production-grade AI systems.
I have thoroughly explored the OpenVINO GenAI repository https://github.com/openvinotoolkit/openvino.genai and relevant documentation to understand its current pipeline implementations. The OpenVINO GenAI library provides ready-to-use generative AI pipelines with simple C++ and Python APIs on top of the OpenVINO runtime, without requiring external runtime dependencies.

For example, Python inference for text-to-image pipelines using Stable Diffusion is structured as:

import openvino_genai as ov_genai

pipe = ov_genai.Text2ImagePipeline(model_path, "CPU")
image_tensor = pipe.generate(prompt)

This shows model loading and inference abstraction already implemented.

Similarly, existing video generation support in GenAI currently focuses on text-to-video using the LTX-Video model:

import openvino_genai as ov_genai

pipe = ov_genai.Text2VideoPipeline(model_path, "CPU")
video = pipe.generate(prompt).video

This confirms that a dedicated video generation pipeline already exists for text conditioning.

Project #4 extends the video diffusion pipeline to support Image + Text → Video generation by incorporating image conditioning. Based on analysis of OpenVINO GenAI pipelines:

Key Integration Points:

• GenAI pipelines wrap compiled models and handle tokenization, scheduler logic, and inference
• Diffusion model samples demonstrate modular assembly (text encoder, UNet, VAE decoder) with benchmark scripts

Design Implications for Image-to-Video:

• Pipeline must accept image tensor + text prompt
• Generative loop integrates image condition with text embeddings into temporal diffusion model
• API maintains consistency: Pipeline(model_path, device) + .generate(...)
• Extend benchmark/sample scripts for validation

The GenAI design is modular and extensible at the pipeline level, making image-conditioned video a natural extension.

I've done some research and built a rough architectural idea of how things will work and how we will implement the pipeline:

To start contributing, I have a few questions:

Would issues related to pipeline structure (like the existing Text2ImagePipeline or Text2VideoPipeline) be a good place to begin? I've been looking at src/cpp/pipelines/ and see how IR models are loaded and inference is managed — would contributing to model loading patterns, scheduler integration, or benchmarking tools for I2V metrics make sense as a first step?

I'm hoping to start with something focused that builds toward the full Image-to-Video pipeline — whether that's extending a utility, adding a helper class, or prototyping a component. If there's an open issue (or planned task) along these lines, I'd love to take it on.
Looking forward to next steps from you guys!!

[FAbdullah17]

Hi @likholat @sgonorov, just wanted to gently follow up on my previous message from two weeks ago about Project #4 (Image-to-Video for LTX Video). I'm still very interested in contributing and would be grateful for any initial thoughts or guidance when you have a moment. Thanks!

[likholat]

Hi Fahad,

Sorry for the late reply, and thanks a lot for your application and for the detailed message.

Please note that the final proposal must be submitted through the GSoC portal webapp to be considered.

Could you please let us know whether you have already completed any good first issues for the OpenVINO repository, or whether you have contributed any PRs to OpenVINO? At the moment, solving GFIs or making a solid contribution to OpenVINO (OpenVINO GenAI) is the requirement for applying.

As for the project itself, both the implementation of image-to-video support and the planning of its architecture are expected to be part of the program, rather than something that is already planned to be added beforehand

Thanks,
Anna

[FAbdullah17]

@likholat – Thank you for the response! I will submit my final proposal through the GSoC portal.

To meet the contribution requirement, I've been actively working on OpenVINO GenAI. Here are my contributions so far:

• PR #3575 (open) – Added Python docstrings to Text2VideoPipeline to improve help() output and align documentation with other pipelines.
• PR #3516 (under review) – Added robust error handling to the ReAct sample to prevent tool-call crashes.
• PR #3530 (closed) – Implemented a LangChain wrapper for OpenVINO GenAI; while this PR was closed to align with the recommended external integration approach, it demonstrates my ability to build integrations and was a valuable learning experience.

I'm excited about Project #4 (Image‑to‑Video) and will incorporate the insights from these contributions into my proposal. Thank you for the guidance!