How is YouTube using AI to Transform Shorts?

YouTube is making big moves in the way creators produce Shorts.

The platform is introducing new AI-powered features that turn ordinary photos into dynamic videos and offer playful generative effects for selfies and doodles.

Engineers and product leads at YouTube and Google Cloud have shared how these tools work and how they manage to run smoothly on mobile devices.

Sarah Ali, YouTube’s Vice President of Product Management for Shorts, says the aim is to enhance creativity while keeping the experience fast and accessible:

“Creators are getting a new ‘Photo to video’ feature, letting a single photo from a camera roll turn into a moving clip.”

Let’s explore how YouTube is using AI to transform Shorts.

Photo to Video

The new “photo to video” feature allows creators to animate still images with minimal effort.

Landscape shots can appear to move, group photos can feel alive, and casual snaps can be given cinematic motion.

Generative effects take this further, transforming doodles into detailed images or reimagining selfies in playful ways. Users can create underwater scenes or even produce videos with a virtual twin.

These effects are powered by Veo 2, with an upgrade to Veo 3 expected before the end of summer 2025.

Complementing these tools, YouTube has launched an “AI playground”.

This space provides pre-filled prompts and examples, letting users instantly generate videos, music, and images.

Ali explained that every AI-generated creation includes SynthID watermarks and labels, ensuring transparency about what is machine-created.

Running AI Effects Smoothly on Phones

One of the biggest technical challenges has been getting complex AI effects to run on mobile devices without lag.

Google Cloud’s Andrey Vakunov and YouTube’s Adam Svystun outlined the solution.

They started with large generative AI models like StyleGAN2 and DeepMind’s Imagen, which create detailed edits but are too heavy for real-time use.

To tackle this, they employed knowledge distillation. A large “teacher” model trains on millions of images and transfers its understanding to a smaller “student” model.

This compact model can run directly on a phone while maintaining the teacher’s ability to produce detailed effects.

Training datasets included diverse ages, genders, and skin tones measured with the Monk Skin Tone Scale, with additional challenges like glasses or different lighting conditions to prepare the model for real-world use.

Preserving Facial Accuracy

Maintaining the integrity of a person’s face is critical in AI editing. The so-called “inversion problem” occurs when effects distort features, skin tone, or clothing.

The YouTube team uses pivotal tuning inversion to train models that preserve facial details. This allows creators to apply makeup, cartoon styling, or other effects without changing a person’s identity.

Once trained, the smaller model is paired with MediaPipe, an open-source framework.

MediaPipe detects faces, applies AI effects, and recombines the image into the video, all in under 33 milliseconds per frame. This keeps videos smooth at over 30 frames per second.

These new tools have already powered over 20 real-time effects on Shorts, including themed masks like “Risen zombie” and expression tools such as “Never blink” or “Always smile”.

Latency is impressively low, at six milliseconds on a Pixel 8 Pro and 10.6 milliseconds on an iPhone 13.

Ali emphasised the purpose of these tools: “These tools make Shorts easier and more fun, but creators themselves are the real draw.”

YouTube sees AI as a support for personal creativity, not a replacement for it. For creators, this means more freedom to experiment, animate, and enhance content while keeping the authentic voice of their channel intact.

YouTube’s AI updates are a clear signal that the platform is pushing the boundaries of mobile video editing.

The rollout begins in the United States, Canada, Australia, and New Zealand, with more regions expected later in 2025.

For creators, the future of Shorts is not just about what you capture, but what AI can help you create from it.