Hiya! So I've been playing around with having Claude make videos for a bit now even had some success posting the results to TikTok (and setup a whole pipeline so Claude can generate and post autonomously). With the release of Nano Banana 2 Lite, I was curious show fast I could make the generation, so last night I gave it a whirl and got down to around 30s for short-form video.
It uses GLM-5.2 fast via Fireworks to generate the scripts and image prompts and, like I said, Nano Banana 2 Lite for the images, gpt-4o-mini-tts for the narration, and ffmpeg to string it all together and add the Ken Burns zoom effect (which still has a shake I haven't been able to get rid of). The video compilation proved to be the blocker once the rest was in place, but I was able to speed that up by putting it on a 64 vCPU EC2.
The cost might be the most interesting aspect as the short form videos tend to be about 25 cents. Almost 90% of that is the images, which are 3.336 cents a piece. Of course, running the big 64 core EC2 to allow for the creation isn't cheap.
It seems like on-demand AI video is coming, and I thought this was an interesting demo of how close it might be in at least one narrow video domain.
For example, I have several posts that explain concepts in detail, with supporting images. They're written in Markdown, and I keep refining them until I'm confident that the average person can read them and immediately understand the concept.
The problem is that text is becoming a less popular medium for learning, while short form videos are increasingly being used to explain the same ideas.
From what I can tell, this tool only lets you provide a prompt. Can it turn my existing Markdown posts and images into videos, or is it mainly intended for generating videos from simple prompts? I'm looking for something that preserves the explanations I've already written rather than creating something entirely new.