Looking back at the response to my previous article
https://www.publish0x.com/short-stories/how-i-built-a-short-animated-film-using-free-ai-tools-withou-xjydngq
I suspect readers were not interested in the film itself as much as in the process behind it.
The internet is full of impressive AI demos, but far fewer people describe what happens when you actually try to build something from those tools. In practice, the work becomes a sequence of small technical compromises: short clips that must be reused, animations that behave unpredictably, characters that move when they should stay still.
What looks like “AI filmmaking” from the outside is often closer to editing, masking, freezing frames, and stitching fragments together until a narrative emerges.
In other words, the real craft is not generating images.
It is forcing unpredictable tools to behave like a film crew.
That realization shaped the way I approached this new project.
When I started adapting the story into a visual script, my goal was simple: remove descriptive prose and replace it with visuals. In other words, treat the text as a screenplay — keep only dialogue and a narrator where absolutely necessary.
But this story introduced an unexpected complication.
The narrator was not an external omnipotent storyteller. It was the internal voice of one of the characters — Jenny’s thoughts.
That meant the film actually required two different versions of the same voice:
- Jenny speaking aloud to the robot.
- Jenny thinking internally, commenting on the situation.
These two voices must be clearly distinguishable to the audience, yet still feel like they belong to the same person.
In human filmmaking this is trivial. Actors do it instinctively. Internal voice and spoken voice differ in subtle ways:
- timbre
- pitch
- rhythm
- confidence
- emotional weight
- degree of politeness
- the sense of “distance” from the listener
As a film director you can even use two different actors with the right chemistry.
However, in the situation of one man doing everything alone, these qualities are extremely difficult to describe technically, even though the human ear detects them instantly.
With AI voices I discovered something interesting. The voices could be made different, but they rarely felt related. They either sounded like two completely different people, or like the same person reading two unrelated paragraphs.
What I needed was something in between — the same personality expressed through two mental states.
After testing many voices, I realized I could not reliably achieve that balance with the tools available to me.
That forced an important decision.
Instead of pretending the story was a screenplay, I returned it to what it originally was: a narrated short story accompanied by visuals.
In other words, the film became a visualized narration, not a pure cinematic script.
This was not the solution I initially wanted. But it turned out to be the correct one for the tools I had.
Independent creators working with AI tools quickly learn a simple rule:
The art is not only in what you create, but in recognizing the limits of the tools and adapting the form to them.
If I had professional actors, I would probably have separated the voices. But working alone with free tools, the cleaner solution was to keep a single narrative voice and let the visuals carry the rest of the story.
Paradoxically, that limitation simplified the film.
The narration now acts as Jenny’s continuous internal perspective, while the visuals show the external events — the strange visitor, the malfunction, and the realization that something very unnatural has entered the café that night.
The AI tools available to me today are much better at illustrating narration than replacing it. In many cases the most effective format is not “AI cinema,” but illustrated storytelling — something closer to graphic literature or animated audiobooks.
Another challenge appeared when generating the actual video fragments.
To keep the story visually coherent, every scene had to preserve the same character appearance. The easiest way to achieve that was to feed the generator a static reference image for each shot. That part worked: the characters stayed recognizable.
But a new problem emerged.
When both an image and a prompt were provided, the generator almost completely ignored the prompt and simply tried to animate the image itself. The resulting clips often had very little to do with the behavior described in the prompt. Characters walked when they should have remained still. Gestures appeared that were never requested. Sometimes the motion looked like a parade march rather than a tired person entering a café.
After several attempts I realized something important: the generator does not really treat the prompt as a strict instruction. Instead, it treats the previously generated video as the new reference point.
So the practical workflow became unexpectedly conversational.
Instead of trying to get the correct result from the first prompt, I started accepting the first clip as a rough draft and then asking for corrections relative to it.
The exchange literally sounded like this:
“Thank you, that’s pretty good. However, can you change…”
And then I would describe what needed to be adjusted in the clip that had just been produced.
In other words, the generator behaved less like a rendering engine and more like an assistant that must be directed step by step. The first attempt establishes the visual structure, and subsequent corrections gradually steer the behavior toward what the scene actually requires.
This iterative approach turned out to be far more reliable than trying to write a perfect prompt from the beginning.
AI video tools today behave less like machines and more like junior collaborators. They produce something plausible quickly, but the real work lies in guiding and correcting them until the result matches the intention.
The short film below is an example of that hybrid process.
