Local AI Video Generator: Build Your Own App (Free, Python, No GPU)

The fox in this video was generated by an AI video model, not in the cloud, but on a computer with no graphics card at all, in about three minutes. In this tutorial we build our own local AI video generator: a small Python web app where you type a prompt, click Generate, and get an MP4 you can play and download. It uses free software (Hugging Face diffusers and Flask) and free open models.

I will also be honest about what “zero cost” means and what hardware you really need. Every run below is real, on a machine with 8 CPU cores, 16 GB of RAM and no GPU.

⚡ TL;DR – About 110 lines of Python and HTML: Flask + diffusers, no front-end framework. – Wan 2.1 T2V 1.3B (Apache-2.0) did not fit on this CPU machine: it needs about 16 GB of RAM on CPU and only 10.8 GB was free. The app refused it with a clean error. – ModelScope text-to-video 1.7B on CPU: 24 frames at 256×256 in 203 seconds (7.9 s per step). – 2 steps gives pure noise; 25 steps is a good starting point. – With an NVIDIA card of 8 GB or more, switch to Wan 2.1 (official: about 4 minutes for 5 s of 480p on an RTX 4090).

Before you start

• Python 3.10+.
• Disk space for model weights (several GB per model).
• Hardware reality check: the code and models are free and there is no API bill, but these models prefer an NVIDIA GPU with 8 GB+ of video memory. CPU works only for the small model.
• Licences: Wan 2.1 is Apache-2.0 (commercial output allowed). ModelScope T2V is CC-BY-NC-4.0 (non-commercial only), low resolution, and can show a faint stock-footage watermark.
• Apple Silicon (mps) and GPU runs were not tested in the video.

Step 1: Pick the models

Why does Wan need about 16 GB on CPU when the video transformer is small? Its text encoder, UMT5-XXL, is 22.7 GB in full precision. Even at half precision that is over 11 GB just to read your prompt. On a GPU, the official scripts keep that encoder on the CPU to save video memory.

Step 2: Set up the environment

# requirements.txt
diffusers
transformers
accelerate
flask
imageio
imageio-ffmpeg
psutil

python3 -m venv .venv && source .venv/bin/activate
# No NVIDIA card: the CPU build of torch is a much smaller download
pip install torch --index-url https://download.pytorch.org/whl/cpu
# With an NVIDIA card, install the CUDA build of torch instead
pip install -r requirements.txt

Step 3: Describe the models in models.py

# models.py
MODELS = {
    "wan": {
        "id": "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
        "license": "Apache-2.0",
        "need_gb": 16,          # on CPU, mostly the UMT5-XXL text encoder
        "kwargs": {},
    },
    "modelscope": {
        "id": "ali-vilab/text-to-video-ms-1.7b",
        "license": "CC-BY-NC-4.0 (non-commercial)",
        "need_gb": 6,
        "kwargs": {"variant": "fp16"},   # smaller half-precision weights
    },
}

Step 4: Write generate.py

# generate.py
import time, torch, psutil
from diffusers import DiffusionPipeline
from diffusers.utils import export_to_video
from models import MODELS

if torch.cuda.is_available():
    DEVICE, DTYPE = "cuda", torch.float16
else:
    DEVICE, DTYPE = "cpu", torch.bfloat16      # half the memory of float32

def free_gb():
    if DEVICE == "cuda":
        free, _ = torch.cuda.mem_get_info()
        return free / 1e9
    return psutil.virtual_memory().available / 1e9

_cache = {}
def load(name):
    if name in _cache:
        return _cache[name]                    # load once, not every click
    m = MODELS[name]
    if free_gb() < m["need_gb"]:
        raise MemoryError(f"{name} needs about {m['need_gb']} GB, only {free_gb():.1f} GB free")
    pipe = DiffusionPipeline.from_pretrained(m["id"], torch_dtype=DTYPE, **m["kwargs"]).to(DEVICE)
    _cache[name] = pipe
    return pipe

def generate(name, prompt, out_path, width=256, height=256, frames=24, steps=25, seed=42):
    pipe = load(name)
    g = torch.Generator(device="cpu").manual_seed(seed)   # repeatable runs
    t0 = time.time()
    result = pipe(prompt=prompt, width=width, height=height,
                  num_frames=frames, num_inference_steps=steps, generator=g)
    export_to_video(result.frames[0], out_path, fps=8)
    return round(time.time() - t0, 1)

The memory check is the most important line. Without it, a model that does not fit can push the machine into swap, freeze it, or get other programs killed. With it, the user gets a clear message instead.

Step 5: Write the Flask server, app.py

# app.py
import os, uuid
from flask import Flask, jsonify, request, send_from_directory
from generate import DEVICE, free_gb, generate

app = Flask(__name__)
OUT = "videos"; os.makedirs(OUT, exist_ok=True)

@app.get("/")
def home():
    return send_from_directory(".", "index.html")

@app.get("/info")
def info():
    return jsonify(device=DEVICE, free_gb=round(free_gb(), 1))

@app.post("/generate")
def gen():
    j = request.get_json()
    name = f"{uuid.uuid4().hex}.mp4"
    try:
        secs = generate(j["model"], j["prompt"], os.path.join(OUT, name),
                        int(j["width"]), int(j["height"]), int(j["frames"]), int(j["steps"]))
    except MemoryError as e:
        return jsonify(error=str(e)), 507        # Insufficient Storage
    return jsonify(url=f"/videos/{name}", seconds=secs)

@app.get("/videos/<path:f>")
def video(f):
    return send_from_directory(OUT, f)

if __name__ == "__main__":
    app.run(port=7860)

Step 6: Build the page, index.html

<!doctype html><meta charset="utf-8"><title>Local AI Video</title>
<p id="dev">…</p>
<textarea id="p" rows="3" cols="60">a red fox running through snow, cinematic</textarea><br>
<select id="m"><option value="modelscope">ModelScope 1.7B</option><option value="wan">Wan 2.1 1.3B</option></select>
W <input id="w" value="256" size="4"> H <input id="h" value="256" size="4">
Frames <input id="f" value="24" size="3"> Steps <input id="s" value="25" size="3">
<button id="go">Generate</button>
<p id="msg"></p><video id="v" controls width="512"></video><a id="dl" download>Download</a>
<script>
const $ = id => document.getElementById(id);
fetch("/info").then(r => r.json()).then(i => $("dev").textContent = `Device: ${i.device}, ${i.free_gb} GB free`);
$("go").onclick = async () => {
  $("msg").textContent = "Generating…";
  const body = { prompt: $("p").value, model: $("m").value, width: $("w").value,
                 height: $("h").value, frames: $("f").value, steps: $("s").value };
  const r = await fetch("/generate", { method: "POST", headers: { "Content-Type": "application/json" },
                                       body: JSON.stringify(body) });
  const j = await r.json();
  if (j.error) { $("msg").textContent = j.error; return; }
  $("msg").textContent = `Done in ${j.seconds} s`;
  $("v").src = j.url; $("dl").href = j.url;
};
</script>

Step 7: Run it and test the limits

python app.py      # http://localhost:7860

/info reported device cpu, 10.8 GB free. Then the real runs:

1. Wan 2.1, 480×320, 17 frames: HTTP 507, “wan needs about 16 GB, only 10.8 GB free”. The check stopped it before it could crash the machine. To be clear, Wan did not run here. On a GPU with 8 GB or more, the same app selects CUDA and Wan is the model to use.
2. ModelScope, 256×256, 24 frames, 25 steps: 203 seconds, 7.9 s per step on 8 CPU cores, giving 3 seconds of video at 8 fps. The unedited result, a red fox running through snow, is small and a bit soft, and a faint Shutterstock watermark is visible: a known flaw from training on stock-footage previews.
3. From the web UI, ModelScope, 256×256, 16 frames, 20 steps: done in 134 seconds, playing in the page with a working download link.

Step 8: Understand what each setting costs

• Steps: a diffusion model starts from random noise and removes a bit each step. My first test, 2 steps with 8 frames, took 10 seconds and was pure noise. Time grows roughly linearly with steps; 25 is a good start.
• Frames: going from 8 to 24 frames raised each step from 3.3 s to 7.9 s.
• Resolution: width × height × frames drives both memory and time. ModelScope was trained at 256 px, so going bigger often looks worse, not better.
• First click also loads the model, so it is slower.
• Fixed seed: the same settings give the same video, which is ideal for comparisons.

Prompt tips for small models

• One subject, one simple motion: “a red fox running through snow”.
• Add one camera or style word: cinematic, close-up, slow pan.
• Avoid text, hands and crowds; small models break on them.
• Keep the seed fixed and change one word at a time.

Hardware guide

• CPU only (like this machine): the small model works, 2 to 4 minutes for a 2 to 3 second clip. Wan 2.1 did not fit.
• NVIDIA with 8 GB: Wan 2.1 1.3B at 480p is the sweet spot. If you run out of VRAM, offload or keep the text encoder on the CPU.
• Bigger models such as Wan 14B want 24 GB or more.
• Apple Silicon: try the mps device; not tested here.

Troubleshooting

• HTTP 507 from /generate: not enough free RAM or VRAM for that model. Close other apps, choose the smaller model, or move to a GPU.
• Output is pure static: too few steps. Use 20 to 25.
• Machine freezes during load: your memory check threshold is too low for that model; raise need_gb.
• Wan frame errors: Wan expects frame counts like 17, 33 or 81 (4k+1).
• Torch downloads gigabytes of CUDA libraries on a CPU box: install from the CPU wheel index first.
• Video will not play in the browser: make sure imageio-ffmpeg is installed so H.264 MP4 is written.

Best practices and upgrades

• Run generation as a background job with a progress bar instead of one long HTTP request.
• Queue requests: one GPU can make one video at a time.
• Save the prompt and seed next to each video so you can recreate it.
• Always check the model licence before publishing output; ModelScope’s is non-commercial.
• Be honest about cost: software and models are free, the hardware and electricity are yours.

FAQ

Is this really free?

The code, the models and the runtime are free, with no API bill or credits. You pay for your own hardware and power.

Can I use the videos commercially?

Wan 2.1 output under Apache-2.0, yes, subject to its terms. ModelScope T2V is CC-BY-NC-4.0, so non-commercial only, and it may show a watermark.

Why bfloat16 on CPU?

It halves memory compared with float32 while keeping a wide numeric range, which helps large models fit in RAM.

How long will a GPU take?

The official Wan 2.1 figure is about 4 minutes for 5 seconds of 480p on an RTX 4090.

Key takeaways

• About 110 lines of Flask, diffusers and HTML give you a prompt-to-MP4 app.
• A memory check before loading turns a machine freeze into a clean 507 error.
• On CPU, ModelScope produced a real clip in 203 seconds; Wan 2.1 did not fit.
• Steps, frames and resolution trade quality for time; start at 25 steps and 256 px.
• With an 8 GB NVIDIA GPU, switch to the Apache-licensed Wan 2.1.