What is Lance?

Lance, from ByteDance's Intelligent Creation Lab, is a native unified multimodal model that handles understanding, generation, and editing of both stills and clips within a single 3B-active-parameter network. Its inference code exposes seven tasks: text-to-image, text-to-video, image-to-video, image editing, video editing, and image or video understanding for captions and visual Q&A, while the galleries demonstrate chained multi-round edits and reasoning clips like a solved maze. T2V and I2V runs can first send the prompt to any OpenAI-compatible endpoint for rewriting, and one local Gradio front end (port 7860) covers every task. Self-hosting means cloning the repo, building a CUDA 12.4+ Python environment, downloading the video-capable or image-only checkpoint from Hugging Face into the downloads folder, then calling the supplied shell script; vLLM-Omni documents a serving recipe, and released scripts fine-tune on local parquet data. The maintainers call it a research artifact, not a polished product: training stayed under a 128 A100 GPU budget, topping out at 768x768 stills and 480p 12 fps footage, so output quality fluctuates and no container images or tagged releases exist.

Should you self-host Lance?

Lance scores 49/100 on selfhost.directory (how scoring works) and ranks #53 of 125 in AI. It has 1 361 GitHub stars.

There was no stable release in the past 12 months, so check the repository's recent activity before you rely on it.

Setup is advanced: plan for several services and some ongoing maintenance. It is Apache-2.0-licensed and written mostly in Python.

Its closest alternative on selfhost.directory is ComfyUI (93/100, 143 714 stars). It scores higher overall, so compare the two before you pick. See Lance vs ComfyUI side by side.

Features

Fine Tuning
Gradio web UI
Image Editing
Image-to-video generation
Image understanding
Multi-turn consistent editing
Prompt EnhancementRequires an external API key
Text-to-image generation
Text-to-video generation
Video Editing
Video reasoning generation
Video understanding

How to configure Lance

ARNOLD_WORKER_GPUNumber of GPUs the training scripts use on the machine.
Number of GPUs the training scripts use on the machine.
CFG_TEXT_SCALE4.0Classifier-Free Guidance scale for text conditioning.
Classifier-Free Guidance scale for text conditioning.
CONFIG_PATHOptional path to a custom validation JSON/JSONL file instead of the task default example config.
Optional path to a custom validation JSON/JSONL file instead of the task default example config.
DATASET_CONFIG_FILEDataset config used by a training launch.
Dataset config used by a training launch.
ENHANCE_PROMPTfalseOptional prompt-rewrite switch for T2V/I2V; needs the openai package and the LANCE_REWRITE_* variables.
Optional prompt-rewrite switch for T2V/I2V; needs the openai package and the LANCE_REWRITE_* variables.
values: true | false
LANCE_REWRITE_API_KEYAPI key for the prompt-rewrite service.
API key for the prompt-rewrite service.
LANCE_REWRITE_BASE_URLBase URL of the OpenAI-compatible rewrite endpoint.
Base URL of the OpenAI-compatible rewrite endpoint.
LANCE_REWRITE_MODEL_NAMEModel name for the prompt-rewrite service.
Model name for the prompt-rewrite service.
MODEL_PATHdownloads/Lance_3BPath to the downloaded Lance model weights (Lance_3B or Lance_3B_Video).
Path to the downloaded Lance model weights (Lance_3B or Lance_3B_Video).
values: downloads/Lance_3B | downloads/Lance_3B_Video
NUM_FRAMES50Number of frames for video generation (max 121); unused for image tasks.
Number of frames for video generation (max 121); unused for image tasks.
NUM_GPUS1Number of GPUs to use for inference.
Number of GPUs to use for inference.
NUM_WORKERSDataloader worker count, lower it for smaller machines.
Dataloader worker count, lower it for smaller machines.
RESOLUTIONvideo_480pBase resolution preset.
Base resolution preset.
values: image_768res | video_480p
SAVE_PATH_GENCLI flag giving the output directory for generated results.
CLI flag giving the output directory for generated results.
--server-nameHost the local Gradio app binds to.
Host the local Gradio app binds to.
--server-portPort the local Gradio app listens on.
Port the local Gradio app listens on.
TASK_NAMECLI flag selecting the inference task.
CLI flag selecting the inference task.
values: t2v | t2i | i2v | image_edit | video_edit | x2t_video | x2t_image
VAL_DATASET_CONFIG_FILEValidation dataset config for training.
Validation dataset config for training.
VALIDATION_DATA_SEED42Random seed for generation reproducibility.
Random seed for generation reproducibility.
VALIDATION_NUM_TIMESTEPS30Number of denoising steps.
Number of denoising steps.
VALIDATION_TIMESTEP_SHIFT3.5Timestep shift parameter for flow matching scheduling.
Timestep shift parameter for flow matching scheduling.
VIDEO_HEIGHT768Spatial height for generation; unused for editing tasks.
Spatial height for generation; unused for editing tasks.
VIDEO_WIDTH768Spatial width for generation; unused for editing tasks.
Spatial width for generation; unused for editing tasks.
WANDB_NAMEExperiment name passed to the training launch.
Experiment name passed to the training launch.