Robust Video Matting

Robust Video Matting is a free project for separating people from video backgrounds. It processes frames with a recurrent neural network that carries temporal memory, and the project says it can perform matting in real time without extra inputs. A browser webcam demo runs the model live and can display recurrent states. Its conversion API accepts a video file or image sequence and can produce a composited video or PNG sequence, with optional alpha and foreground outputs. Models are provided for PyTorch, TorchHub, TorchScript, ONNX, TensorFlow, TensorFlow.js, and CoreML. MobileNetv3 is recommended for most uses; ResNet50 is larger and offers small performance improvements. The project reports GPU performance figures, but cautions that file conversion is much slower than tensor throughput because the script lacks hardware video encoding and decoding and parallel tensor transfers. CoreML models require iOS 13 or later and do not support dynamic resolution. The code is released under GPL-3.0.

Robust Video Matting offers model formats and conversion options for separating people from video backgrounds. Check the conversion-speed caveat and CoreML limits when choosing a workflow.

It suits developers and video workflows that need human matting, background replacement, or model exports across supported frameworks. The repository also provides a Colab demo for trying videos with a free GPU.

Robust Video Matting is a free, open-source human-matting project for separating people from video. It suits developers and technically confident creators who can work with model files or conversion tools. Its appeal is real-time matting without extra inputs; the file-conversion script is a less convincing choice when speed is the priority.

Overview

A recurrent neural network with temporal memory processes frames to isolate people, supporting background replacement at resolutions up to 4K. Rather than a complete video-editing app, this is a model and toolkit for integrating matting into a workflow.

The code and pretrained models are released under GPL-3.0, a practical advantage for free experimentation but a license teams should consider before incorporating the project into distributed software. Developed at ByteDance Inc., the project also links usage, training and evaluation documentation.

Key features

• Matting without extra inputs: The project describes real-time processing without additional inputs, which is useful for live or continuous footage where supplying guidance for frames would interrupt a workflow.
• Two model variants: MobileNetv3 is recommended for most use cases. ResNet50 is larger and brings small performance improvements, so it is a specialized option rather than an obvious upgrade for every user.
• Broad model formats: Implementations or exports cover PyTorch, TorchHub, TorchScript, ONNX, TensorFlow, TensorFlow.js and CoreML. This gives developers several integration routes; ONNX models use opset 12 and are tested with CPU and CUDA backends.
• Live and file-based evaluation: A browser webcam demo runs the model live and can visualize recurrent states. A Colab demo lets users test their own videos with a free GPU, a useful way to evaluate the model without first setting up local GPU resources.
• Conversion outputs: The conversion API accepts video files or image sequences and can produce a composited video or PNG sequence, with optional alpha and foreground outputs. Those options suit custom pipelines, though the script is expected to run much slower than the reported model throughput because it lacks hardware video encoding and decoding and parallel tensor transfers.
• Mobile considerations: CoreML models require iOS 13 or later, and CoreML does not support dynamic resolution. The README also links a third-party NCNN C++ Android project, so Android use involves a separate implementation rather than one of the listed model exports.

Performance figures are hardware-specific: the README reports 172 FPS at HD and 154 FPS at 4K on an RTX 3090 with FP16, batch size 1 and frame chunk 1; it also reports 104 FPS at HD and 76 FPS at 4K on an Nvidia GTX 1080 Ti. These figures describe model performance, not the slower end-to-end video conversion script.

Pricing

Robust Video Matting — free. The free GPL-3.0 plan includes pretrained models and source code. There are no paid tiers in the plan structure, making it a fit for users who can supply their own workflow and technical setup. The trade-off is that this is not a packaged editing service with a separate convenience tier.

Platforms

The project lists Android, API, iOS, Linux, self-hosted and web support. CoreML has the iOS 13 minimum and fixed-resolution limitation; Android support is linked through a third-party NCNN C++ project. The browser demo and framework exports provide additional ways to evaluate or integrate the model.

Who it's for

Choose it if you need human-focused video matting, want to integrate a model into software, or value access to source code and multiple framework formats. It is less suitable if you expect a turnkey editor or need the conversion script itself to deliver the reported tensor speeds. Teams should also make sure GPL-3.0 fits their intended distribution.

Pros and cons

• Pro: Real-time matting without additional inputs supports continuous video workflows.
• Pro: Multiple model formats, plus video and image-sequence conversion outputs, give developers flexibility in how they integrate and use the project.
• Pro: Free source code, pretrained models and a Colab demo lower the barrier to evaluation.
• Con: The video-conversion script runs much slower than model throughput because it lacks hardware video encoding and decoding and parallel tensor transfers.
• Con: CoreML cannot handle dynamic resolution, and its models require iOS 13 or later.
• Con: GPL-3.0 may not suit every team’s software distribution requirements.

Alternatives

For a broader set of options, browse Video Background Removal Software or AI Video Background Removers.

• Cutout.Pro is a freemium option spanning mobile, desktop and web platforms, with a $5 a month subscription billed monthly for 80 credits; choose it if a credit-based subscription better suits your workflow.
• Pixelcut offers freemium background-removal tools across mobile and web; choose it if you want that broader platform mix.
• Knockout API has a free plan with 30 images per month and a metered pay-as-you-go option; it may suit workflows centered on API-based image processing rather than video matting.
• BackgroundRemover is a free, open-source project that can be installed locally with pip or from source; consider it for local background-removal work.
• Media.io AI Object Remover is a web-only paid option with a three-day trial and 1080p export; consider it if those terms and resolution suit your needs.
• VCam offers a free Starter plan for Windows and macOS with up to 500MB storage and a watermark; it is an option for desktop users comfortable with those limits.
• VideoBGNinja is a web option with a free email pass that renews monthly and allows four minutes of video; choose it if that allowance meets your needs.
• BGRemove.video offers three successful removals per 24-hour window for videos up to 60 seconds and 2 GB, with transparent WebM output and no watermark; consider it for occasional short clips.

Verdict

Robust Video Matting is best for developers and technical creators who want a free, adaptable human-matting model with real-time processing and multiple integration formats. Choose it for control over the model and workflow; look elsewhere if you need a polished editing app or expect fast end-to-end file conversion without building around its limitations.