git-annex

git-annex manages large files alongside Git while keeping the file contents outside Git itself. Git tracks filenames and metadata; git-annex stores annexed contents in a distributed store across repository clones. It can synchronize, archive, and back up data online or offline, and can resume interrupted transfers. The project describes it as designed for Git users who are comfortable with the command line.

Its assistant can create a synchronized folder across computers, Android devices, removable drives, NAS appliances, and cloud services. Built-in special remotes include S3, Glacier, rclone, rsync, web, WebDAV, and Android adb. Encrypted special remotes can encrypt file contents and use HMAC hashing for filenames, although the project says encryption is mostly not used by default. Install instructions cover Linux distributions, macOS, Android, and Windows. The project says file size is limited by the filesystem, and repositories with hundreds of thousands of files are supported; larger repositories may slow down because of Git limitations. It explicitly says git-annex is not a backup system, though it can be part of an archival workflow or deliver files to another backup system. The free software project provides a user forum and contact page.

git-annex separates file contents from Git's filenames and metadata while supporting synchronization across varied storage locations. It can be part of an archive or backup workflow, but users should not treat it as a backup system on its own.

git-annex suits Git users comfortable with the command line who need to coordinate large files across repositories and storage locations. It may also fit archival workflows, but the project says it is not itself a backup system.

git-annex manages large files alongside Git while keeping their contents outside Git itself. It best suits command-line-oriented Git users coordinating data across computers and varied storage. Its storage flexibility is a strength, but it needs a separate backup plan.

Overview

Git tracks filenames and metadata; git-annex keeps file contents in a distributed store across repository clones. That separation lets Git coordinate files without carrying their contents, a useful fit for archives or data collections that move among different storage locations.

The trade-off is operational responsibility. Users choose and manage the storage workflow, and git-annex explicitly says it is not a backup system on its own. It can support an archival system or deliver files to a separate backup system.

Key features

The assistant creates a synchronized folder across computers, Android devices, removable drives, NAS appliances and cloud services. Built-in special remotes include S3, Glacier, rclone, rsync, web, WebDAV and Android adb. This range is valuable for users who want control over where data lives; it also favors people comfortable configuring a Git-based workflow over those seeking a ready-made cloud drive.

Transfers can resume after interruption, which helps with large files and unreliable connections. File size is limited by the filesystem. Repositories with hundreds of thousands of files are supported, though larger repositories may slow down because of Git limitations.

Encrypted special remotes can encrypt file contents and use HMAC hashing for filenames. Encryption is mostly not used by default, so users handling sensitive data need to configure an encrypted remote deliberately.

The project maintains a security page with known vulnerabilities and fixes, and directs private vulnerability reports through its contact page. DataLad also uses git-annex to provide access to scientific data from varied sources.

Pricing

git-annex is free and open source. Its plan costs nothing and manages large files without storing file contents in Git. There is no free trial; no paid tier is needed for the stated functionality.

Platforms

Platforms include Android, Linux, macOS, self-hosted deployments, web and Windows. Official installation instructions cover Linux distributions, macOS, Android and Windows, with package and other installation methods.

Who it's for

git-annex is for Git users who are comfortable with the command line and need to coordinate large files across devices or storage services. It is a strong fit for distributed archives and flexible data workflows, especially when users want to choose storage backends. Those seeking a managed backup product or a simple cloud file service should look elsewhere.

Pros and cons

• Pros: Keeps file contents outside Git while retaining filenames and metadata, which suits repositories that need to track large data collections.
• Pros: Supports a broad mix of storage remotes and can resume interrupted transfers, giving users options across online and offline workflows.
• Pros: Free, open source and available across major desktop platforms and Android.
• Cons: It is not a backup system by itself, so users must arrange a separate backup workflow.
• Cons: Encryption is mostly not enabled by default, requiring deliberate setup for sensitive files.
• Cons: Very large repositories may slow down due to Git limitations.

Alternatives

For model development and experiment tracking, Weights & Biases offers a free plan with five model seats, 5 GB per month of storage and 1 GB per month of Weave data ingestion. Roboflow may suit computer vision workflows, with a free tier that includes 10 monthly credits, enough for about 30 model trainings or 80,000 inferences.

Apache Iceberg, Apache Paimon and Delta Lake are free options for readers choosing a data lake or storage framework. Databricks Notebooks offers a free edition with one serverless workspace and limited compute size and usage. Dolt is a free, open-source program of about 100 MB, while its DoltHub Pro plan costs $5 a month, billed monthly, for 100 MB of private data. Graviti has a free Starter plan with 100 GB of private cloud storage, 50,000 rows and XS compute for two hours.

Browse more options in Data Version Control Tools.

Verdict

Choose git-annex if you use Git at the command line and want to coordinate large files across storage you control. Its distributed storage model and broad remote support are compelling, and the software is free. Look elsewhere if you need backups handled for you or expect encryption to be enabled without configuration.