- Home
- Data Lineage Tools
- Spline
Spline
What it is
Spline is a free, open-source tool for tracking data lineage and pipeline structure. Agents collect metadata from Apache Spark and Python 3.9-or-newer functions, with a proof-of-concept adapter for OpenLineage metadata. They send pipeline information to the server through HTTP REST or Kafka; the Consumer API then makes lineage data available to the web UI and third-party applications. Spline represents lineage as graphs for viewing. The server stores data in ArangoDB and requires version 3.11 or newer. Deployment components come as Docker containers and JAR or WAR artifacts, and the server can also run in a standard Java environment. The project identifies regulatory requirements, documenting business logic, and finding performance bottlenecks as use cases. By default, the Spark agent tracks persistent writes, such as output to files, Hive tables, or databases. Kafka transport is optional. SSL certificate validation is enabled by default, and the documentation advises against disabling it in production. The project is hosted by AbsaOSS and lists an Apache-2.0 license; its free plan is listed as free.
For and against
For
- Tracks Spark and Python pipeline metadata
- Displays lineage as graphs
- Offers HTTP REST or Kafka transport
- Apache-2.0 licensed and free
Against
- Requires ArangoDB 3.11 or newer
- Spark tracks persistent writes by default
- Kafka gateway is optional
The verdict
Spline provides a self-hosted route to visualizing pipeline lineage, with Spark and Python agent support. Check its ArangoDB requirement and default Spark tracking scope against your needs.
Who it suits
Spline may suit teams that need to document data flows from Spark or Python pipelines and view lineage in a web UI. It is aimed at users able to operate its self-hosted components and ArangoDB.
Plans and prices
Spline
Free and open-source tool; self-hosted components
Read 10 Oct 2026 · absaoss.github.io
On the record
- Deployment
- Components are available as Docker containers and JAR or WAR artifacts, and the server can run in a standard Java environment.absaoss.github.io · read 10 Oct 2026
- Framework support
- The project provides agents for Apache Spark and arbitrary functions written in Python 3.9 or newer, plus a proof-of-concept adapter for OpenLineage metadata.absaoss.github.io · read 10 Oct 2026
- License
- The GitHub repository lists the project under the Apache-2.0 license.github.com · read 10 Oct 2026
- Lineage flow
- Agents send pipeline metadata to the server through its Producer API using HTTP REST or Kafka, and the Consumer API makes lineage data available to the UI and third-party applications.absaoss.github.io · read 10 Oct 2026
- Maker
- The project is hosted in the AbsaOSS GitHub organization, described as ABSA Open Source and located in Johannesburg, South Africa.github.com · read 10 Oct 2026
- Purpose
- Spline is a free, open-source tool for automatically tracking data lineage and data pipeline structure.absaoss.github.io · read 10 Oct 2026
- Security configuration
- SSL certificate validation is enabled by default, and the documentation says not to disable it in production.absaoss.github.io · read 10 Oct 2026
- Spark tracking limit
- By default, the Spark agent tracks only persistent writes, such as writes to files, Hive tables, or databases.absaoss.github.io · read 10 Oct 2026
- Storage
- Spline Server stores lineage data in ArangoDB, with ArangoDB version 3.11 or newer required.absaoss.github.io · read 10 Oct 2026
- Transport integration
- Agents can send lineage metadata over HTTP REST or Kafka; the Kafka gateway is optional.absaoss.github.io · read 10 Oct 2026
- Use cases
- The project identifies regulatory requirements, documenting business logic, and finding performance bottlenecks as target problems.absaoss.github.io · read 10 Oct 2026
- Visualization
- Spline processes lineage data into graph form and provides a web UI for viewing it.absaoss.github.io · read 10 Oct 2026
