Arroyo
What it is
Arroyo is a free, open-source distributed stream-processing engine for stateful computations on data streams, with sub-second results. Users define pipelines in analytical SQL, including complex queries, windows, and joins. The engine supports checkpointing for fault tolerance and recovery, as well as exactly-once processing, and says it can scale to millions of events per second. Listed connectors include Kafka, Confluent Cloud, Kinesis, MQTT, NATS, MySQL, Postgres, Redis, Redpanda, Delta Lake, Iceberg, HTTP, WebSocket, and Webhooks. It reads and writes JSON, Avro, Parquet, text, and binary, and includes more than 300 SQL functions. Users can extend SQL with Rust scalar, aggregate, and asynchronous functions. The web UI supports connection management, query development and testing, and pipeline monitoring; pipelines can also be managed through a REST API. A single-node cluster is intended for testing and development. Production deployments use a distributed cluster with Arroyo’s scheduler or Kubernetes. Checkpoint and artifact storage can use S3-compatible stores, R2, GCS, Azure Blob Storage, or local filesystems.
For and against
For
- Supports exactly-once processing
- Offers more than 300 SQL functions
- Includes connectors for databases and messaging systems
- Web UI and REST API manage pipelines
Against
- Single-node cluster is intended for testing and development
- Production deployment uses a distributed cluster
The verdict
Arroyo combines SQL-defined stream processing with checkpointing, connectors, and a web UI or REST API for pipeline management. Its deployment guidance distinguishes single-node testing from distributed production use.
Who it suits
Arroyo suits teams and developers building stateful data-stream pipelines with SQL, and those who need Rust user-defined functions. Its single-node cluster is intended for testing and development; production deployments use a distributed cluster.
Plans and prices
Arroyo open-source
Apache 2.0 engine · self-hosted on Linux/macOS, Docker or Kubernetes
Read 15 Oct 2026 · doc.arroyo.dev
Cloudflare Workers Paid with Pipelines
50 GB/month SQL transforms · 50 GB/month sink delivery · unlimited stream ingress · $0.04/GB additional SQL
Read 15 Oct 2026 · blog.cloudflare.com
Open-source Arroyo Engine
Apache 2.0 licensed; single binary; self-hosted
Read 4 Oct 2026 · arroyo.dev
On the record
- Acquisition
- Cloudflare announced on April 10, 2025 that it acquired Arroyo and would integrate its engine into Cloudflare Pipelines, Queues, Workers and R2.blog.cloudflare.com · read 15 Oct 2026
- Async UDF limits
- Async Rust UDFs have configurable concurrency and timeout settings; Python UDFs do not support async functions.doc.arroyo.dev · read 15 Oct 2026
- Checkpoint storage
- Production deployments can store checkpoints in S3, Google Cloud Storage, Azure Blob Storage or compatible stores such as Minio, R2 and Backblaze B2.doc.arroyo.dev · read 15 Oct 2026
- Cloudflare managed limits
- Cloudflare's current Pipelines pricing page lists unlimited stream ingress, 50 GB/month included SQL transforms and 50 GB/month included sink delivery on Workers Paid.developers.cloudflare.com · read 15 Oct 2026
- Company history
- Arroyo’s founders say they started the company in 2022 and open-sourced the engine in 2023.arroyo.dev · read 4 Oct 2026
- Company legal name
- Crunchbase identifies the company legal name as Arroyo Systems, Inc.crunchbase.com · read 15 Oct 2026
- Company location and investors
- Crunchbase lists Berkeley, California as headquarters and Y Combinator as the investor in its pre-seed round.crunchbase.com · read 15 Oct 2026
- Connectors
- Documented connectors include Kafka, Confluent Cloud, Kinesis, MySQL, Postgres, MQTT, NATS, Redis, Redpanda, Fluvio, Filesystem, Iceberg, WebSocket, SSE, Webhook and HTTP polling.doc.arroyo.dev · read 15 Oct 2026
- Data formats
- Native formats include JSON, Avro, Parquet, raw text and raw binary, with custom formats possible through UDFs.arroyo.dev · read 15 Oct 2026
- Deployment
- Arroyo can run as a single binary, Docker container or Kubernetes deployment, with SQLite for local use and Postgres recommended for distributed production.doc.arroyo.dev · read 15 Oct 2026
- Exactly-once processing
- Arroyo advertises exactly-once processing across out-of-order data and machine failures, using checkpointed state.arroyo.dev · read 15 Oct 2026
- Fault tolerance
- The engine supports state checkpointing for fault tolerance and pipeline recovery, and exactly-once processing.doc.arroyo.dev · read 4 Oct 2026
- Formats and functions
- Arroyo natively reads and writes JSON, Avro, Parquet, text, and binary, and includes over 300 SQL functions.arroyo.dev · read 4 Oct 2026
- Founder and launch
- Y Combinator lists Arroyo as a Winter 2023 company founded in 2022 with founders Micah Wylde and Jackson Newhouse.ycombinator.com · read 15 Oct 2026
- GitHub project
- The public repository describes Arroyo as a Rust distributed stream-processing engine and showed about 5.1k stars and 379 forks when opened.github.com · read 15 Oct 2026
- Helm packaging
- Artifact Hub lists an official Arroyo Helm chart for Kubernetes and showed chart/application version 0.15.0.artifacthub.io · read 15 Oct 2026
- Joins
- Arroyo SQL supports stream joins including inner, left, right and full joins, plus windowed and updating-data joins.arroyo.dev · read 15 Oct 2026
- Kafka integration
- Kafka sources and sinks support self-hosted Kafka, Amazon MSK and Confluent Cloud, with exactly-once or at-least-once sink modes.doc.arroyo.dev · read 15 Oct 2026
