Distributed Systems
Apache Iggy: a persistent message stream in Rust
apache/iggy on GitHub
By Mira Vance
Published 22 October 2026 · Last reviewed 22 October 2026
Apache Iggy's 0.9.0 release put Viewstamped Replication into the same iggy-server binary that runs a single node, and one configuration key, cluster.enabled, decides which one you get.
Drafted with AI assistance, researched, verified and edited by the HastCode editorial team. We publish only as many pieces as the Author can verify this way.
TL;DR
• Apache Iggy is a persistent, append-only message streaming log written in Rust: streams group topics, topics split into partitions, and consumer groups read them, over QUIC, WebSocket, TCP or HTTP.
• It is built from the ground up, not on top of Kafka or a SQL database, with a thread-per-core, shared-nothing design on io_uring.
• Since 0.9.0, the standard server can run as a replicated cluster; single node stays the default.
• Adopt it for new Linux workloads that want Kafka's shape in one binary. Wait if you need stable upgrades: until 1.0.0, an upgrade means a fresh data directory.
In 60 seconds
These two commands from the README's Docker section start a server reachable from the host with root credentials iggy/iggy, then list streams with the CLI inside the container. The README says the SYS_NICE capability, the seccomp setting and unlimited locked memory are required.
Architecture
Walk the diagram left to right. A client connects over TCP, QUIC, WebSocket or HTTP; shard 0 accepts the connection and the connection's owning shard decodes the request. A shard has its own single-threaded compio runtime, and requests move between shards by message passing rather than shared locks.
The router sends each partition request to the shard that owns that partition, chosen by hashing its IggyNamespace; every shard, shard 0 included, owns partitions. Metadata (streams, topics, users, consumer groups) takes the other branch: it commits only on shard 0 and is written to metadata/journal.wal, the VSR write-ahead log (architecture docs).
On the write path, the partition's primary admits the write and replicates it through Viewstamped Replication: it numbers the operation, sends Prepare and commits once a quorum answers PrepareOk. The data is flushed with vectored I/O to the partition's current segment, a .log file paired with an .index file that tracks offsets and timestamps. Consumer offsets live on the server, in each partition's offsets/ directory.
Two policies decide what an acknowledgement means: durability for messages and consumer_offset_durability for offsets. Both default to replicated, a quorum commit that does not wait for stable storage; persisted also requires stable-storage copies on the quorum (README).
Signals
Use it if
• You want Kafka's model (streams, topics, partitions, consumer groups, server-side offsets) as a single binary with no external broker or database.
• You run Linux and care about tail latency: the README names sub-millisecond tail latency as a primary goal, and the project publishes runs on its benchmarking platform for you to compare against your own hardware.
• Your services are polyglot: there are SDKs for Rust, C#, Java, Python, Node.js and Go.
Don't use it if
• Existing Kafka clients and connectors must work unchanged. The Kafka gateway is in development and not yet in a release; use Kafka or a Kafka-API-compatible broker.
• You need in-place upgrades between versions. Until 1.0.0, an upgrade means starting from a fresh data directory; use a broker with a stable on-disk format.
• You rely on broker-side compression. Server-side message compression is not supported yet; use Kafka, or compress in your producers.
Alternatives
• Apache Kafka: the reference partitioned log, on the JVM, with the largest client and connector ecosystem; choose it when ecosystem beats footprint.
• Redpanda: a Kafka-API-compatible broker in C++, so existing Kafka clients work today, which Iggy's unreleased gateway cannot yet promise.
• NATS JetStream: the persistence layer of the NATS server, a fit when you want messaging and streams from one lightweight system rather than a dedicated log.
• Apache Pulsar: separates serving brokers from storage, a heavier deployment that Iggy's single binary avoids.
Changelog
• server-0.9.0 18 September 2026
• server-0.8.0 22 April 2026
• server-0.7.0 25 February 2026
• server-0.6.0 9 December 2025
• server-0.5.0 10 August 2025
• server-0.4.300 17 April 2025
• server-0.4.214 22 February 2025
• server-0.4.213 21 February 2025
• server-0.4.212 21 February 2025
• server-0.4.211 21 February 2025
FAQ
Is Apache Iggy production ready?
Apache Iggy has not reached 1.0: its own clustering docs say the cluster protocol and configuration can still change between pre-release versions. Until 1.0.0, an upgrade means starting from a fresh data directory, so plan for that before storing data you must keep.
Does Apache Iggy support clustering and replication?
Yes. Since 0.9.0, Apache Iggy ships Viewstamped Replication in the standard iggy-server binary; clustering is off by default and enabled with cluster.enabled. The docs recommend at least three replicas, which tolerate one unavailable replica.
Can Kafka clients connect to Apache Iggy?
Not yet. Apache Iggy's Kafka gateway, a Kafka wire-protocol listener, is in development and not part of a release; the 0.9.0 release post calls it a foundation layer, not a working bridge.
Which languages have Apache Iggy SDKs?
Apache Iggy has SDKs for Rust, C#, Java, Python, Node.js (TypeScript) and Go, with C++ and PHP in progress.
Does Apache Iggy compress messages?
Not on the server yet. Apache Iggy's README says topic compression values are reserved for future support, and points to message headers for manual compression today.
What are the alternatives to Apache Iggy?
The main alternatives to Apache Iggy are Apache Kafka (the largest ecosystem), Redpanda (Kafka-API-compatible today), NATS JetStream (messaging plus persistence in one server) and Apache Pulsar (brokers separated from storage).
How is Apache Iggy licensed?
Apache Iggy is released under the Apache License 2.0 and developed in the Apache Software Foundation's apache GitHub organisation.
Verdict
Apache Iggy is a serious, carefully documented take on the partitioned log: a thread-per-core Rust server that now replicates with Viewstamped Replication in the same binary that runs a single node. We recommend it for new, Linux-hosted workloads where you control the clients and want a small operational footprint, and for teams curious enough to benchmark it against their own hardware. We would not yet move a Kafka estate onto it: there is no released Kafka gateway, and before 1.0 every upgrade means a fresh data directory. A 1.0 release with a stable on-disk format, or a shipped Kafka gateway, would change that answer.
Sources
• Apache Iggy architecture documentation
• Apache Iggy clustering documentation: Viewstamped Replication
• Apache Iggy 0.9.0 release blog post
• https://github.com/apache/iggy
• https://github.com/apache/iggy/releases
• https://github.com/apache/iggy/graphs/contributors
• https://github.com/apache/iggy/tree/master/.github
• https://github.com/apache/iggy/blob/master/README.md
Get new Explainers by email
How real systems are built, in your inbox. No spam; unsubscribe any time.
More in Distributed Systems
1. ConceptMetastable failures: why systems stay down after the trigger is goneA metastable failure outlives its trigger because a loop such as retries keeps the system overloaded. How the loop works and what breaks it.26 October 2026
2. Repo Profileetcd: a Raft-replicated key-value store for coordination dataetcd is a Raft-replicated, linearizable key-value store for coordination-sized data; fast disks and a few-gigabyte ceiling are the price.22 October 2026
Related
1. Repo ProfileAgentgateway: a Rust gateway for HTTP, gRPC, MCP and LLM trafficAgentgateway is a Rust proxy that routes HTTP, gRPC, MCP, A2A and LLM traffic under one Gateway API control plane; minor releases still break config.22 October 2026
2. Repo ProfileOpenShell: NVIDIA's sandboxed runtime for autonomous AI agentsOpenShell is NVIDIA's self-hosted sandbox runtime for AI agents: kernel-enforced policy and a credential proxy, at the cost of a fast-moving 0.1 API.22 October 2026
3. Repo Profilepgrust: a Postgres rewrite in Rust, built with AI agentspgrust is a wire-compatible rewrite of Postgres in Rust, written largely by AI agents; fast on its own benchmarks, AGPL-3.0, not yet for production.22 October 2026