Jocko

Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)

From travisjeffery·Updated June 14, 2026·View on GitHub·

- Implement Kafka in Go - Protocol compatible with Kafka so Kafka clients and services work with Jocko - Make operating simpler - Distribute a single binary - Use Serf for discovery, Raft for consensus (and remove the need to run ZooKeeper) - Smarter configuration settings - Able to use percentages of disk space for retention policies rather than only bytes and time kept - Handling size configs when you change the number of partitions or add topics - Learn a lot and have fun The project is written primarily in Go, distributed under the MIT License license, first published in 2016. It has gained significant community traction with 5,010 stars and 372 forks on GitHub. Key topics include: distributed-systems, go, kafka, messaging, queue.

Latest release: 0.0.1

November 15, 2017View Changelog →

Jocko

Kafka/distributed commit log service in Go.

Goals of this project:

Implement Kafka in Go
Protocol compatible with Kafka so Kafka clients and services work with Jocko
Make operating simpler
Distribute a single binary
Use Serf for discovery, Raft for consensus (and remove the need to run ZooKeeper)
Smarter configuration settings
- Able to use percentages of disk space for retention policies rather than only bytes and time kept
- Handling size configs when you change the number of partitions or add topics
Learn a lot and have fun

TODO

Hiatus Writing Book

I’m writing a book for PragProg called Building Distributed Services with Go. You can sign up on this mailing list and get updated when the book’s available. It walks you through building a distributed commit log from scratch. I hope it will help Jocko contributors and people who want to work on distributed services.

Reading

Project Layout

├── broker        broker subsystem
├── cmd           commands
│   └── jocko     command to run a Jocko broker and manage topics
├── commitlog     low-level commit log implementation
├── examples      examples running/using Jocko
│   ├── cluster   example booting up a 3-broker Jocko cluster
│   └── sarama    example producing/consuming with Sarama
├── protocol      golang implementation of Kafka's protocol
├── prometheus    wrapper around Prometheus' client lib to handle metrics
├── server        API subsystem
└── testutil      test utils
    └── mock      mocks of the various subsystems

Building

Local

Clone Jocko

$ go get github.com/travisjeffery/jocko

Build Jocko
```
$ cd $GOPATH/src/github.com/travisjeffery/jocko
$ make build
```
(If you see an error about dep not being found, ensure that
$GOPATH/bin is in your PATH)