Skip to main content

System overview

Bucketeer is a scalable and efficient feature flag and experimentation platform designed to enhance developer productivity by enabling real-time feature management.

Bucketeer overview architecture​

Bucketeer architecture

Client and Server SDKs​

The Bucketeer SDKs, available for various platforms (Go, Node.js, iOS, Android, JavaScript, and Flutter), serve as the primary interface for applications and servers interacting with the Bucketeer backend. These SDKs retrieve user evaluation and feature flag data while incorporating a caching layer to minimize latency and improve performance. Client SDKs only support remote evaluations. The Server SDKs support both remote and local evaluations, which help to reduce latency when evaluating the user locally.

Gateway Communication​

SDKs and the Admin Console communicate with the Bucketeer backend using HTTPS protocol via the Envoy API Gateway and Envoy Web Gateway. Envoy allows the processing of gRPC and REST requests using the same backend core.

  • Protocol Details: Envoy is used as a sidecar container. Requests routed from the Envoy gateway are transcoded to gRPC before reaching the core Bucketeer gRPC API server for faster and more efficient data exchange. Envoy is also used for fault tolerance as we apply a circuit breaker.

Core Bucketeer Backend​

The Bucketeer API server handles the main business logic and provides interfaces to control other components. It implements various backend tasks, including managing feature flag data, performing user evaluations, user targeting, experimentation, auto operations, and many others.

Batch Servers and Stan Model​

Batch servers run essential jobs that are triggered via APIs for caching, sending notifications, auto operations, calculating experiment data using the Stan model, and data retention.

Event Publishing and Processing​

Bucketeer uses a message queue for event streaming. The subscriber service pulls evaluation and goal events from the SDK client and audit logs to persist the data into the storage layer.

Two backends are supported, selected with the pubsub-type option on each service:

  • Google Pub/Sub (default): recommended for production deployments on GCP.
  • Redis Streams: lets you run Bucketeer without a cloud message queue. The Docker Compose environment uses this backend.

Storage and Cache​

Bucketeer implements a storage layer using these stacks:

  • Operational database: stores feature flags, experiments, goals, notifications, and many other data with related configurations. Both MySQL and PostgreSQL are supported, selected with the storage-type option on each service (mysql is the default).
  • Redis: high-performance cache to support low-latency reading.
  • Data warehouse: stores and analyzes user evaluation and goal events, which feed the experiment results and the Insights page. BigQuery, MySQL, and PostgreSQL are supported, selected with the type field of the data warehouse configuration.
Choosing a storage stack

The operational database and the data warehouse are configured independently, so you can, for example, keep flag configuration in MySQL while sending events to BigQuery. Using the same engine for both is also supported, which is the simplest option for self-hosted deployments that do not use a cloud data warehouse.