repository·main·Indexed 19 days ago
https://github.com/bucketeer-io/bucketeerAn enterprise-grade, open-source feature management and experimentation platform. Bucketeer enables developers to manage feature flags, perform Bayesian A/B testing, and automate progressive rollouts with high scalability and self-hosting flexibility. The platform includes a web dashboard for administration and an API gateway for SDK and client integration.
{environment-id}:goal_event_retry:*. This is primarily used for cleaning up end-to-end (e2e) test data from Redis.Bucketeer uses a Stan model (pkg/experimentcalculator/stan/experiment.stan) to estimate conversion rates using a Binomial likelihood with an implicit Beta(1, 1) prior.
Model Structure:
g), conversions per variation (x), and users per variation (n).p for each variation, bounded between 0 and 1.x[i] ~ binomial(n[i], p[i]).prob_upper) and the probability of each variation being the best (prob_best).data {
int<lower=0> g; // Number of variations
int<lower=0> x[g]; // Conversions per variation
int<lower=0> n[g]; // Users per variation
}
parameters {
real<lower=0, upper=1> p[g]; // Conversion rates (what we estimate)
}
model {
for(i in 1:g){
x[i] ~ binomial(n[i], p[i]); // Likelihood
}
// Implicit prior: p[i] ~ uniform(0, 1) = Beta(1, 1)
}
generated quantities {
matrix[g, g] prob_upper; // Pairwise: is p[i] > p[j]?
real prob_best[g]; // Is p[i] the best?
for(i in 1:g){
real others[g-1];
others = append_array(p[:i-1], p[i+1:]);
prob_best[i] = p[i] > max(others) ? 1 : 0;
for(j in 1:g){
prob_upper[i, j] = p[i] > p[j] ? 1 : 0;
}
}
}To prevent decisive Bayes Factors in one arm from triggering a false "safe to stop" verdict for an entire experiment, Bucketeer implements an all-or-nothing guard.
calcGoalResult: Skips value posterior inference for the entire goal if any variation has zero user count, mean, or variance.fillSequentialBayesFactors: Sets the value BF to $1.0$ for all arms unless every arm (including nil/missing entries) has valid sufficient statistics.This ensures that ValueSafeToStop=true is only triggered when the entire multi-arm experiment has sufficient data to be evaluated safely.
Bucketeer implements a serverless-style architecture for its background services to reduce compute engine and PubSub costs. Instead of running continuous background processes that consume resources even when idle, the system uses a cron job pattern to manage event processing.
AutoOps:
event-persister-evaluation-events-opsevent-persister-goal-events-opsExperiments:
event-persister-evaluation-events-dwhevent-persister-goal-events-dwhexperiment-calculatorBucketeer has moved i18n (internationalization) logic from the backend to the frontend. Instead of returning localized strings, the backend now returns structured error information using GRPC's ErrorInfo. This allows the frontend to handle translation using a messageKey and metadata.
An error response follows this structure:
{
"code": 3,
"message": "rpc error: code = InvalidArgument desc = account:invalid email",
"details": [
{
"reason": "INVALID",
"domain": "account.bucketeer.io",
"metadata": {
"messageKey": "InvalidArgumentError",
"email": "email.com",
"field_1": "APIKey"
}
}
]
}details:reason: The error reason (e.g., "INVALID").domain: The service name generating the error (e.g., "account.bucketeer.io").metadata.messageKey: The unique identifier used by the frontend to look up the correct translation template.metadata.<key>: Optional structured data used to populate template variables in the frontend (e.g., "email": "email.com").{
"code": 3,
"message": "rpc error: code = InvalidArgument desc = account:invalid email",
"details": [
{
"reason": "INVALID",
"domain": "account.bucketeer.io",
"metadata": {
"messageKey": "InvalidArgumentError",
"email": "email.com",
"field_1": "APIKey"
}
}
]
}The System Notification Center is a broadcast inbox feature designed for the Bucketeer admin console. It allows system admins to author and publish platform-wide announcements using Markdown, which are then visible to all authenticated console users.
Key features include:
Important Distinction: This feature is distinct from the legacy subscription domain (formerly misnamed as notification), which manages external delivery channels like Slack or FCM.
Bucketeer provides 95% Credible Intervals for its metrics. Unlike frequentist confidence intervals, a Bayesian credible interval provides a direct probability statement:
"There is a 95% probability that the true parameter is in this interval."
Credible intervals are derived by taking the 2.5th and 97.5th percentiles from the posterior samples after they have been sorted.
Bucketeer uses Redis Streams to store critical event data. You must never configure Redis with memory eviction policies (such as allkeys-lru or volatile-lru), as this will result in permanent data loss.
To ensure maximum data safety, use the following configuration:
--maxmemory-policy noeviction (CRITICAL)--maxmemory 4gb (Adjust based on your RAM capacity)--appendfsync always (For maximum durability, though it impacts performance)Standard Docker Compose settings for Bucketeer include --appendonly yes and --appendfsync everysec to balance performance and durability.
Notifications support internationalization (i18n) through a localization model where tags, title, and content (Markdown) are stored per language in the notification_localization table.
How resolution works:
language provided in the request to resolve content.For complex models involving multiple variations and pairwise comparisons, Bucketeer uses Markov Chain Monte Carlo (MCMC) sampling. Specifically, it utilizes the HMC-NUTS algorithm (Hamiltonian Monte Carlo with No-U-Turn Sampler).
Bucketeer uses Bayesian statistical methods rather than the traditional Frequentist approach to analyze A/B test results.
Unlike the Frequentist approach, which provides a binary answer (e.g., 'is the difference statistically significant at p < 0.05?'), the Bayesian approach provides a full probability distribution. This allows users to answer questions like: 'What is the probability that Variation B is better than Variation A?'
This is calculated using Bayes' Theorem:
Posterior = (Likelihood × Prior) / Evidence
The Docker Compose setup uses Docker Secrets to manage MySQL credentials instead of plain text environment variables. This improves security by using files with restricted permissions.
./secrets/mysql_root_password.txt./secrets/mysql_password.txt600 (read/write for owner only).secrets/ directory is automatically excluded from version control.To automatically create these secret files, use the setup command. To regenerate them, use the regenerate command.
make docker-compose-regenerate-secrets