Feature Flag Service: Overrides, Rollouts and Targeting

Feature Flag Service: Overrides, Rollouts and Targeting

You've joined the platform team that owns flagsvc, the small service every product team calls to ask "is this feature on for this user?". Support has reported a few bugs in per-user overrides, and two features are queued behind them: gradual percentage rollouts and segment targeting. Everything runs in memory, and run handles one batch of operations against a fresh service.

Codebase

How your code is run

run(operations: list[list]) -> list executes the operations in order against a fresh service and returns one result per operation. Errors are returned as strings, never raised.

• Each error is returned as "error: <code>", e.g. "error: unknown flag".
• An unknown operation name returns "error: unknown operation". A wrong number of arguments, a malformed flag or segment name, an empty or non-string user_id, a non-boolean default, or an attrs that is not an object returns "error: invalid arguments". Names are lowercase letters, digits, ., - and _ (see config.NAME_PATTERN). User ids are case-sensitive.
• set_rollout, define_segment and add_rule exist in the dispatcher but currently answer "error: not supported".

Worked example

[["define", "new-checkout", false],
 ["evaluate", "new-checkout", "u1"],
 ["set_override", "new-checkout", "u1", true],
 ["evaluate", "new-checkout", "u1"],
 ["evaluate", "new-checkout", "u2"],
 ["evaluate", "dark-mode", "u1"]]

returns

[true, false, true, true, false, "error: unknown flag"]

Part 1 — Fix the override bugs (PLAT-512, PLAT-518, PLAT-525)

Goal: per-user overrides behave exactly as documented below.

Reports:

1. "We pinned banner off for one account on a flag whose default is on, and that account still sees the banner."
2. "set_override accepted 1 as the value, and a script that sent 0 was happily accepted too."
3. "clear_override says true even when I never set an override for that user."

Rules

1. An override is a JSON boolean. Anything else (numbers, strings, null, lists) returns invalid value, and nothing is stored.
2. An override of false is an explicit decision: it takes precedence over the flag's default exactly like an override of true does. Overriding the same user again replaces the old value.
3. Overrides belong to one (flag, user) pair; they never affect other users or other flags.
4. clear_override returns true if it removed an override and false if there was none. After clearing, the user is evaluated as if no override had ever been set.
5. Check order: the flag must exist (unknown flag) before the value is checked.

Part 2 — Percentage rollouts (PLAT-540)

Goal: set_rollout switches a flag on for a stable percentage of users.

A flag may have a rollout percentage. A user is in the rollout when their bucket is below the percentage's threshold; users in the rollout get true, everyone else falls through to the flag's default. (Rollouts never switch anyone off.)

Rules

1. A user's bucket is int(hex[:8], 16) % 10000, where hex is the hexadecimal SHA-256 digest of the UTF-8 string "<flag>:<user_id>". Use hashlib; Python's built-in hash() changes between processes. flagsvc/hashing.py is where this belongs.
2. percent is an integer or float from 0 to 100 inclusive; booleans, strings and out-of-range numbers return invalid percent. null removes the rollout and returns true. The threshold is round(percent * 100), and a user is in the rollout when bucket < threshold (strictly). So 0 includes nobody, 100 includes everybody, and 12.5 covers buckets 0..1249.
3. Calling set_rollout again replaces the previous percentage. Raising a percentage never removes a user who was already in.
4. A user override (Part 1) still wins over the rollout, in both directions.
5. Evaluation order is override, then rollout, then default.

Example. With new-checkout defined with default false, the buckets of u0, u4, u7 and u8 are 5510, 2686, 3758 and 71:

[["define", "new-checkout", false],
 ["set_rollout", "new-checkout", 30],
 ["evaluate", "new-checkout", "u0"],
 ["evaluate", "new-checkout", "u4"],
 ["evaluate", "new-checkout", "u7"],
 ["evaluate", "new-checkout", "u8"],
 ["set_rollout", "new-checkout", 40],
 ["evaluate", "new-checkout", "u7"]]

returns [true, true, false, true, false, true, true, true]. The threshold for 30 is 3000, so u4 (2686) and u8 (71) are in and u0 (5510) and u7 (3758) are out; raising the rollout to 40 (threshold 4000) brings u7 in.

Part 3 — Segment targeting rules (PLAT-561)

Goal: serve a specific value to users who match a segment, using attributes passed at evaluation time.

evaluate now accepts an optional third argument, attrs: a JSON object such as {"plan": "pro", "country": "DE"}. Calls without it behave exactly as before (attrs is {}).

A segment is a named list of conditions; a user is in the segment when all conditions match (logical AND). A rule attaches a segment to a flag with a value to serve.

Conditions are [attribute, operator, operand]:

Rules

1. define_segment stores the segment and returns true; defining an existing name replaces it, and flags that already use it see the new definition. name is validated like a flag name (invalid arguments); the condition list must be non-empty and every condition must be a three-element list with a non-empty string attribute, one of the operators above, and an operand of the right shape, otherwise invalid condition.
2. A condition on an attribute the user does not have is false for every operator, neq included. Strings compare case-sensitively. Booleans never equal numbers: true is not 1. gte/lt are false if the attribute is not a number.
3. add_rule appends a rule to the flag. Check order: unknown flag, unknown segment, then invalid value (the value must be a boolean). Rules are tried in the order they were added and the first rule whose segment matches wins, even if it serves false.
4. Precedence, highest first: (1) user override, (2) the first matching targeting rule, (3) percentage rollout, (4) flag default. A matching rule that serves false ends evaluation, so a rollout cannot switch that user back on.

Example

[["define", "search.v2", false],
 ["define_segment", "pro-eu", [["plan", "eq", "pro"], ["country", "in", ["DE", "FR"]]]],
 ["add_rule", "search.v2", "pro-eu", true],
 ["evaluate", "search.v2", "u1", {"plan": "pro", "country": "DE"}],
 ["evaluate", "search.v2", "u1", {"plan": "pro", "country": "US"}],
 ["evaluate", "search.v2", "u1"],
 ["set_override", "search.v2", "u1", false],
 ["evaluate", "search.v2", "u1", {"plan": "pro", "country": "DE"}]]

returns [true, true, true, true, false, false, true, false].

Notes

• store.py, models.py and validation.py are known-good; the Part 1 bugs are in the override handling.
• You may use the AI assistant freely. Later parts must not break earlier ones.
• Parts 1-3 are all in scope; there is no stretch part.