Huddle Messaging Core: Unread Badges, Threads and Cursors

Huddle Messaging Core: Unread Badges, Threads and Cursors

You've joined the team that owns the messaging backend of Huddle, a workplace chat product: channels, threaded replies, @mentions, and the unread badges next to each channel. The backend has been live for a month and support is drowning in tickets about wrong badges and "Load older" buttons that repeat or skip messages. Part 1 is the incident cleanup. Parts 2 to 4 are the roadmap behind it, and some of them deliberately change rules you will have just made correct.

Codebase

How your code is run

run(operations: list[list]) -> list executes the operations in order against a fresh server and returns one result per operation. Every operation is [name, ts, *args]; ts is an integer number of seconds and never decreases. Errors are returned as strings, never raised: "error: <code>".

Conventions

• Argument shape. An argument that is the wrong type (a number where a string is expected, a bool where an int is expected, a missing or extra argument, a ts that is not an int) returns "error: invalid arguments", and this is checked before any other error. Channel ids and user ids passed to operations must be non-empty strings. Message ids only need to be strings: a string that is not a live message id is message not found. limit and cursors have their own errors (below).
• User ids used to create or join (create_channel, join) must match [A-Za-z0-9_]+ and must not be channel; otherwise invalid arguments.
• Message ids are m-<n>. <n> is one server-wide sequence number (not per channel), starting at 1, that advances by one for every message actually stored. Failed operations never use one up and deleted messages' numbers are never reused. A message keeps its id and its place in the channel forever. Compare sequence numbers as integers: m-10 is newer than m-9.
• A message's ts is the ts of the operation that posted it.
• A live message is one that has not been deleted.
• Errors listed for an operation are checked in the order shown, after the argument-shape check.

• invalid text: the text is empty or only whitespace. The text is stored exactly as given.
• edit: only the author may edit. An edit is allowed while ts - posted_ts <= 900 (900 itself is allowed, 901 is not).
• message not found for mark_read also covers a message that belongs to a different channel. A deleted message is still a valid place to mark as read (rule 2 below).
• A message line is m-3 ana@12: hello (id author@ts: text), with (edited) appended once the text has been edited at least once (even if the new text equals the old).

Part 1: Fix the badges and the paging (SUP-4410, SUP-4417, SUP-4432)

Goal: make unread counts, mention counts and history paging follow the rules below. Support's tickets:

1. "Load older messages shows the same message twice, or skips one, if somebody deleted something while I was scrolling."
2. "My badge shows messages that were deleted, and it counts things I wrote myself. After somebody edits an old message my badge sometimes drops to zero."
3. "@bo pings Bobby, @Bo and @bobby. Editing a message to add or remove a mention never changes anyone's badge. People who join a big channel see hundreds of unread messages."

Rules

1. Markers. Every member has one read marker per channel: a sequence number. A member added by create_channel, or by join, starts with the marker at the highest sequence number the server has assigned so far (in any channel), so history from before they arrived is never unread.
2. mark_read moves the marker to the sequence number of the given message, which must have been posted in that channel (it may since have been deleted). (Part 3 changes what happens when this would move the marker backwards.)
3. unread counts the live messages in the channel whose sequence number is greater than the caller's marker and that were written by somebody else.
4. mentions counts the live messages whose sequence number is greater than the caller's marker, that were written by somebody else, and that mention the caller by name (@user) or everyone (@channel). A message counts once even if it does both. (Once Part 2 arrives, a message can be a mention without being counted as unread.)
5. Mention syntax. A mention is an @ that is not directly preceded by a letter, digit or underscore, followed by the longest run of letters, digits and underscores. Matching is case-sensitive and that whole run must equal the user id (or channel, which mentions everyone). Examples are listed below the rules.
6. Mentions follow the current text. After an edit, the message mentions whoever its new text mentions. An edit never changes the message's id, position, or whether it is before or after anyone's marker.
7. delete removes the message from history, from unread and from mentions. Its id stays reserved.
8. history returns the live messages of the channel in sequence order (oldest first), one rendered line each. options is an object with optional limit (an int from 1 to 50, default 20), and at most one of before / after (a cursor). Any other key, a non-object, or both before and after is invalid arguments. Otherwise:

• no cursor: the newest limit messages;
• before: c: the newest limit messages whose sequence number is less than the cursor's position;
• after: c: the oldest limit messages whose sequence number is greater than the cursor's position.

1. Cursors. older is encode_cursor(seq) (from chat/cursors.py) for the first item of the page, to hand back as before; it is null if no live message is older than that item. newer is encode_cursor(seq) for the last item, to hand back as after; it is null if no live message is newer than that item. An empty page has both null. A cursor is just a position in the channel: its meaning must not depend on how many messages have been posted, edited or deleted since it was issued, even if the very message it was issued at has been deleted. A token that decode_cursor rejects (or a cursor that is not a string) is invalid cursor; a limit that is not an int in 1..50 (or is a bool) is invalid limit. Checks run: option shape, channel, membership, limit, cursor.
2. In a batch, a cursor in options may be written "$<n>.older" or "$<n>.newer": the corresponding cursor from the result of the n-th operation of the same run call (0-based). If that result has no such cursor (an error, or null), it is invalid cursor. This is handled in api.py.

On rule 5: bo is mentioned by "@bo hi", "(@bo)", "hey @bo.", "@@bo", "@bo-x" (the run stops at -) and "a\n@bo"; bo is not mentioned by "@bobby", "@Bo", "@bo_x", "bo@example.com" or "x@bo". "@channels" does not mention everyone; "@channel!" does.

Worked example

[["create_channel", 1, "general", ["ana", "bo"]],
 ["post", 10, "general", "ana", "morning @bo"],
 ["post", 11, "general", "bo", "hi!"],
 ["post", 12, "general", "ana", "standup in 5"],
 ["unread", 13, "bo", "general"],
 ["history", 14, "bo", "general", {"limit": 2}],
 ["history", 15, "bo", "general", {"limit": 2, "before": "$5.older"}]]

returns

[true, "m-1", "m-2", "m-3", {"unread": 2, "mentions": 1},
 {"items": ["m-2 bo@11: hi!", "m-3 ana@12: standup in 5"], "older": "cG9zOjI", "newer": null},
 {"items": ["m-1 ana@10: morning @bo"], "older": null, "newer": "cG9zOjE"}]

(bo joined an empty server, so the marker is 0; both ana posts are unread, one of them mentions bo. The cursor strings are opaque and shown only for completeness.)

tests/test_messages.py::test_editing_a_message_keeps_its_place_in_history currently fails.

Part 2: Threads (HUD-512)

Goal: add threaded replies, per-thread following, and "also send to channel". Unread counts must stay correct when people follow, unfollow, edit and delete.

Rules

1. A reply's parent must be a live top-level message of that channel (message not found otherwise, including a parent from another channel). A parent that is itself a reply is not a thread root. A reply takes the next sequence number like any message, so replies and top-level messages share one numbering and one marker per channel. For follow/unfollow, membership is checked against the parent's channel.
2. Following. A member follows a thread when the latest of these events says so: the thread's root author follows from the moment the root is posted (an implicit starting state); posting a reply makes the replier follow; follow makes the caller follow; unfollow makes the caller not follow. Following is a current state: it is evaluated when unread is asked, not when a reply was posted. Deleting a reply does not change anyone's following state.
3. Unread, revised. A live message counts toward unread for a member when its sequence number is greater than their marker, somebody else wrote it, and one of: it is a top-level message; it is a reply that was also sent to the channel; it is a reply in a thread the member follows right now. It counts once.
4. Mentions, revised. The mentions rule of Part 1 applies to replies too and does not depend on following: a reply that mentions you counts even in a thread you do not follow.
5. History (the channel timeline) contains top-level messages and replies sent with also_channel, in sequence order. Replies not sent to the channel never appear there. A reply that is in the timeline is rendered with (re m-<root>) after the text; a root that has at least one live reply gets [1 reply] / [<n> replies] at the very end of its line. Suffix order is always (re ...), (edited), [n replies]. thread renders replies without the (re ...) part.
6. Edits and the edit window work for replies like any message. Editing a reply's text changes who it mentions as in Part 1.
7. Deleting a root deletes its whole thread: every reply (including ones sent to the channel) disappears from history, thread, unread and mentions, and none of their ids can be used again. Deleting a single reply removes just that reply (and lowers the root's reply count).
8. mark_read accepts reply ids and moves the same single per-channel marker.

Example

[["create_channel", 1, "eng", ["ana", "bo", "cy"]],
 ["post", 10, "eng", "ana", "deploy plan?"],
 ["reply", 11, "eng", "bo", "m-1", "I'd wait for @cy"],
 ["reply", 12, "eng", "cy", "m-1", "fine by me", true],
 ["unread", 13, "ana", "eng"],
 ["unread", 13, "cy", "eng"],
 ["history", 14, "ana", "eng", {}],
 ["thread", 15, "ana", "eng", "m-1"]]

returns

[true, "m-1", "m-2", "m-3",
 {"unread": 2, "mentions": 0},
 {"unread": 2, "mentions": 1},
 {"items": ["m-1 ana@10: deploy plan? [2 replies]", "m-3 cy@12: fine by me (re m-1)"], "older": null, "newer": null},
 ["m-2 bo@11: I'd wait for @cy", "m-3 cy@12: fine by me"]]

ana wrote the root, so she follows the thread: m-2 and m-3 count (m-1 is her own), and nothing mentions her. cy replied, so he follows too: m-1 (top-level) and m-2 count, m-3 is his own, and m-2 mentions him. Only m-1 and m-3 are in the timeline.

Part 3: Muting, and read markers that sync across devices (HUD-540)

Goal: members can mute noisy channels, and mobile apps now sync read positions from several devices, so updates arrive out of order. This part changes two rules from earlier parts.

Rules

1. Muted channels. While a member has a channel muted, unread reports "unread": 0 and "mentions" counts only messages that mention them by name (@user); @channel mentions are not counted. Muting changes nothing else: the marker still moves, nothing is forgotten, and unmute immediately restores the normal numbers. mute and unmute are idempotent.
2. Markers never move backwards (replaces Part 1 rule 2). mark_read (and sync_reads) set the marker to the larger of the current marker and the given message's sequence number. Marking an older message is accepted (true) and has no effect. Errors are unchanged.
3. sync_reads applies a device's list of positions in one go. It is all-or-nothing: if the list or any pair is malformed (not a non-empty list of [channel, message_id] string pairs) the result is invalid arguments; otherwise, if any update fails, the first failing update in list order (with mark_read's checks and order) determines the error. In every error case no marker changes. Otherwise each update is applied as in rule 2 (several updates for one channel are fine, in any order).
4. The joining marker (Part 1 rule 1) is unchanged.

Example

[["create_channel", 1, "ops", ["ana", "bo"]],
 ["post", 10, "ops", "ana", "page is down @bo"],
 ["post", 11, "ops", "ana", "@channel all hands"],
 ["post", 12, "ops", "ana", "still down"],
 ["mute", 13, "bo", "ops"],
 ["unread", 14, "bo", "ops"],
 ["mark_read", 15, "bo", "ops", "m-3"],
 ["mark_read", 16, "bo", "ops", "m-1"],
 ["unmute", 17, "bo", "ops"],
 ["unread", 18, "bo", "ops"]]

returns

[true, "m-1", "m-2", "m-3", true,
 {"unread": 0, "mentions": 1},
 true, true, true,
 {"unread": 0, "mentions": 0}]

(Muted, bo is told only about the direct @bo. The late mark_read of m-1 cannot undo the read of m-3.)

Part 4: Channel report at scale (HUD-563)

Goal: moderators open a "who is behind?" report on channels with tens of thousands of messages, and a bulk import tool replays history. Both must be fast, and so must unread.

Rules

1. bulk_post posts count messages as if by count separate post operations (same ts, consecutive sequence numbers, indexed and counted exactly like normal posts). For i = 0 .. count-1 the author is authors[i % len(authors)] and the text is msg <i>, with @<target> appended when target is not null, every > 0 and (i + 1) % every == 0. authors is a non-empty list of strings, count an int in 1..100000, every an int >= 0, target null, "channel" or a valid user id; otherwise invalid arguments (checked first). If any author is not a member, the result is not a member and nothing is posted.
2. unread_report considers every member of the channel and takes their unread and mentions exactly as unread would report them right now (muting and following included). It returns the rows [user, unread, mentions] of the members whose unread is at least min_unread, sorted by unread descending and then user ascending, cut to the first limit rows. min_unread and limit must be ints >= 1 (not bool), otherwise invalid arguments (checked first).
3. Hidden tests post tens of thousands of messages and interleave many unread_report and unread calls with mark_read, edits, deletes, follows and mutes. A solution that rescans the channel for every member on every report will not finish. Every rule from Parts 1 to 3 must still hold exactly.

Example

[["create_channel", 1, "random", ["ana", "bo", "cy"]],
 ["bulk_post", 5, "random", ["ana", "bo"], 5, 2, "cy"],
 ["mark_read", 6, "cy", "random", "m-2"],
 ["unread_report", 7, "random", 3, 10],
 ["unread_report", 8, "random", 1, 2]]

returns

[true, "m-1..m-5", true,
 [["bo", 3, 0], ["cy", 3, 1]],
 [["bo", 3, 0], ["cy", 3, 1]]]

The texts are msg 0, msg 1 @cy, msg 2, msg 3 @cy, msg 4, written by ana, bo, ana, bo, ana. cy read up to m-2, so m-3..m-5 are unread and m-4 mentions them. ana has 2 unread and is cut by the limit of 2 in the second report.

Notes

• ids.py, cursors.py, errors.py and config.py are known-good.
• You may use the AI assistant freely; you are expected to verify what it produces.
• Later parts must not break earlier ones except where a part says it replaces a rule; hidden tests re-check earlier behaviour after the later features exist.
• Parts 1 to 3 is a strong result. Part 4 is a stretch.