Screaming into the void

Message queues, events streams, events sourcing, those fronts have almost become as entrenched as the language and editor debates. In the effort to win this religious war, engineers forget their actual use case and default to screaming into the void. But that's not the only way to go.

Screaming into the void is what happens when architecture ends at the choice of tools: You pick Kafka or Kinesis, set up a topic, add a couple subscribers and start publishing events. Nice and decoupled. The service sending the event does not have to pay attention to the consumers at all. Everyone can listen and decide what to do with the event they receive.

There is even a valid use case with this. Our old friend the Enterprise Service Bus often worked this way. Attached systems emit status events, like data modifications, inputs, deletions. Consumers on the bus would follow along and if there was something to do in reaction to a status change, they would do it. If this is your usecase, if you care about keeping other systems merely informed of the latest developments (like a news paper), you're all set. Keep going.

What else is there?

Of course we have to talk about everybody's favorite: CQRS, Command-Query-Responsibility-Separation. The quest to build this often enough leads to everyone screaming (their commands) into the void. But in this case you do care about your message getting received and you do care about what it effects. Screaming into the void is not good enough, in this case.

A command stream is more akin to a mail or telegraph system. You may not know which operator precisely will execute your command, but do know where to address your message to ensure it gets done. Architecturally, this means topic separation, at-least-once-processing, maybe dispatching.

Event sourcing is another audience favorite, that regularly gets mixed in when implementing the screaming-into-the-void pattern. Events get written into a queue and then are used as a basis to compute a current (or historic) state as needed.

For this application, screaming into the void is not only an anti-pattern, but actively opposing the use case. State change events (the source for event sourcing) get mixed with status updates (the consequence of a computed state change), commands (events that may or may not lead to state changes), commands, and anything else people see fit to send.

What do I choose?

Before making a choice, get clarity on what exactly you want to achieve.

Want to send commands to another system, have them processed in order, and generally care about the outcomes of your events? Try a message queue or at least ensure that the relevant constraints and processing guarantees apply.

Want to rely on your events as a data source that you can rearrange and modify and recompute to get to a state? Pick an actual event store or at least ensure that the topics are cleanly separated and events are guaranteed to be persisted to your expectations.

Want to keep your environment appraised of what's going on, but don't care about how they react what you do? Go ahead and buy Twitter scream into the void. Set up your favorite stream and publish your updates into it, for the world to consume or ignore.

By @cmw@dysfunctional.technology