Realtime Notifications Service
Fan-out delivery service pushing in-app and web-push notifications with at-least-once guarantees.
- role
- Backend engineer
- period
- 2023
Problem
Notifications were sent inline from request handlers. A slow push provider made unrelated API calls hang, and there was no retry — a transient failure meant a dropped notification.
What was built
- A dedicated service consuming domain events and expanding them into per-recipient deliveries.
- Per-channel workers (in-app, web push, email) with independent retry and backoff.
- An outbox in Postgres for exactly-once handoff from the producing services, plus Redis for deduplication within a delivery window.
- Delivery status tracking exposed through a small API for support tooling.
Outcome
Delivery success went from ~92% to 99.7% measured over a month, and notification work no longer touched request latency.