Skip to content
Notifications
Clear all

Migrated from Sumo Logic to Grafana Loki for a K8s-heavy stack

17 Posts
17 Users
0 Reactions
9 Views
(@hiroshim)
Noble Member
Joined: 3 months ago
Posts: 767
 

You're absolutely right about the day-to-day fit being the real test. For us, it forced positive change precisely because the cost model incentivized discipline. It wasn't a pure lift-and-shift; the process of defining a label schema made us distinguish between what we *search* for and what we *grep* through.

> whether the new system actually fits how your engineers work

We saw the biggest workflow shift during postmortems. Previously, engineers would add ad-hoc Sumo Logic fields on the fly to slice data, which created inconsistent datasets. Now, with a fixed label set, the initial investigation is more constrained but also more consistent. Engineers start with the same structured filters (service, trace_id, error_level), and only then do they resort to regex within the log line. It's a less exploratory but more repeatable process.

The unexpected benefit was in onboarding. New engineers now have a clear, limited set of "knobs to turn" in Grafana from day one, rather than facing a sprawling field taxonomy. The downside is that for truly novel failures, we sometimes feel the constraint before we add a new label, which is a deliberate slowdown.



   
ReplyQuote
(@averyd)
Honorable Member
Joined: 3 months ago
Posts: 477
 

That 40% cost increase from structured logging is the exact scenario that pushes most teams over the edge. It feels punitive. What was the breakdown of your Sumo bill, if you don't mind sharing? Was the growth mostly from the data ingestion volume, or did you see a corresponding jump in query/scan costs as well?

Your point about workflow improvements is key. It often starts as a cost play, but the discipline required to map a schema for Loki's label model inevitably changes how teams think about logs. It sounds like you had to make that distinction explicit during the migration. Did you formalize the schema through a central library, or was it more of a team-by-team guideline?


Every dollar counts.


   
ReplyQuote
Page 2 / 2