Your logs fail you exactly when you need them

The scene is always the same. Somebody reports that “it did not work” a while ago. You go to the logs and find millions of lines like this one:

2026-09-03 14:22:07 ERROR Error processing payment

You do not know which user, which order, or what happened before. You search by time and forty similar errors come up. None of them tells you which one was that person’s.

The problem is not the volume of logs. It is that they are written to be read by a human line by line, which is exactly what nobody does when there is a real problem.

1. Write objects, not sentences

A structured log line is an object with fields, not a sentence:

// Before
logger.error(`Error processing payment for order ${order.id}`);

// After
logger.error('payment_failed', {
  order_id: order.id,
  user_id: user.id,
  amount: order.total,
  currency: order.currency,
  gateway: 'stripe',
  error_code: err.code,
  retry: attempt,
});

Now you can ask things that were impossible before: how many payments failed per error code, whether they cluster on one gateway, what the average amount of the failing ones is. That is querying, not reading.

Two rules that make the difference:

The message is a stable identifier, not a description. payment_failed can be grouped and counted. “Error processing payment for order 4821” is unique by definition, so it groups with nothing.

Never put personal data or secrets in the fields. Logs tend to have long retention, to travel to third-party services, and to be readable by more people than the database is. The user’s identifier, yes; their email, their ID number or their card number, no.

2. One identifier that runs through everything

With structured logs you can already filter by user_id. But a single request passes through several services, and generates dozens of lines in each. What is missing is a thread tying them together.

That thread is the trace context, and it is standardised: the W3C traceparent HTTP header.

traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
             ‾‾ ‾‾‾‾‾‾‾‾‾ trace id ‾‾‾‾‾‾‾‾‾‾‾‾ ‾‾ span id ‾‾ ‾‾

The trace id is the same across the whole operation, even when it crosses six services. The span id identifies the specific step.

What matters in practice: if that identifier appears on every log line, a user report turns into a single query.

logger.error('payment_failed', {
  trace_id: currentContext.traceId,
  span_id: currentContext.spanId,
  order_id: order.id,
  error_code: err.code,
});

And you do not have to thread it through every function by hand. The OpenTelemetry kits inject it automatically into the most common logging frameworks — Log4j, SLF4J, Python’s logging module — which means you get the correlation without rewriting a single line of your existing logs.

It is worth knowing the state of OpenTelemetry’s log support per language, because it is uneven: stable in Java, .NET, C++ and PHP; beta in Go and Rust; in development in Python, JavaScript, Ruby, Swift, Elixir and Kotlin. General adoption is around 48% of organisations, with another 25% planning it.

If your language is still in development, the correlation works anyway: it is adding two fields to the object you are already writing.

3. Log decisions, not steps

The most common mistake is not logging too little. It is logging a lot of what is useless:

logger.info('Entering processPayment');       // noise
logger.info('Validating card');               // noise
logger.info('Calling the gateway');           // noise
logger.info('Response received');             // noise

None of that will help you at three in the morning. This will:

// A decision: why the system took this path and not another
logger.info('payment_routed', {
  gateway: 'stripe',
  reason: 'currency_not_supported_on_primary',
  currency: 'ARS',
});

// A limit that was crossed
logger.warn('retries_exhausted', { order_id, attempts: 3, last_code: '502' });

// External data that arrived different from what was expected
logger.warn('unexpected_response', {
  service: 'gateway',
  expected: 'approved|declined',
  received: 'pending_review',
});

The question for deciding whether a line deserves to exist: will it let me explain why the system did what it did? If the answer is no, it is noise that also costs money to store.

The third example is the most valuable and the least written. When an external service starts returning a value your code does not account for, that log line is the difference between understanding it in five minutes and understanding it in two days.

A note on sampling

When volume grows, the reaction is to sample: keep one trace in a hundred. That makes sense for the ones that went fine.

Sample the successes, never the failures. A trace with an error is precisely the one you are going to need, and it is rare by definition: keeping all of them costs little. The right configuration discards most successful requests and keeps 100% of the ones that ended badly or took too long.

Where to start

If you have to pick one single change: put the trace id on every log line. That is what turns “something failed two hours ago” into a query that returns exactly that person’s lines, across every service, in order.

The rest makes life better. This changes how long it takes you to find a problem.