DQL timeframe: from, to and why a query returns few rows

You run a DQL query and it returns a handful of rows, or none, although you know the data exists. Very often the culprit is the DQL timeframe: the time window applied to the query is not the one you think. This article explains where that window comes from, how the from, to and timeframe parameters change it, and which other causes give a thin result. It applies to Dynatrace SaaS (DQL and Grail); Dynatrace Managed is not covered.

For an SRE team the stakes are practical: a query that undercounts errors skews a diagnosis, an SLI or a post-mortem. It is better to know within seconds whether the problem comes from the window, a filter or a limit.

In short

With no time parameter in the query, the window of the user interface applies (the « global timeframe »).
According to the documentation, the default window is 2 hours, unless set in the application or the API.
from and to in fetch override the interface selector.
A limit placed before an aggregate, an over-strict filter or sampling also explain a reduced result.

Where the DQL timeframe of a query comes from

The Dynatrace DQL documentation, on the « How to use DQL queries » page, recommends choosing the window with the interface controls. When the query says nothing, the time range selected in the interface is applied. The same page states that, unless set in the application or in the API, the default timeframe is 2 hours.

The direct consequence: a bare fetch logs reads the logs of the active window. If the selector shows the last 2 hours and the incident happened yesterday, the query returns few rows, or none. Nothing is broken. The query answers the question asked, but over a different period from the one you meant.

This dependence on the interface has another effect: two colleagues running the same query with different selectors get different results. For a shared notebook, dashboard or workflow, writing the window into the query avoids the trap.

Setting the window with from, to and timeframe

According to the « commands example » documentation, the fetch command accepts the optional parameters from, to, timeframe, samplingRatio and scanLimitGBytes. The examples below come from that page. The comments are ours.

Relative durations with from and to

Valid durations for from and to are offsets from now(): -25m means 25 minutes before now, -2h two hours before. The query fetch logs, from:-24h, to:-2h therefore reads the period from 24 hours to 2 hours in the past. It is handy for comparing with the recent period or leaving out the last, still incomplete minutes.

According to the « How to use DQL queries » page, these parameters take precedence over the selection made in the interface. If a query contains from:-10m, switching the selector to 7 days changes nothing: the query keeps reading 10 minutes. This is the second big source of surprise, the mirror image of the first.

Absolute range with timeframe

For a fixed range, such as the window of a past incident, use timeframe with two ISO 8601 timestamps separated by a slash, as in the documentation example: fetch logs, timeframe: "2021-10-20T00:00:00Z/2021-10-28T12:00:00Z". According to the documentation, from is only used when no explicit timeframe is given.

Why a DQL query returns few rows: the checklist

Work through these in order, from most to least common. Each point takes seconds to check.

Which window applies? Look at the interface selector and search the query for from, to or timeframe. Add an explicit window, for example from:-24h, and rerun.
Is a limit cutting too early? The limit command limits the number of returned records. The « DQL best practices » page warns against putting limit before aggregating unless intended, otherwise the aggregates are wrong.
Is the filter too strict? filter keeps only the records that match the condition. Remove conditions one at a time to see which one eliminates everything.
Is there a samplingRatio? For logs, a value of 10, 100, 1000 or 10000 returns about 1/N of the raw records. The result is non-deterministic.
Does a bucket restrict the source? The fetch syntax accepts a bucket parameter; the best-practices page cites it as a way to limit the data read. A badly chosen bucket excludes the records you want.
Is the scan cap in play? scanLimitGBytes controls the amount of uncompressed data read by the fetch stage; the default is 500 GB, and -1 analyses all data in the window. On a very large window, check this parameter.

A realistic before and after

Here are two queries that count log errors. The first combines three traps: an implicit window, a limit before the aggregate and a dependence on the selector. The second fixes them. The field names are those commonly used for logs; adapt them to your environment.

In the first, limit 20 reduces the stream to twenty records before summarize: the counter can never exceed 20, whatever the real volume. In the second, the window is written down and the counter covers every error from the last 24 hours. If you only want to display a few sample rows, put the limit after the aggregation, or in a separate query.

Checking that the query reads the right period

After the fix, check in three steps:

Run the query with a wide window (from:-7d) and then a narrow one: the row count should vary consistently.
Compare with a minimal query, fetch logs, from:-24h | summarize count(), to learn the total volume before filtering.
For a past incident, switch to timeframe with absolute timestamps, so the result stops moving when the selector changes.

On large volumes, keep the official best practice in mind: a shorter window gives better performance on identical data, and you should filter early, before selecting fields or aggregating. Narrowing the window should therefore not be a blind reflex: pick the shortest one that covers your question.

Best practices

Write the window into shared queries (notebooks, dashboards, workflows) rather than depending on the selector.
Filter early, sort last, and never put limit before an aggregate without a reason.
Document sampling: if you use samplingRatio:100, multiply the count by 100 as in the documentation example, and say the result is approximate.
Use timeframe for post-mortems so the analysis stays reproducible, in the blameless spirit of our DQL-based post-mortems.

Going further

If fetch, filter and aggregations are still new to you, start with the module on DQL fundamentals. For ready-to-use queries on logs and traces, see our operations query examples, and for the logic of filter and summarize, the article on DQL commands. The principle is the same as for group by semantics, covered in our summarize guide: a DQL query does exactly what you write, not what you imagine.

On the OpenTelemetry side, this behaviour also applies to the logs and traces you send: if an exporter has only been delivering for a few minutes, a window that is too old will show nothing. Check the period before suspecting the Collector.

Frequently asked questions

What is the default window of a DQL query?

With no time parameter, the query uses the window chosen in the interface. According to the Dynatrace documentation, the default is 2 hours, unless set in the application or the API.

What is the difference between from/to and timeframe?

from and to accept durations relative to now(), such as -24h, or timestamps. timeframe describes a full range, for example two ISO 8601 dates separated by a slash. from is only used when no explicit timeframe is given.

Why does my counter stay stuck at a small number?

Look for a limit placed before the summarize. The documentation advises against this order, unless intended, because it makes aggregates wrong.

Sources

span.observer is an independent blog, not affiliated with Dynatrace LLC. Product names belong to their respective owners; the sample queries are educational and should be adapted to your environment.

Originally published on span.observer. French version: https://span.observer/fr/dql-timeframe-from-to-requete-peu-de-lignes/.