* Add support for git worktrees in the tools image
* Coalesce read configuration for recent data cutoff
`query_frontend.search.query_ingesters_until` is removed in favor of only `query_frontend.search.query_backend_after`
* Rever doc changes
* Improve chlog entry
* Update tempo-distributed doc to be community maintained; remove GET
* Apply suggestion from @knylander-grafana
* Apply suggestion from @knylander-grafana
* added spanfilter to filter out spans
* return the error if the filters are not correctly formatted
* add filters to the userconfigurable overrides
* add core logic for drop items cache
* wired up everything
* some tweaks
* refactor
* fmt and manifest
* tighten names
* rename metric
* drop already buffered counterparts when a span is filter out
* add a new test and documentation
* Update docs/sources/tempo/operations/manage-advanced-systems/user-configurable-overrides.md
Co-authored-by: Suraj Nath <9503187+electron0zero@users.noreply.github.com>
* Update docs/sources/tempo/configuration/_index.md
Co-authored-by: Suraj Nath <9503187+electron0zero@users.noreply.github.com>
* Update docs/sources/tempo/configuration/_index.md
Co-authored-by: Suraj Nath <9503187+electron0zero@users.noreply.github.com>
* bail early when the span kind is not client or server
* Update docs/sources/tempo/operations/manage-advanced-systems/user-configurable-overrides.md
Co-authored-by: Suraj Nath <9503187+electron0zero@users.noreply.github.com>
---------
Co-authored-by: Suraj Nath <9503187+electron0zero@users.noreply.github.com>
* tempo-cli: support relative time strings (now, now-1h) for start/end args
Add support for relative time strings across all CLI commands that
accept start/end arguments, matching Grafana's time string format.
example: now-1h, now, now-30m, etc.
Affected commands:
- query api search
- query api search-tags
- query api search-tag-values
- query api metrics query-range
- query search
Help text previously said "ISO8601 format" or showed a ISO 8601 format but the code was parsing with
`time.RFC3339` (2006-01-02T15:04:05Z07:00). ISO 8601 is a broad standard that allows many representations
including timestamps without timezone offsets.
RFC 3339 is a strict profile of ISO 8601 that always requires a timezone offset (Z or +HH:MM).
Help text is update to say RFC3339 with an example.
We now accept a uniform time format across all commands: either RFC3339 absolute timestamps or relative time strings.
query search: previously accepted timestamps without timezone (2024-03-15T10:30:00) using the layout "2006-01-02T15:04:05".
This is valid ISO 8601 but not valid RFC 3339. These timestamps are now rejected.
Users must include a timezone offset (e.g. 2024-03-15T10:30:00Z) or use relative time (e.g. now-1h).
* update docs
* Add CHANGELOG.md
* trim spaces before parsing and add tests
* metrics-generator: Add per-label cardinality sanitizer
Adds a per-label cardinality cap that replaces only the high-cardinality
label value with "__overflow__" while preserving all other dimensions.
Implementation:
- CardinalitySanitizer: per-label HLL sketch tracking with cached
overLimit flag updated every 15s. Always inserts original value hash
to prevent estimate oscillation.
- ChainSanitizer: composes sanitizers in order (DRAIN first, then
cardinality sanitizer as hard backstop).
- New config field: max_cardinality_per_label (0 = disabled).
* metrics-generator: Add docs for per-label cardinality sanitizer
Rename overflow value to __cardinality_overflow__ for clarity and add
troubleshooting docs explaining the per-label cardinality limit, its
PromQL query, configuration, and how it interacts with the series and
entity limiters.
* metrics-generator: Move max_cardinality_per_label to per-tenant override
Move max_cardinality_per_label from static registry config to a
per-tenant runtime override, consistent with max_active_series and
max_active_entities. The CardinalitySanitizer now receives the
Overrides interface and refreshes the cached value every maintenance
tick (15s), allowing operators to tune it per tenant without restart.
* move from piggybacking on sanitizer and have a standalone per label limiter
sanitizer is there to sanitize the labels, so we shouldn't piggyback and rely
on the chained sanitizer hack to ensure per label limiter runs after all sanitizer.
we create a new LabelLimiter interface and run it after Sanitizer and decouple from Sanitizer
* extract maxSeriesFunc and maxEntityFunc to type
* test the overflow -> recovery -> overflow liftcycle
* make the test more strict
* rework docs
* track label_names and add a demand estimate metric as well
* Add tests for MetricsGeneratorMaxCardinalityPerLabel override
* optimize hot path
* Update TestPerLabelLimiter_ConcurrentAccess test
* Add BenchmarkPerLabelLimiter to check perf and allocs
* Add CHANGELOG.md
* Update config index page
* drop noopLabelLimiter from tests
* always run s.doPeriodicMaintenance first to refresh the config value
Add QPS and Latency panels for the external endpoint metric
(tempo_querier_external_endpoint_request_duration_seconds).
Co-authored-by: Cursor <cursoragent@cursor.com>
Hanzo Dev