Skip to content

Fix flaky test spring kafka template produce and batch consume#11168

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 2 commits intomasterfrom
ygree/fix-flaky-kafka-batch
Apr 21, 2026
Merged

Fix flaky test spring kafka template produce and batch consume#11168
gh-worker-dd-mergequeue-cf854d[bot] merged 2 commits intomasterfrom
ygree/fix-flaky-kafka-batch

Conversation

@ygree
Copy link
Copy Markdown
Contributor

@ygree ygree commented Apr 21, 2026 •
edited
Loading

What Does This Do

Fix test spring kafka template produce and batch consume

Sort both sides by the same key - producer span ID, so the correspondence is derived from the data rather than assumed by arrival order. Extract the three produce spans and sort them by spanId. Sort the consumer trace groups by the parentId of their kafka.deliver span, which equals the produce span ID of the message they consumed. By construction produceSpans[i].spanId == sortedConsumerTraces[i].queueSpan.parentId, making the assertion stable regardless of partition assignment or async callback interleaving. Offsets are then verified per-span by reading them from the actual ConsumerRecord linked via the x-datadog-parent-id header, preserving exact coverage without hardcoding.

Motivation

The test hardcodes a fixed mapping between consumer trace group positions (trace[1] → first in sort order) and producer span positions (trace(0)[6] → 3rd produce span). These two orderings are independent. Most of the time they happen to align, but ~1% of the time they diverge, causing the childOf assertion to find the wrong parent span.

Additional Notes

test spring kafka template produce and batch consume sends 3 messages to a 2-partition topic and asserts the three kafka.deliver queue spans are children of specific kafka.produce spans using hardcoded indices into the sorted trace (trace(0)[2], [4], [6]). Because partition assignment is non-deterministic, the consumer trace groups arrive in the ListWriter in a different order run-to-run, so the hardcoded index-to-parent mapping is wrong whenever the partition distribution changes.

Both the passing run and the failing run emitted exactly the same span structure: one 7-span producer trace and three 2-span consumer/queue traces. What differed was the pairing order.

Passing run — consumer queue spans arrived in parent order 1396, 1395, 1397, which happened to match the test's assumed order:

  s_id=1399 → p_id=1396   offset=0 partition=1
  s_id=1405 → p_id=1395   offset=0 partition=0
  s_id=1409 → p_id=1397   offset=1 partition=0

Failing run — partition 1 received two messages (offsets 0 and 1), partition 0 received one (offset 0), so the consumer queue spans arrived in parent order 1417, 1419, 1418 instead:

  s_id=1423 → p_id=1417   offset=0 partition=1
  s_id=1427 → p_id=1419   offset=1 partition=1   ← test expected p_id=1418 here
  s_id=1429 → p_id=1418   offset=0 partition=0

The assertion span.parentId(1419) == parent.spanId(1418) failed.

Log fragment:

15:33:24.676 [Test worker] Started span: DDSpan [ t_id=1416, s_id=1417, p_id=1415 ] trace=.../kafka.produce/...
15:33:24.679 [Test worker] Started span: DDSpan [ t_id=1416, s_id=1418, p_id=1415 ] trace=.../kafka.produce/...
15:33:24.680 [Test worker] Started span: DDSpan [ t_id=1416, s_id=1419, p_id=1415 ] trace=.../kafka.produce/...

15:33:24.682 [-C-1] Started span: DDSpan [ t_id=1416, s_id=1423, p_id=1417 ] trace=.../kafka.deliver/...
15:33:24.684 [-C-1] Started span: DDSpan [ t_id=1416, s_id=1427, p_id=1419 ] trace=.../kafka.deliver/...
15:33:24.684 [-C-1] Started span: DDSpan [ t_id=1416, s_id=1429, p_id=1418 ] trace=.../kafka.deliver/...

assert span.parentId == parent.spanId
       |    1419     |        1418
       DDSpan [ t_id=1416, s_id=1427, p_id=1419 ]  // queue span
       DDSpan [ t_id=1416, s_id=1418, p_id=1415 ]  // expected produce span

Contributor Checklist

Jira ticket: IDMPL-375

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

@ygree ygree self-assigned this Apr 21, 2026
@ygree ygree added type: bug Bug report and fix tag: no release notes Changes to exclude from release notes tag: ai generated Largely based on code generated by an AI or LLM labels Apr 21, 2026
@ygree ygree force-pushed the ygree/fix-flaky-kafka-batch branch from cde2dd1 to 4688370 Compare April 21, 2026 05:23
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Apr 21, 2026 •
edited
Loading

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master ygree/fix-flaky-kafka-batch
git_commit_date 1776784552 1776789337
git_commit_sha 63343ba 534d85c
See matching parameters
Baseline Candidate
ci_job_date 1776790556 1776790556
ci_job_id 1616383143 1616383143
ci_pipeline_id 108843453 108843453
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Apr 21, 2026 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master ygree/fix-flaky-kafka-batch
git_commit_date 1776784552 1776789337
git_commit_sha 63343ba 534d85c
release_version 1.62.0-SNAPSHOT~63343baba3 1.62.0-SNAPSHOT~534d85ccf8
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1776791285 1776791285
ci_job_id 1616383131 1616383131
ci_pipeline_id 108843453 108843453
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-2-bd67s7x8 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-2-bd67s7x8 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 63 metrics, 8 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~534d85ccf8, baseline=1.62.0-SNAPSHOT~63343baba3

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.057 s) : 0, 1057292
Total [baseline] (11.057 s) : 0, 11056749
Agent [candidate] (1.056 s) : 0, 1055720
Total [candidate] (11.128 s) : 0, 11127641
section appsec
Agent [baseline] (1.258 s) : 0, 1258028
Total [baseline] (10.977 s) : 0, 10976702
Agent [candidate] (1.261 s) : 0, 1261086
Total [candidate] (11.095 s) : 0, 11095329
section iast
Agent [baseline] (1.242 s) : 0, 1241507
Total [baseline] (11.531 s) : 0, 11530539
Agent [candidate] (1.25 s) : 0, 1249562
Total [candidate] (11.429 s) : 0, 11429384
section profiling
Agent [baseline] (1.185 s) : 0, 1184724
Total [baseline] (10.96 s) : 0, 10960484
Agent [candidate] (1.186 s) : 0, 1185685
Total [candidate] (10.98 s) : 0, 10980251
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent appsec 1.258 s 200.737 ms (19.0%)
Agent iast 1.242 s 184.215 ms (17.4%)
Agent profiling 1.185 s 127.432 ms (12.1%)
Total tracing 11.057 s -
Total appsec 10.977 s -80.046 ms (-0.7%)
Total iast 11.531 s 473.79 ms (4.3%)
Total profiling 10.96 s -96.264 ms (-0.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.056 s -
Agent appsec 1.261 s 205.366 ms (19.5%)
Agent iast 1.25 s 193.842 ms (18.4%)
Agent profiling 1.186 s 129.965 ms (12.3%)
Total tracing 11.128 s -
Total appsec 11.095 s -32.311 ms (-0.3%)
Total iast 11.429 s 301.743 ms (2.7%)
Total profiling 10.98 s -147.389 ms (-1.3%) 
gantt
    title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~534d85ccf8, baseline=1.62.0-SNAPSHOT~63343baba3

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.216 ms) : 0, 1216
crashtracking [candidate] (1.215 ms) : 0, 1215
BytebuddyAgent [baseline] (633.536 ms) : 0, 633536
BytebuddyAgent [candidate] (633.167 ms) : 0, 633167
AgentMeter [baseline] (29.632 ms) : 0, 29632
AgentMeter [candidate] (29.55 ms) : 0, 29550
GlobalTracer [baseline] (248.925 ms) : 0, 248925
GlobalTracer [candidate] (249.392 ms) : 0, 249392
AppSec [baseline] (32.297 ms) : 0, 32297
AppSec [candidate] (32.296 ms) : 0, 32296
Debugger [baseline] (59.661 ms) : 0, 59661
Debugger [candidate] (59.648 ms) : 0, 59648
Remote Config [baseline] (590.136 µs) : 0, 590
Remote Config [candidate] (585.013 µs) : 0, 585
Telemetry [baseline] (7.996 ms) : 0, 7996
Telemetry [candidate] (9.479 ms) : 0, 9479
Flare Poller [baseline] (7.435 ms) : 0, 7435
Flare Poller [candidate] (4.374 ms) : 0, 4374
section appsec
crashtracking [baseline] (1.211 ms) : 0, 1211
crashtracking [candidate] (1.223 ms) : 0, 1223
BytebuddyAgent [baseline] (672.07 ms) : 0, 672070
BytebuddyAgent [candidate] (674.173 ms) : 0, 674173
AgentMeter [baseline] (12.181 ms) : 0, 12181
AgentMeter [candidate] (12.11 ms) : 0, 12110
GlobalTracer [baseline] (248.553 ms) : 0, 248553
GlobalTracer [candidate] (248.886 ms) : 0, 248886
AppSec [baseline] (185.099 ms) : 0, 185099
AppSec [candidate] (185.529 ms) : 0, 185529
Debugger [baseline] (66.537 ms) : 0, 66537
Debugger [candidate] (66.766 ms) : 0, 66766
Remote Config [baseline] (554.451 µs) : 0, 554
Remote Config [candidate] (570.98 µs) : 0, 571
Telemetry [baseline] (7.882 ms) : 0, 7882
Telemetry [candidate] (7.838 ms) : 0, 7838
Flare Poller [baseline] (3.477 ms) : 0, 3477
Flare Poller [candidate] (3.432 ms) : 0, 3432
IAST [baseline] (24.212 ms) : 0, 24212
IAST [candidate] (24.276 ms) : 0, 24276
section iast
crashtracking [baseline] (1.232 ms) : 0, 1232
crashtracking [candidate] (1.244 ms) : 0, 1244
BytebuddyAgent [baseline] (817.075 ms) : 0, 817075
BytebuddyAgent [candidate] (822.351 ms) : 0, 822351
AgentMeter [baseline] (11.585 ms) : 0, 11585
AgentMeter [candidate] (11.635 ms) : 0, 11635
GlobalTracer [baseline] (240.458 ms) : 0, 240458
GlobalTracer [candidate] (242.242 ms) : 0, 242242
AppSec [baseline] (25.837 ms) : 0, 25837
AppSec [candidate] (31.701 ms) : 0, 31701
Debugger [baseline] (65.182 ms) : 0, 65182
Debugger [candidate] (65.281 ms) : 0, 65281
Remote Config [baseline] (532.695 µs) : 0, 533
Remote Config [candidate] (535.831 µs) : 0, 536
Telemetry [baseline] (7.765 ms) : 0, 7765
Telemetry [candidate] (7.868 ms) : 0, 7868
Flare Poller [baseline] (3.371 ms) : 0, 3371
Flare Poller [candidate] (3.462 ms) : 0, 3462
IAST [baseline] (31.577 ms) : 0, 31577
IAST [candidate] (26.893 ms) : 0, 26893
section profiling
ProfilingAgent [baseline] (93.922 ms) : 0, 93922
ProfilingAgent [candidate] (93.851 ms) : 0, 93851
crashtracking [baseline] (1.182 ms) : 0, 1182
crashtracking [candidate] (1.195 ms) : 0, 1195
BytebuddyAgent [baseline] (691.563 ms) : 0, 691563
BytebuddyAgent [candidate] (691.719 ms) : 0, 691719
AgentMeter [baseline] (9.146 ms) : 0, 9146
AgentMeter [candidate] (9.334 ms) : 0, 9334
GlobalTracer [baseline] (207.095 ms) : 0, 207095
GlobalTracer [candidate] (207.732 ms) : 0, 207732
AppSec [baseline] (32.778 ms) : 0, 32778
AppSec [candidate] (32.902 ms) : 0, 32902
Debugger [baseline] (65.86 ms) : 0, 65860
Debugger [candidate] (65.814 ms) : 0, 65814
Remote Config [baseline] (586.257 µs) : 0, 586
Remote Config [candidate] (573.39 µs) : 0, 573
Telemetry [baseline] (7.77 ms) : 0, 7770
Telemetry [candidate] (7.728 ms) : 0, 7728
Flare Poller [baseline] (3.552 ms) : 0, 3552
Flare Poller [candidate] (3.506 ms) : 0, 3506
Profiling [baseline] (94.482 ms) : 0, 94482
Profiling [candidate] (94.402 ms) : 0, 94402
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~534d85ccf8, baseline=1.62.0-SNAPSHOT~63343baba3

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.066 s) : 0, 1066118
Total [baseline] (8.859 s) : 0, 8858571
Agent [candidate] (1.057 s) : 0, 1056611
Total [candidate] (8.851 s) : 0, 8850954
section iast
Agent [baseline] (1.234 s) : 0, 1233997
Total [baseline] (9.6 s) : 0, 9600134
Agent [candidate] (1.229 s) : 0, 1229108
Total [candidate] (9.579 s) : 0, 9578552
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.066 s -
Agent iast 1.234 s 167.88 ms (15.7%)
Total tracing 8.859 s -
Total iast 9.6 s 741.562 ms (8.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent iast 1.229 s 172.497 ms (16.3%)
Total tracing 8.851 s -
Total iast 9.579 s 727.598 ms (8.2%) 
gantt
    title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~534d85ccf8, baseline=1.62.0-SNAPSHOT~63343baba3

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.235 ms) : 0, 1235
crashtracking [candidate] (1.221 ms) : 0, 1221
BytebuddyAgent [baseline] (638.95 ms) : 0, 638950
BytebuddyAgent [candidate] (633.484 ms) : 0, 633484
AgentMeter [baseline] (29.804 ms) : 0, 29804
AgentMeter [candidate] (29.583 ms) : 0, 29583
GlobalTracer [baseline] (250.766 ms) : 0, 250766
GlobalTracer [candidate] (249.551 ms) : 0, 249551
AppSec [baseline] (32.651 ms) : 0, 32651
AppSec [candidate] (32.373 ms) : 0, 32373
Debugger [baseline] (59.387 ms) : 0, 59387
Debugger [candidate] (59.003 ms) : 0, 59003
Remote Config [baseline] (596.566 µs) : 0, 597
Remote Config [candidate] (591.397 µs) : 0, 591
Telemetry [baseline] (8.12 ms) : 0, 8120
Telemetry [candidate] (7.964 ms) : 0, 7964
Flare Poller [baseline] (8.205 ms) : 0, 8205
Flare Poller [candidate] (6.574 ms) : 0, 6574
section iast
crashtracking [baseline] (1.27 ms) : 0, 1270
crashtracking [candidate] (1.216 ms) : 0, 1216
BytebuddyAgent [baseline] (813.248 ms) : 0, 813248
BytebuddyAgent [candidate] (808.402 ms) : 0, 808402
AgentMeter [baseline] (11.416 ms) : 0, 11416
AgentMeter [candidate] (11.392 ms) : 0, 11392
GlobalTracer [baseline] (238.717 ms) : 0, 238717
GlobalTracer [candidate] (239.052 ms) : 0, 239052
IAST [baseline] (31.694 ms) : 0, 31694
IAST [candidate] (28.977 ms) : 0, 28977
AppSec [baseline] (26.252 ms) : 0, 26252
AppSec [candidate] (26.559 ms) : 0, 26559
Debugger [baseline] (62.079 ms) : 0, 62079
Debugger [candidate] (63.55 ms) : 0, 63550
Remote Config [baseline] (521.496 µs) : 0, 521
Remote Config [candidate] (522.598 µs) : 0, 523
Telemetry [baseline] (7.617 ms) : 0, 7617
Telemetry [candidate] (7.639 ms) : 0, 7639
Flare Poller [baseline] (3.363 ms) : 0, 3363
Flare Poller [candidate] (3.362 ms) : 0, 3362
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master ygree/fix-flaky-kafka-batch
git_commit_date 1776784552 1776789337
git_commit_sha 63343ba 534d85c
release_version 1.62.0-SNAPSHOT~63343baba3 1.62.0-SNAPSHOT~534d85ccf8
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1776791853 1776791853
ci_job_id 1616383133 1616383133
ci_pipeline_id 108843453 108843453
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-4-fxj4rf4i 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-4-fxj4rf4i 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 2 performance regressions! Performance is the same for 19 metrics, 15 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:iast_FULL:high_load worse
[+126.125µs; +310.378µs] or [+2.409%; +5.929%]		 unsure
[+148.071µs; +782.342µs] or [+1.184%; +6.255%]		 unstable
[-97.749op/s; +51.437op/s] or [-12.544%; +6.601%]		 5.453ms 12.973ms 756.094op/s 5.235ms 12.507ms 779.250op/s
scenario:load:petclinic:profiling:high_load worse
[+458.424µs; +1407.715µs] or [+2.509%; +7.704%]		 unsure
[+0.146ms; +1.871ms] or [+0.495%; +6.345%]		 unstable
[-35.122op/s; +17.497op/s] or [-14.068%; +7.008%]		 19.204ms 30.495ms 240.844op/s 18.271ms 29.486ms 249.656op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~534d85ccf8, baseline=1.62.0-SNAPSHOT~63343baba3
    dateFormat X
    axisFormat %s
section baseline
no_agent (19.21 ms) : 19018, 19402
.   : milestone, 19210,
appsec (19.094 ms) : 18899, 19289
.   : milestone, 19094,
code_origins (17.95 ms) : 17774, 18126
.   : milestone, 17950,
iast (18.069 ms) : 17893, 18244
.   : milestone, 18069,
profiling (18.694 ms) : 18505, 18882
.   : milestone, 18694,
tracing (18.006 ms) : 17827, 18185
.   : milestone, 18006,
section candidate
no_agent (19.299 ms) : 19109, 19488
.   : milestone, 19299,
appsec (18.71 ms) : 18518, 18902
.   : milestone, 18710,
code_origins (17.722 ms) : 17548, 17895
.   : milestone, 17722,
iast (17.806 ms) : 17631, 17980
.   : milestone, 17806,
profiling (19.38 ms) : 19184, 19576
.   : milestone, 19380,
tracing (18.309 ms) : 18126, 18492
.   : milestone, 18309,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 19.21 ms [19.018 ms, 19.402 ms] -
appsec 19.094 ms [18.899 ms, 19.289 ms] -115.851 µs (-0.6%)
code_origins 17.95 ms [17.774 ms, 18.126 ms] -1.26 ms (-6.6%)
iast 18.069 ms [17.893 ms, 18.244 ms] -1.141 ms (-5.9%)
profiling 18.694 ms [18.505 ms, 18.882 ms] -515.887 µs (-2.7%)
tracing 18.006 ms [17.827 ms, 18.185 ms] -1.203 ms (-6.3%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 19.299 ms [19.109 ms, 19.488 ms] -
appsec 18.71 ms [18.518 ms, 18.902 ms] -588.54 µs (-3.0%)
code_origins 17.722 ms [17.548 ms, 17.895 ms] -1.577 ms (-8.2%)
iast 17.806 ms [17.631 ms, 17.98 ms] -1.493 ms (-7.7%)
profiling 19.38 ms [19.184 ms, 19.576 ms] 81.468 µs (0.4%)
tracing 18.309 ms [18.126 ms, 18.492 ms] -989.632 µs (-5.1%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~534d85ccf8, baseline=1.62.0-SNAPSHOT~63343baba3
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.271 ms) : 1259, 1284
.   : milestone, 1271,
iast (3.359 ms) : 3309, 3408
.   : milestone, 3359,
iast_FULL (5.935 ms) : 5875, 5995
.   : milestone, 5935,
iast_GLOBAL (3.699 ms) : 3640, 3758
.   : milestone, 3699,
profiling (2.233 ms) : 2212, 2254
.   : milestone, 2233,
tracing (1.909 ms) : 1892, 1925
.   : milestone, 1909,
section candidate
no_agent (1.242 ms) : 1230, 1254
.   : milestone, 1242,
iast (3.428 ms) : 3376, 3479
.   : milestone, 3428,
iast_FULL (6.118 ms) : 6055, 6181
.   : milestone, 6118,
iast_GLOBAL (3.696 ms) : 3628, 3763
.   : milestone, 3696,
profiling (2.253 ms) : 2231, 2275
.   : milestone, 2253,
tracing (1.905 ms) : 1888, 1921
.   : milestone, 1905,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.271 ms [1.259 ms, 1.284 ms] -
iast 3.359 ms [3.309 ms, 3.408 ms] 2.087 ms (164.2%)
iast_FULL 5.935 ms [5.875 ms, 5.995 ms] 4.663 ms (366.8%)
iast_GLOBAL 3.699 ms [3.64 ms, 3.758 ms] 2.428 ms (190.9%)
profiling 2.233 ms [2.212 ms, 2.254 ms] 961.443 µs (75.6%)
tracing 1.909 ms [1.892 ms, 1.925 ms] 637.111 µs (50.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.242 ms [1.23 ms, 1.254 ms] -
iast 3.428 ms [3.376 ms, 3.479 ms] 2.186 ms (176.1%)
iast_FULL 6.118 ms [6.055 ms, 6.181 ms] 4.876 ms (392.7%)
iast_GLOBAL 3.696 ms [3.628 ms, 3.763 ms] 2.454 ms (197.6%)
profiling 2.253 ms [2.231 ms, 2.275 ms] 1.011 ms (81.4%)
tracing 1.905 ms [1.888 ms, 1.921 ms] 662.903 µs (53.4%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master ygree/fix-flaky-kafka-batch
git_commit_date 1776784552 1776789337
git_commit_sha 63343ba 534d85c
release_version 1.62.0-SNAPSHOT~63343baba3 1.62.0-SNAPSHOT~534d85ccf8
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1776791516 1776791516
ci_job_id 1616383136 1616383136
ci_pipeline_id 108843453 108843453
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-5-dt9peoeb 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-5-dt9peoeb 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~534d85ccf8, baseline=1.62.0-SNAPSHOT~63343baba3
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.974 s) : 14974000, 14974000
.   : milestone, 14974000,
appsec (14.789 s) : 14789000, 14789000
.   : milestone, 14789000,
iast (18.167 s) : 18167000, 18167000
.   : milestone, 18167000,
iast_GLOBAL (17.939 s) : 17939000, 17939000
.   : milestone, 17939000,
profiling (14.786 s) : 14786000, 14786000
.   : milestone, 14786000,
tracing (14.902 s) : 14902000, 14902000
.   : milestone, 14902000,
section candidate
no_agent (15.406 s) : 15406000, 15406000
.   : milestone, 15406000,
appsec (14.681 s) : 14681000, 14681000
.   : milestone, 14681000,
iast (18.581 s) : 18581000, 18581000
.   : milestone, 18581000,
iast_GLOBAL (18.05 s) : 18050000, 18050000
.   : milestone, 18050000,
profiling (15.366 s) : 15366000, 15366000
.   : milestone, 15366000,
tracing (15.087 s) : 15087000, 15087000
.   : milestone, 15087000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.974 s [14.974 s, 14.974 s] -
appsec 14.789 s [14.789 s, 14.789 s] -185.0 ms (-1.2%)
iast 18.167 s [18.167 s, 18.167 s] 3.193 s (21.3%)
iast_GLOBAL 17.939 s [17.939 s, 17.939 s] 2.965 s (19.8%)
profiling 14.786 s [14.786 s, 14.786 s] -188.0 ms (-1.3%)
tracing 14.902 s [14.902 s, 14.902 s] -72.0 ms (-0.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.406 s [15.406 s, 15.406 s] -
appsec 14.681 s [14.681 s, 14.681 s] -725.0 ms (-4.7%)
iast 18.581 s [18.581 s, 18.581 s] 3.175 s (20.6%)
iast_GLOBAL 18.05 s [18.05 s, 18.05 s] 2.644 s (17.2%)
profiling 15.366 s [15.366 s, 15.366 s] -40.0 ms (-0.3%)
tracing 15.087 s [15.087 s, 15.087 s] -319.0 ms (-2.1%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~534d85ccf8, baseline=1.62.0-SNAPSHOT~63343baba3
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.481 ms) : 1470, 1493
.   : milestone, 1481,
appsec (3.835 ms) : 3611, 4058
.   : milestone, 3835,
iast (2.278 ms) : 2207, 2348
.   : milestone, 2278,
iast_GLOBAL (2.318 ms) : 2247, 2388
.   : milestone, 2318,
profiling (2.518 ms) : 2301, 2735
.   : milestone, 2518,
tracing (2.078 ms) : 2024, 2132
.   : milestone, 2078,
section candidate
no_agent (1.482 ms) : 1470, 1493
.   : milestone, 1482,
appsec (3.813 ms) : 3591, 4035
.   : milestone, 3813,
iast (2.28 ms) : 2210, 2350
.   : milestone, 2280,
iast_GLOBAL (2.324 ms) : 2253, 2395
.   : milestone, 2324,
profiling (2.099 ms) : 2044, 2155
.   : milestone, 2099,
tracing (2.076 ms) : 2022, 2131
.   : milestone, 2076,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.481 ms [1.47 ms, 1.493 ms] -
appsec 3.835 ms [3.611 ms, 4.058 ms] 2.353 ms (158.9%)
iast 2.278 ms [2.207 ms, 2.348 ms] 796.408 µs (53.8%)
iast_GLOBAL 2.318 ms [2.247 ms, 2.388 ms] 836.573 µs (56.5%)
profiling 2.518 ms [2.301 ms, 2.735 ms] 1.036 ms (70.0%)
tracing 2.078 ms [2.024 ms, 2.132 ms] 596.7 µs (40.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.482 ms [1.47 ms, 1.493 ms] -
appsec 3.813 ms [3.591 ms, 4.035 ms] 2.331 ms (157.3%)
iast 2.28 ms [2.21 ms, 2.35 ms] 798.485 µs (53.9%)
iast_GLOBAL 2.324 ms [2.253 ms, 2.395 ms] 842.401 µs (56.8%)
profiling 2.099 ms [2.044 ms, 2.155 ms] 617.41 µs (41.7%)
tracing 2.076 ms [2.022 ms, 2.131 ms] 594.609 µs (40.1%)

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Apr 21, 2026 •
edited
Loading

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master ygree/fix-flaky-kafka-batch
git_commit_date 1776784552 1776789337
git_commit_sha 63343ba 534d85c
See matching parameters
Baseline Candidate
ci_job_date 1776790619 1776790619
ci_job_id 1616383145 1616383145
ci_pipeline_id 108843453 108843453
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 2 metrics, 0 unstable metrics.

scenario Δ mean throughput
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume worse
[-7365.085op/s; -3954.126op/s] or [-3.753%; -2.015%]
See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume unsure
[-8183.638op/s; -172.331op/s] or [-2.680%; -0.056%]

@ygree ygree added the inst: kafka Kafka instrumentation label Apr 21, 2026
@ygree ygree force-pushed the ygree/fix-flaky-kafka-batch branch from 4688370 to 3fc43ca Compare April 21, 2026 16:18
@ygree ygree force-pushed the ygree/fix-flaky-kafka-batch branch from 3fc43ca to 534d85c Compare April 21, 2026 16:36
ygree
ygree commented Apr 21, 2026
View reviewed changes
Comment thread ...va-agent/instrumentation/kafka/kafka-clients-0.11/src/test/groovy/KafkaClientTestBase.groovy
consumerRecords.each {
records.add(it)
}
@Override
Copy link
Copy Markdown
Contributor Author

@ygree ygree Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reformats here and below are probably due to a spotless trigger on commit. I'm not sure what has changed.

@ygree ygree changed the title Fix flaky KafkaClientSplitByDestinationForkedTest batch consume Fix flaky test spring kafka template produce and batch consume Apr 21, 2026
@ygree ygree marked this pull request as ready for review April 21, 2026 17:34
@ygree ygree requested review from a team as code owners April 21, 2026 17:34
@ygree ygree mentioned this pull request Apr 21, 2026
Closed
@ygree
Copy link
Copy Markdown
Contributor Author

ygree commented Apr 21, 2026

/merge

@gh-worker-devflow-routing-ef8351
Copy link
Copy Markdown

gh-worker-devflow-routing-ef8351 Bot commented Apr 21, 2026 •
edited
Loading

View all feedbacks in Devflow UI.

2026-04-21 21:27:15 UTC ℹ️ Start processing command /merge

2026-04-21 21:27:20 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 2h (p90).

2026-04-21 23:13:59 UTC ℹ️ MergeQueue: This merge request was merged

@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot merged commit f2b26be into master Apr 21, 2026
592 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot deleted the ygree/fix-flaky-kafka-batch branch April 21, 2026 23:13
@github-actions github-actions Bot added this to the 1.62.0 milestone Apr 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AlexeyKuznetsov-DD AlexeyKuznetsov-DD AlexeyKuznetsov-DD approved these changes

Assignees

@ygree ygree

Labels

inst: kafka Kafka instrumentation tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: bug Bug report and fix

Projects

None yet

Milestone

1.62.0

Development

Successfully merging this pull request may close these issues.

2 participants

@ygree @AlexeyKuznetsov-DD