)]}'
{
  "commit": "fd39ed6deeed50104b5262d8228f5ae3ceab7bfd",
  "tree": "fed1073d1d11890f557892423a5a99c1a1541802",
  "parents": [
    "7390f8a82653490b3313623c43fe95025b32ea78"
  ],
  "author": {
    "name": "Sam Rawlins",
    "email": "srawlins@google.com",
    "time": "Wed Sep 23 10:08:22 2026 -0700"
  },
  "committer": {
    "name": "GitHub",
    "email": "noreply@github.com",
    "time": "Wed Sep 23 10:08:22 2026 -0700"
  },
  "message": "Fix flakiness in the eval integration test (#10008)\n\nI looked at three different logs of devtools tests flaking out on Linux bots on LUCI,\n\nAll three logs failed in eval_integration_test.dart:\n\n* Log 1 \u0026 Log 2: EvalOnDartLibrary asyncEval supports expressions that do not start with the await keyword timed out after 1 minute (TimeoutException: Test timed out after 1 minutes).\n* Log 3: EvalOnDartLibrary asyncEval returns the result of the future completion timed out after 1 minute.\n* Cascading failures: On all three runs, following the timeout, the test runner attempted retries (due to retry: 3), but every retry and subsequent test failed with evaluate: (-32000) Service connection disposed.\n\n1. Garbage Collection of reader before evaluation began:\nIn eval_on_dart_library.dart:390-490, the reader list was previously created in a standalone eval (widgetInspectorService.toId(\u003cdynamic\u003e[], \"$readerGroup\")). Because WidgetInspectorService only holds objects using\nWeakReference, nothing held a strong reference to \u003cdynamic\u003e[] in the target isolate during the network round-trip between DevTools and the target app. If garbage collection occurred in that window, widgetInspectorService.\ntoObject in the subsequent eval returned null, throwing:\n  Unhandled exception: type \u0027Null\u0027 is not a subtype of type \u0027List\u003cdynamic\u003e\u0027 in type cast\nBecause this exception occurred before the try/finally block inside the closure, postEvent(\"future_completed\", ...) was never posted, causing DevTools to hang waiting on future_completed until the test timed out after 1\nminute.\n2. Pinning loop stopped prematurely:\nThe pinning loop (while (!isDone \u0026\u0026 ++bufferTicks \u003c\u003d 20)) intended to pin reader until retrieved. However:\n   * As soon as isDone was set to true, !isDone became false and the loop terminated immediately (providing zero buffer while postEvent traveled to DevTools and DevTools issued evalInstance). If a GC occurred during that\n    window, toObject returned null.\n   * For evaluations taking longer than 1 second (20 ticks of 50ms), ++bufferTicks \u003c\u003d 20 became false while the future was still pending, causing reader to be unpinned before completion.\n3. Unhandled error in target isolate:\nAny exception occurring during reader initialization or eval setup in the target app closure was unhandled, preventing future_completed from ever firing and causing the target isolate to crash/disconnect.\n4. Stale environment reuse on connection drop:\nIn flutter_test_environment.dart:100-125, _needsSetup was not re-evaluated if the VM service connection dropped (!connectedState.value.connected). When a test timed out and the process disconnected, retries attempted to\nreuse the disposed connection, resulting in evaluate: (-32000) Service connection disposed.\n\n1. eval_on_dart_library.dart\n   * Allocated `final reader \u003d \u003cdynamic\u003e[];` directly inside the evaluated async function and registered it with `widgetInspectorService.toId(reader, \"$readerGroup\") as String`, eliminating the preliminary eval and ensuring\n    reader is strongly referenced from the moment of allocation.\n   * Transmitted reader_id (and any initialization error) directly in postEvent(\"future_completed\", ...).\n   * Wrapped the entire async function in an outer try/catch that reports errors back via postEvent rather than hanging DevTools.\n   * Kept reader strongly pinned in the target isolate in the finally block by awaiting in a loop and accessing reader.length until evalInstance calls disposeGroup (or up to 10 seconds timeout).\n2. flutter_test_environment.dart\n   * Added !serviceConnection.serviceManager.connectedState.value.connected to setupEnvironment\u0027s re-initialization condition so that if the VM service connection is disposed or dropped, the test environment is\n    automatically re-created for subsequent tests/retries.\n3. eval_integration_test.dart\n   * Added an explicit test survives garbage collection while the future is pending that triggers full garbage collections in the target isolate via getAllocationProfile(..., gc: true) while the future is pending.\n   * Removed tags: skipForCustomerTestsTag and retry: 3 now that the flakiness is resolved.\n\n• Ran dart analyze on modified files: No issues found.\n• Ran dart format: All modified files formatted.\n• Ran 5 consecutive test suites of packages/devtools_app/test/shared/eval_integration_test.dart: All 5 runs passed (5/5 tests passing per run).",
  "tree_diff": [
    {
      "type": "modify",
      "old_id": "3dea8867f24f3e296f844f88d488ed0cee5bacbf",
      "old_mode": 33188,
      "old_path": "packages/devtools_app/test/shared/eval_integration_test.dart",
      "new_id": "ccc6c7caa7984edc238b4eff764e2d0964111537",
      "new_mode": 33188,
      "new_path": "packages/devtools_app/test/shared/eval_integration_test.dart"
    },
    {
      "type": "modify",
      "old_id": "d85b3281dd21af290d033ede7b03cdd9d669131b",
      "old_mode": 33188,
      "old_path": "packages/devtools_app/test/test_infra/flutter_test_environment.dart",
      "new_id": "71906b77fb13dbb63170347b43f3c99d2d5df6a9",
      "new_mode": 33188,
      "new_path": "packages/devtools_app/test/test_infra/flutter_test_environment.dart"
    },
    {
      "type": "modify",
      "old_id": "9115a73747c6ba8208749ea55346a176df7f78fa",
      "old_mode": 33188,
      "old_path": "packages/devtools_app_shared/CHANGELOG.md",
      "new_id": "49654528020fb62645c827fb2b685b53af3097bf",
      "new_mode": 33188,
      "new_path": "packages/devtools_app_shared/CHANGELOG.md"
    },
    {
      "type": "modify",
      "old_id": "8df2f92135359769ec273c21f9db3870442279b1",
      "old_mode": 33188,
      "old_path": "packages/devtools_app_shared/lib/src/service/eval_on_dart_library.dart",
      "new_id": "e831211384c8424f45bfb1242a0e0ad314c02f0f",
      "new_mode": 33188,
      "new_path": "packages/devtools_app_shared/lib/src/service/eval_on_dart_library.dart"
    }
  ]
}