commit 6be50a489f8471d962fa3b5a6048085adb1e980b
author Jens Johansen <jensj@google.com> Wed Feb 19 23:12:17 2025 -0800
committer Commit Queue <dart-scoped@luci-project-accounts.iam.gserviceaccount.com> Wed Feb 19 23:12:17 2025 -0800
tree b026996f5fa1855a0678ad9d8229255515d1c119
parent 3702cbe573ea2b64f9b6a5dff116f53e431e0791

[analyzer] Fix signature calculation when merging yaml files with "error" keys

**TL;DR**
In certain situations a new signature was created for every
analyzer start causing more work than needed to be done and filling up
the cache for no reason. The fix can cause the startup cost to improve
significantly.

**Details**
When merging analysis options yaml files (when one file includes
another), and both include data under "analyzer: errors:" we get
inconsistent signatures because of ordering issues. This causes "old
work" to be redone and more files to be written.

This CL fixes the issue by not introducing a semi-random order when
merging.

For the contexts I have open, the following files gives (gave) a bad
(inconsistent) signature:
pkg/analyzer/test/analysis_options.yaml
pkg/front_end/analysis_options.yaml
pkg/analysis_server/test/analysis_options.yaml

Looking at the time it takes from the first read/write until the last:
Before:
37.148419
37.273016
36.807384

Now:
14.164655
15.251373
15.108677

While the difference is easily visible, for good measure:

```
Difference at 95.0% confidence
        -22.2347 +/- 1.0223
        -59.9702% +/- 2.7573%
        (Student's t, pooled s = 0.45103)
```

And from the column "time" in top (note that I'm running from source so
that this includes the initial dart compile of the analyzer):
Before:
1:10.84
1:09.81
1:26.98

Now:
0:38.03
0:38.72
0:37.79

Again the difference is easily visible, but for good measure:

```
Difference at 95.0% confidence
        -37.6967 +/- 15.4529
        -49.6815% +/- 20.3658%
        (Student's t, pooled s = 6.81767)
```

If clearing the cache and opening (waiting for writes to complete),
closing and repeating, before this CL I got this with the contexts I
have open:

After 1st open:
```
$ du -chs ~/.dartServer/
186M    total
```

After the 2nd open:
```
$ du -chs ~/.dartServer/
212M    total
```

 => +26MB

After the 3rd open:
```
$ du -chs ~/.dartServer/
247M    total
$ find ~/.dartServer/ | wc -l
25713
```

 => +35MB

After the 4th open:
```
$ du -chs ~/.dartServer/
272M    total
$ find ~/.dartServer/ | wc -l
27769
```

 => +25MB; +2056 files

After the 5th open:
```
$ du -chs ~/.dartServer/
297M    total
$ find ~/.dartServer/ | wc -l
29825
```

 => +25MB; +2056 files

After the 6th open:
```
$ du -chs ~/.dartServer/
321M    total
$ find ~/.dartServer/ | wc -l
31881
```

 => +24MB; +2056 files

And we seem to have reached a point where we can guess the 7th if we want to.

And with the fix in this CL:

After 1st open:
```
$ du -chs ~/.dartServer/
186M    total
$ find ~/.dartServer/ | wc -l
20816
```

After 2nd:
```
$ du -chs ~/.dartServer/
186M    total
$ find ~/.dartServer/ | wc -l
20816
```

After the 3rd:
```
$ du -chs ~/.dartServer/
186M    total
$ find ~/.dartServer/ | wc -l
20816
```

And it seems that it's stable.

We might also want to sort the strings as ordering (probably?) doesn't
matter, but I'll leave that for another CL.

Change-Id: Id06ca84e8b063bb31a261753f56620443f341a2e
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/410700
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Konstantin Shcheglov <scheglov@google.com>