commit a98011a5ef726d08d1bc14ee19e1fc64c4266254
author Jens Johansen <jensj@google.com> Thu Mar 06 23:43:58 2025 -0800
committer Commit Queue <dart-scoped@luci-project-accounts.iam.gserviceaccount.com> Thu Mar 06 23:43:58 2025 -0800
tree 5d2c948a22557794856b920d6bce6aa9a6840d26
parent 219471cf2ed9f1c6419d9b780355480321aeb1f0

[analyzer] Initial LSP server e2e benchmark

A benchmark that launches the dart analyzer server in language server
protocol mode and communicates with is as an IDE.
In this case the benchmark generates between 16 and 1024 copies of the
abstract scanner (to get a large amount of code) as well as imports and
exports all the files in cycles and/or chains; performs an edit,
requests completion and times initial startup (with no cache),
completion after change and when it's done analyzing after the change.

It does this in several modes than change the way the files are imported
and exported:
* ImportCycle where file1 imports file2 etc and the last file imports
  file1. There are no exports.
* ImportChain where file1 imports file2 etc and the last file doesn't
  import anything. There are no exports.
* ImportExportCycle where file1 imports and exports file2 etc and the
  last file imports and exports file1.
* ImportExportChain where file1 imports and exports file2 etc and the
  last file doesn't import or export anything.
* ImportCycleExportChain where file1 imports and exports file2 etc and
  the last file imports file1 but doesn't export anything.

For ImportCycle, ImportChain and ImportExportChain things appear to
scale ~linear and - using AOT - have timeings in this ballpark (this is
specifically for ImportCycle):

+------+-----------+------------+------------+
| Size |  Initial  | Completion | Fully done |
+------+-----------+------------+------------+
|   16 |   0.46561 |   0.158765 |    0.40474 |
|   32 |  0.901167 |   0.268819 |   0.859874 |
|   64 |  1.657207 |   0.428747 |   1.488365 |
|  128 |  3.178606 |   0.843576 |   3.040237 |
|  256 |  6.015557 |   1.737661 |   6.010487 |
|  512 |  12.08567 |   2.979242 |  11.736878 |
| 1024 | 24.273368 |   6.101671 |  24.018495 |
+------+-----------+------------+------------+

For ImportExportCycle and ImportCycleExportChain it scales worse and
e.g. ImportExportCycle looks like this:

+------+-----------+------------+------------+
| Size |  Initial  | Completion | Fully done |
+------+-----------+------------+------------+
|   16 |   0.46673 |   0.169486 |   0.406448 |
|   32 |  0.875871 |   0.242876 |    0.85543 |
|   64 |  1.583077 |   0.465915 |   1.506953 |
|  128 |  3.198071 |   0.903894 |    3.09165 |
|  256 |  6.786677 |   2.149489 |   6.779569 |
|  512 | 17.346131 |    8.92149 |  17.971033 |
| 1024 | 63.358453 |  46.152089 |  65.401559 |
+------+-----------+------------+------------+

(In the tables 'Completion' is time until completion answers after a
top-level change and 'Fully done' is time until the analyzer stops
analyzing after a top-level change).

Change-Id: Id7214c0d6c14199f39c0c8a6a8b4941a0e575dc3
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/413401
Reviewed-by: Brian Wilkerson <brianwilkerson@google.com>
Commit-Queue: Jens Johansen <jensj@google.com>