tree 77ffb814ca6796c47805194b3956c9c2c432ab39
parent 9ff5ac12c6efec99b8d6580c7a7154bb5e265add
author Vyacheslav Egorov <vegorov@google.com> 1620042428 +0000
committer commit-bot@chromium.org <commit-bot@chromium.org> 1620042428 +0000

[vm] Fix race when setting identity hash codes.

We might interrupt Object._objectHashCode(obj) helper to serve a
vm-service request between checking for unset hash code and setting
a new hash code. While serving this request we might set the hash
code of [obj] (e.g. when generating heap snapshot or in
Instance::PrintJSONImpl). Later Object._objectHashCode will call
Object_setHash, which (on 64-bit platforms) assumes that
hash field in the object header is set to 0 and uses bitwise-or
to initialize it. This leads to a mismatch between hash code that
the first invocation of _objectHashCode will return and the
value stored in the header (because we OR non-zero value set
by vm-service with a value which _objectHashCode intended to
use as a hash).

This CL changes Object_setHash to avoid overwriting or mangling
the hash value if it was already set.

We also fix hash code generation in vm-service to ensure that we
only generate values which are valid Smis because the rest of the
code expects that.

TEST=pkg/front_end/test/incremental_compiler_leak_test.dart

Cq-Include-Trybots: luci.dart.try:vm-kernel-linux-release-simarm64-try,vm-kernel-precomp-linux-release-simarm64-try
Change-Id: Ica913af8bc1cfef0ad60a9e7504531ee4de53015
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/197400
Commit-Queue: Vyacheslav Egorov <vegorov@google.com>
Reviewed-by: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Ben Konyi <bkonyi@google.com>