tree 52c0df2fbef71b55feb9672649c924a808949808
parent 1c7a7d0c41ea737130041d0f784213fca073db4d
author Vyacheslav Egorov <vegorov@google.com> 1657111696 +0000
committer Commit Bot <commit-bot@chromium.org> 1657111696 +0000

[vm] Avoid UB in FinalizeHash(hash)

FinalizeHash(hash) was trying to avoid UB in expression 1 << 32
by casting 1 to uintptr_t. This type however is not wide enough
on 32-bit platforms.

Instead just use explicit comparison hashbits < kBitsPerInt32 to
avoid overflow in left shift.

This bug went unnoticed for a while because it the only place
where we call FinalizeHash(hash) is in the snapshot profile
writer code and it only triggers when gen_snapshot is a
32-bit binary - which is only true on Windows, as Mac and Linux
seem to use simarm_x64 configuration instead.

This UB was explicitly affecting the code behavior because C++
compiler would either optimize out or change behavior of any
code that consumed value produced by FinalizeHash(hash).

Fixes https://github.com/flutter/flutter/issues/97764

TEST=vm/cc/DirectChainedHashMap

Change-Id: I39f2b09e7516c875b765e5a065d1c1331f89fa33
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/250741
Commit-Queue: Slava Egorov <vegorov@google.com>
Reviewed-by: Daco Harkes <dacoharkes@google.com>