Skip to content

Commit 9ac65c9

Browse files
committed
ARM64: Force acquire semantics on pointer loads to fix weak memory ordering
On ARM64's weakly-ordered memory model, pointer loads (LoadP/LoadN) can be reordered with subsequent dependent loads by the hardware. This causes failures in concurrent data structures used by virtual thread scheduling (SynchronousQueue, LinkedTransferQueue) where a loaded pointer is immediately dereferenced but the dependent field load sees stale data. Force MemNode::acquire at IR creation time in LoadNode::make() for all LoadP/LoadN nodes on AARCH64 when barrier_data == 0. This must happen at IR creation rather than instruction selection because C2's optimizer can eliminate or reorder loads between those phases — a load removed by LoadNode::Identity() or hoisted by Ideal() will never reach the matcher to receive a barrier. Loads with barrier_data != 0 (GC reference barriers) are excluded since GC barrier patterns emit their own ordering sequences and have no acquire variants in the .ad file. Also reverts needs_acquiring_load() to the upstream form (is_acquire() only), since the opcode-based check is now redundant — all pointer loads already have acquire set on the IR node.
1 parent b8d5fca commit 9ac65c9

1 file changed

Lines changed: 5 additions & 4 deletions

File tree

src/hotspot/share/opto/memnode.cpp

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -971,8 +971,9 @@ Node* LoadNode::make(PhaseGVN& gvn, Node* ctl, Node* mem, Node* adr, const TypeP
971971
case T_FLOAT: load = new LoadFNode (ctl, mem, adr, adr_type, rt, mo, control_dependency); break;
972972
case T_DOUBLE: load = new LoadDNode (ctl, mem, adr, adr_type, rt, mo, control_dependency, require_atomic_access); break;
973973
#ifdef AARCH64
974-
// On ARM64, use acquire semantics for all pointer loads to ensure proper ordering
975-
case T_ADDRESS: load = new LoadPNode (ctl, mem, adr, adr_type, rt->is_ptr(), MemNode::acquire, control_dependency); break;
974+
// On ARM64, use acquire semantics for pointer loads to ensure proper ordering.
975+
// Only when barrier_data == 0; GC barrier patterns handle their own ordering.
976+
case T_ADDRESS: load = new LoadPNode (ctl, mem, adr, adr_type, rt->is_ptr(), barrier_data == 0 ? MemNode::acquire : mo, control_dependency); break;
976977
#else
977978
case T_ADDRESS: load = new LoadPNode (ctl, mem, adr, adr_type, rt->is_ptr(), mo, control_dependency); break;
978979
#endif
@@ -981,7 +982,7 @@ Node* LoadNode::make(PhaseGVN& gvn, Node* ctl, Node* mem, Node* adr, const TypeP
981982
#ifdef _LP64
982983
if (adr->bottom_type()->is_ptr_to_narrowoop()) {
983984
#ifdef AARCH64
984-
load = new LoadNNode(ctl, mem, adr, adr_type, rt->make_narrowoop(), MemNode::acquire, control_dependency);
985+
load = new LoadNNode(ctl, mem, adr, adr_type, rt->make_narrowoop(), barrier_data == 0 ? MemNode::acquire : mo, control_dependency);
985986
#else
986987
load = new LoadNNode(ctl, mem, adr, adr_type, rt->make_narrowoop(), mo, control_dependency);
987988
#endif
@@ -990,7 +991,7 @@ Node* LoadNode::make(PhaseGVN& gvn, Node* ctl, Node* mem, Node* adr, const TypeP
990991
{
991992
assert(!adr->bottom_type()->is_ptr_to_narrowoop() && !adr->bottom_type()->is_ptr_to_narrowklass(), "should have got back a narrow oop");
992993
#ifdef AARCH64
993-
load = new LoadPNode(ctl, mem, adr, adr_type, rt->is_ptr(), MemNode::acquire, control_dependency);
994+
load = new LoadPNode(ctl, mem, adr, adr_type, rt->is_ptr(), barrier_data == 0 ? MemNode::acquire : mo, control_dependency);
994995
#else
995996
load = new LoadPNode(ctl, mem, adr, adr_type, rt->is_ptr(), mo, control_dependency);
996997
#endif

0 commit comments

Comments
 (0)