broadcastable

(1) align starting from the right, going to left
example:
D1 = torch.Size([27,27])
D2 = torch.Size([27])
# 27 27
#    27

(2) the dimension sizes must either be equal, one of them is 1, or one of them does not exist.

internally, it will do, create 1 here and will broadcast:
# 27 27
# 1  27

it is a bug
the issue comes from sliently adding the dimension here, which is 1
need to keepdims = True !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

###################### another trick
p = p /p,sum(1,keepdim=True)
--->
p /= p.sum(1,keepdim=True) # inplace operation, faster, less memory

################# how to normalize each row
probs = counts / counts.sum(1,keepdim=True)
each row summartion == 1


################### another way to think of softmax
logits = encoded_input * Weights
counts = logits.exp() # logits can be negative or positive. using exp() will make sure all positives with all original negatives converted to [0,1]
# so that we can normalize it and generate new numbers that can represent probabilities with sum of them == 1