Just to help clarify it to jekyll001.
We obviously prefer to 5bet with a range of hands that give us maximum FE.
The more you have blockers to premium in his hands -> the more he fold -> the more you have FE.
I'd look a wider calling range that consists TT+,AQ+. Mathematically we are looking at:
6 * 5 combos of pp + 2 * 16 combos of AK and AQ = 62 combos
1.with Ax where x < T we block 3 combos of AA and 8 combos of AK and AQ =
(total 11 combos blocked) villain calling range reduces to 51.
2.with Ay where y = T,J we also block 3 combos of yy =
(total 14 combos blocked) villain calling range reduces to 48.
3.with KQ you block 6 combos of QQ and KK and 8 combos of AK and AQ =
(total 14 combos blocked) villain calling range reduces to 48.
4. with a suited connector we block 0 combos.
5. with a PP we block 0 combos.
Lets look at the equities of these hands:
1. A6o 25.9% , A6s 29.9%
2a. ATo 26.2% , ATs 30.2%
2b. AJo 27.7% , AJs 31.7%
3. KQo 27.5% , KQs 31.4%
4. 87s 31.205%
5 77 37%
conclusion:
The maximum FE is achieved with KQ and AT,AJ which reduces villain calling range by 22% (14 combos out of 62).
The maximum equity is achieved with a PP.
now lets do an equity compare between 5bet jamming 1(Ax) and 5(pp)
the general formula is:
fe*37bb + (1-fe)* (equity*200bb - 90bb)
If we put the numbers both in both formulas we give us positive results.
Now:
1. we gain extra 22% * 37bb ~= 8bb for the extra fe.
5. we gain extra 7% equity from 200bb = 14bb.
In order to determine which range has more ev we need to know what's our fe. this fe is actually the range of hands that villain 4bet bluff with them.
so in order to decide when ev of 1 is > ev of 5 we need to solve:
for 1 -> equity * 200bb -90bb = -30bb
for 5 -> equity * 200bb - 90bb = -16bb
8bb + (1-fe) * (-30bb) > (1-fe)* (-16bb)
1 > 5 <-> fe > 0.42 , that means that we need villain to 4bet bluff with a 42% frequency to make 1 > 5.
Now that brings us to the question of whether do we even 3bet with those hands at the first place, I'd leave that question open to anther discussion.
edit: correction of the general formulas.