-location on tree would limits builds

sure, but if you want an ancestral call, changes are extremly high youre either a marauder, templar or duelist. ranger can make her way there too. this is not a big problem.

-counter to design philosophy

if their design philosophy is to have us gem swap, then this is a pro.

at that point why not make it an innate property of melee/mandatory keystone for nearly all melee

for the same reason we dont give all melee skills splash, leech etc.

very few changes exist in a vacuum. they would obviously have change other things as well, but it would be for the greater good because mechanics like gem swapping are not healthy for the game; its poor design that hinders a smoother gameplay experience.