He was bitter about it because it was a slap in the face, pure disrespect. Robert only gave Stannis Dragonstone so he had an excuse to deny him Storm's End by giving it to Renly. Storm's End should've gone to Stannis as Robert's eldest brother. Storm's End was by far the superior land to inherit, with more people and hugely better income, not to mention that it had been the seat of their house for centuries. Robert only gave him Dragonstone instead because he was pissed that Stannis "let" Viserys & Daenerys get away. Robert had ordered him to take Dragonstone (which had been held by Rhaella & supporters for about 9 months following the fall of King's Landing) but Ser Willem Darry spirited them away to Braavos before Stannis even got there. There was literally nothing he could have done about it.

"I never asked for Dragonstone. I never wanted it."

- Stannis, ACoC Prologue