So If I was as far away from our sun as pluto, would I still not freeze almost immediately?

Not really, because your heat has nothing to transfer to. If you were in Pluto's atmosphere (does it have one?) then you would because your heat have something to transfer to.

The only difference is that you wouldn't get any MORE heat energy from the sun...so yes you would get colder faster, but you aren't going to flash freeze or anything.

You know how when you put your warm hand on a icy window in winter and your hand heats up that window? That is because the window is pulling heat from your hand. In space, there is nothing to really pull heat from you, so the only heat you lose is the tiny bit of heat you emit as radiant energy, which is a lot less than you'd think (since the air isn't there to pull additional heat from you).