When you say 'this kind of stuff' do you mean exchange encrypted messages with people using public key encryption software like gpg? If so, I think it depends a lot on how (secure vs. convenient) you want to be, and a little on what OS you're running. Since you can generate as many key pairs as you want, I'd just play around with it at first, getting familiar with the tools and not worrying too much about ultrasecurity. Once you are, or if you are already familiar with key rings and how to add and list and delete and encrypt and sign and stuff, then if you wanted to try bumping the security up a notch you could designate an old laptop that is never connected to anything - no wifi, no ethernet, and definitely no autorun - as the encrypt/sign/decrypt machine and use a thumbdrive to move encrypted messages to it and from it. That way, even if all of the machines you've got connected to networks got riddled with trojans and keyloggers, none of them would ever have access to your private key or see the keystrokes you used to create messages, encrypt, decrypt, sign, etc.

That can get old for stuff that doesn't really need that level of security though. I don't do it much anymore, but my real-world use of point-to-point encryption wasn't very defensive against already-infiltrated machines; there was an IT dept watching the network and I ran client-side firewalls and tripwire on my machines to keep an eye out for that. It generally involved billing systems and needing to send or receive big lists of customer data. I'd typically generate new keys every time I needed them, swap them with someone in the clear, then voice-verify hashes on a phone call that had to take place anyway to talk about other data processing stuff. It wasn't all that high tech, but if someone wanted to attack something like that, they'd have to be waiting for it, pull network shenanigans, substitute keys, intercept a phone call, do good voice impersonations, and carry off a convincing conversation about the rest of the job... or take the easier route and just do something that I didn't consider, which is the real danger when it comes to this stuff and why it's a bad idea to assume or try to invent anything related to crypto.