590 hours in the game and counting here, and I think I am good enough now to give a bit of advice here. Feel free too correct me if I am wrong/stupid in something, I'm open to criticism.
For colonizing, always go for the coastal provinces first, no exceptions. The Carribean also has one of the best New World trade nodes (along with Cheeosake Bay) so I recommend grabbing the chain of 6 tax islands there if you're in it for trade. If you're going to the New World for tarrifs only, then I wouldn't worry too much about getting The Carribean super-early on.
Honestly, that's pretty much all for colonization. It's IMO the most boring and underdeveloped part of the game besides peacetime and I wouldn't recommend to anyone a colonization-focused nation for their first playthrough.
For conquest, if you're a small nation, do not, and I repeat DO NOT ever go over 100% overextension. Atleast until you feel you can handle a good amount of OE later on (like 1600s or something like that, don't go over 100% OE. Personally, if I play as Brandenburg or Byzantium, I like taking a bunch of vassals until the 1500s when falling behind Admin tech won't hurt as much as it would in 1450.
Also don't get shitloads of AE early on. I made that mistake as France in my early days and got coalitoned by half of Europe. Take enough land that you'll benifit a lot from the war, but not enough to make too many people outside your region. Example as Brandenburg: My initial conquests in the early 1450s against the Tuetons and their allies involve me taking my core Nuemark and full annexing Mecklenburg (whos only province I fabricate a claim on before I declare war). In this way, I gain around 10 basetax and in doing so, the coalition against me will be consisting only of a couple of your rivals and a couple OPMs. They'll never declare war on me though because of my Austrian and Polish alliances. Also something I forgot to mention is remember to ally the big powers in your general area at the beginning. Examples include: In South Germany/Italy: Ally France. In North/Central Germany and the Balkans: Ally Austria and Poland. In India: Ally the biggest nation you can get on your side that is your religion.
That's it for conquest until I decide to expand it/fix it.
http://www.eu4wiki.com/Combat_width#Combat_width
This is a wiki page I always have open when playing EU4. A lot of people like to use stacks of 18 (8 Infantry/Inf - 2 Cavalary/Cav - 8 Artillery/Art) or stacks of 20 (6 Inf - 4 Cav - 10 Art/9 Inf - 2 Cav - 9 Art). But I personally have found that ugrading how many units I have in my stacks each time combat width is increased to be the best way to create armies. I have completed a list of all the army comps to use depending on mil tech on paper, and I will post it here if you want OP.
For vassals, if you're feeding them and you feel like their big enough to be annexed (If you're feeding Persia I recommend giving them only their cores and nothing else unless you have completed Influence+Diplo ideas and have a statesman. If you're feeding Iraq, definitely give them more land than the cores they own.). If they're just a "I'm gonna vassalize you because I don't feel like paying the Admin to core your lands" vassal, then annex them right away, they probably wont be adding much too your armies anyway.
I will edit this tomorrow after I go to sleep. Also to everyone else that reads this, feel free to criticize any mistakes I made in grammar/game mechanics/everything.