Non-IT person in a somewhat formal city (DC), and my work clothes are still pretty much my casual clothes.

I keep a pretty small closet and try to make sure there's as much overlap between my work clothes and casual clothes as possible without feeling out-of-place in either scenario. For example, I have one pair of black skinny jeans and one pair of straight-leg blue jeans that I can wear in either scenario. If I'm at work, I'll pair that with a cashmere sweater and ankle boots. If I'm not at work, maybe I'll pair it with a cotton crop top and sneakers. My "work apparel" (thanks to a casual dress code) is basically some combo of pants and basic sweaters, with a wool dress that occasionally makes an appearance.

I shop a whooole lotta Uniqlo, Topshop for jeans, and Everlane for a few of the sweaters. Rather than a work/casual divide in my budget, I just spend the entirety of it on whatever I feel is the most versatile while also fitting into my #aesthetic.