It really works! A reliably fashion-forward style is to pair the t-shirt under a sweater, with only the collar poking out, keeping you warm while also adding a rich layer to the look.
If your t-shirts typically play a supporting role, standing as a clean background for more complex garments, or hidden all together as a base layer, then budget-friendly options will do the trick.