This is a quick one, because I’m writing in advance of a trip to New England, but for me, the best secondary source I’ve read that helped me understand the people I was clothing is John Styles’ The Dress of the People. That’s not to say that I haven’t read more, and found period resources and collections of resources equally useful. If you’re doing a poor woman in America, Don Hagist is your man, with Wenches, Wives and Serving Girls (now Wives, Slaves, and Servant Girls. Read Good Wives, Nasty Wenches, and Uneasy Patriarchs for more on period terminology).
I like Styles because he helps us understand the why of people’s clothing, and their wants. For me, context is key (I harp on this a lot) so insight into how many shifts are usual, the fashion for pocket watches, and the activity of the second hand clothes market is really helpful. So despite my love of shiny satin gowns and fashion of all eras, among the few books I didn’t send to storage when I moved was The Dress of the People. I think that’s a strong recommendation.