I pretty much have discovered exactly what you have. I wear IWB at 3:30 - 4:00 and nobody has ever noticed - ever! I only wear untucked T-Shirts and Collared pull over shirts. Ant my shirts are not too loose fitting - just enough. I carry a Glock 27 with a spare mag both with Pearce grip extensions or sometimes a Kahr P45.
It's blistering here in Houston and simple T's and pull-overs are all I'm prepared to wear! In fact, anytime I see anybody in jeans that is sporting some sort of vest I immediately suspect they are CHL.
Last Sunday at family lunch my nephew triggered an impromptu gun "show and tell" showing off his new Walther PPK. My brother followed with his XD40 and when I reached back and produced a Glock 27 out of nowhere and cleared it my 16 year-old daughter leaned over to my wife and said "Did he just pull a gun out of his pocket? Does he always carry that around!?!?!" I have been toting religiously for 8 months now and she has NEVER noticed. That was the best Litmus test for me ever: if someone that close to me who sees me everyday has never noticed the nobody likely will.
As someone here on this board said and says "everyone walks around in their own little world" which I now understand to be completely true.