My CTAC is by far the most used holster that I have. It isn't the most comfortable, but isn't bad either. The reason you experienced such discomfort is, as you said, because you didn't have your gun in it. You would think it would not make that much difference, but IT DOES!
There is a piece of Kydex, between your skin and the gun, that extends up to provide a barrier between the two. You shouldn't hold it against the CTAC because you won't be wearing the holster without the gun typically.
I always wear a "wife beater", (forgive the term, but that's what my son calls them and I don't have a more descriptive term of my own). This provides a considerable amount of comfort. Next comes the holster and then a dark shirt one size larger than I normally wear. A shirt like this covers a mulitude of sins should you bend over or do something to cause your gun to print.
I used to wear it just behind the hip, but now I wear it in either Appendix or left-side cross-draw positions. With a short barrel gun it's not too uncomfortable if you keep you pants pulled up when you sit down. This makes the gun much more accessable when you're in the car. Try drawing from behind the hip while sitting in a chair or car. Also, this is good for retention as well. Now, if you have a rather large belly or long barrel gun, this might not work well for you. For me it works perfect and forces me to keep that stomach flat too.
-ss