Agree.dalto wrote:There is no reason why you cannot carry an XD 9mm. I know several people that do.
A good IWB will easily conceal that gun.
You describe yourself as 5'9" and 180 lb. You should have no trouble concealing your XD. Many of us successfully carry and conceal full sized pistols from all sorts of manufacturers. The secret (for all sizes of guns) is that you'll have to make some adjustments to the way you dress. This has been true for most of us. But that said, a compact version of your XD will conceal quite easily. Ditto for compact Glocks, M&Ps, 1911s, etc., etc. Mine are a 1911 and an M&P.
If pocket carry is your game, then either the LCP or the P3AT Keltech would serve you well, although .380 is sort of a minimum for self-defense. Just as popular in pocket pistols, very nearly as easy to conceal as the LCP Et al, are the J-frame sized alloy framed revolvers. There are a number of fine examples by various manufacturers, including the Ruger LCR. .38 Special +P is a better choice for self defense than .380. Personally, I pocket carry a scandium framed S&W .357 magnum. Lots of horsepower in a very light and easily concealable package.