Have a buddy who has a CCW take a look at you, walk around a store with you, things like that, and comment on what he sees. Once it's hard for someone who has a trained eye to know (but maybe not impossible) then typical folks will never notice.
It probably won't be printing that gets you, since most holsters are pretty darn good about covering that anymore. More than likely while walking, your arms aren't going to swing correctly. The one covering your gun will "hover" for a while. You're going to try to "protect" the gun subconsciously. You're going to try to adjust everything constantly.
So wear your empty holster all the time now. Concentrate on walking as naturally as possible. Just relax, and wear it all the time while at home. Take out the trash, get things off of high shelves, bend over a lot, and just get used to moving naturally and not acting like you have a weird hip disorder, and you'll be fine.