YUP! this is a likely reason. The membrane of the condom acts as a barrier to sperm as well as an insulation to sensations. Most men can still achieve sufficient stimulation to reach an orgasm, other like me rarely do.

What you can do is to lubricate only the Glans of your penis before rolling on the condom. Doing this will often cause sufficient slippage within the condom to transfer the friction required. In addition, I recommend waiting until you are at the brink of an orgasm yet not so close to the trigger point that you lose control before applying the condom. If you need additional stimulation, ask your partner to stroke the shaft just before entering.

Many young men nowadays operate under the misguided misconception that the best way to achieve an orgasm is thru lots and Lots and LOTS of stroking. WRONG. This is the reason for all that fooling around and then Foreplay, later. If necessary, try different positions in order to improve contact and friction.

Give all this a try and see if the situation improves.

Did you ever try doggy? I think guys can get off faster that way sometimes especially if they go up higher so they are hitting your inside front (if you get what I mean, its hard to describe).