Need some advice on confidence issues...?

I am a very pive kind of guy and I was wondering if anybody could give me some words of wisdom as to why I am not as "strong" as other people. I don't seem to have the same intensity as those around me. One of my issues is that I am always afraid of how people will react to me if I do take on the aggressor . One thing I constantly do is enact confrontational situations in my mind where I am the aggressor or the one who is more vocal but that is far from what it is in real life. I was just hoping to get some advice from those who faced this kind of problem in their younger years and if they realized anything that made them less inhibited about their actions.