A common mistake of people who obtain English fluency with improper pronunciation and intonation is the speed in which they speak. Because they do not have perfect pronunciation and intonation, it is difficult to understand people who speak too fast. This is the biggest problem I have seen from people who gain confidence in speaking.
In order to improve your communication, and to hear "excuse me" less often, it is important to have the correct speed of speaking. For people with confidence, my advice is to slow down and to speak clearly. Don't blend words together too much and make sure to separate words with a small pause for words that are difficult to pronounce.