I think the ability to understand has a bigger effect on ability to speak (respond) than most people realise. You have to be able to understand considerably better than to speak because what you are listening to is much more complex than what you are going to say. And if you are struggling to try to understand, or are worrying about how well you have understood, it makes responding much more difficult. To say it in another way, if you understood completely and effortlessly, it would make responding a lot easier.
Another problem is trying to compose what you are going to say while the other person is talking. That disturbs your listening and understanding, and makes responding even harder. I'm not good at it, but I think it is better to try to concentrate 100% on listening until the other person is finished. Then TAKE A PAUSE. Don't try to answer immediately. Take a moment to think and compose, and put your mind in gear. Don't blurt out something too quickly. Use short sentences. Use simple straightforward sentences. Don't get trapped in the middle of an overly complex sentence and an advanced way of saying something that is just too confusing. That ability will come in time