First of all, making small talk is challenging also for native speakers. This is so because small talk can practically mean talking about anything - and variable topics entail understanding and use of quite specialist words.
Anybody may feel a bit apprehensive when a question is popped to him or her out of the blue. Moreover, fear of verbal incompetence can lead to “blocking”. They slow down or stop speaking completely because of a lack of self-confidence