Unai Emery's side lost 1-0 to newly promoted Sheffield United at Bramall Lane on Monday and Evra condemned Arsenal's performance.
"Sheffield United deserved the win, but I'm not surprised about Arsenal," Evra told Sky Sports.
"I used to call them 'my babies' 10 years ago, and they are still, when I look at them and think they are 'my babies,' that's the truth -- I'm not being disrespectful when I say that."
Xhaka responded to Evra after the game and questioned why former players were quick to criticise Arsenal.
"We have to stop speaking about mental [issues] or bull---- like this," he said. "For me it's the same if you play home or away. You have to win. You have to show big character. You have to show good games and not to find always the same excuse.
"A lot of people speak too much. I hear that the first time he [Evra] speaks something against us. I have a lot of respect about him, because he was a great player. But you have to be careful what you say in those situations as well.
"For me it's strange because they were in the same situation as us, they were players as well.
"Maybe sometimes it was good, sometimes it was not so good. But how I say if you speak every weekend, every game, bull---- like this they speak, they don't get respect about what they say.
"I think we did a lot of good things. I think Sheffield had one chance, it was the goal. That's it."
Arsenal dropped outside of the top four after suffering their second Premier League defeat this season.
Arsenal face Guimaraes at the Emirates on Thursday in the Europa League before hosting Crystal Palace on Sunday.