- ESPN+ guide: LaLiga, Bundesliga, MLS, FA Cup, more
- Don't have ESPN? Get instant access
- ESPN Insider Notebook: Pogba out, Tchouameni in?
The result sees United fall out of the top four and four points behind leaders Liverpool.
"We deserved to lose," United midfielder Paul Pogba told the BBC. "To be honest, we have been having these kind of games for a long time.
"We have not found the problem, conceding easy goals, stupid goals. We need to be more mature, play with more experience and arrogance in a good way. We need to find something, we need to change.
"We need to find the right mentality and the right tactic. Stick with it and resolve the problem."
Leicester went close to opening the scoring after five minutes but Harry Maguire cleared off the line to prevent Vardy from scoring into an empty net.
The hosts dominated the early stages but United took the lead after 19 minutes when Greenwood exchanged passes with Bruno Fernandes and fired a stunning strike into the top corner.
The game came to life with 12 minutes remaining when Soyuncu fired home from close range from a corner to put Leicester in the lead.
Four minutes later, substitute Rashford made it 2-2 when he latched onto Victor Lindelof's pass.
However, Leicester went back ahead seconds later after Vardy pounced onto Ayoze Perez's ball.
Daka sealed the win for Leicester in stoppage time with a header at the back post to increase pressure on Solskjaer.
"It was massively important," Vardy told the BBC. "The most important thing was the performance because we had let our standards slip. We had the international break to reflect on things."