Man Utd off to worst Premier League start after Chelsea draw
Interim boss Ruud van Nistelrooy, who will be replaced by Ruben Amorim later his month, sprinted down the touchline in celebration after Bruno Fernandes put United in front from the penalty spot 20 minutes from time.
Caicedo quickly replied for the Blues, who edge above Arsenal on goal difference into fourth place in the table.
A point leaves United still down in 13th, six points off the top four.
The Red Devils' return of just 12 points from 10 games is United's worst start to a league campaign since 1986/87.
Amorim was not in attendance as he prepares to take charge of Sporting Lisbon for the one of the final times against Manchester City in the Champions League on Tuesday.
United were far more competitive than in 3-0 demolitions by Liverpool and Tottenham at home this season, but showed many of the same flaws that cost Ten Hag his job.
Van Nistelrooy enjoyed a 5-2 demolition of an under-strength Leicester in his first match in charge to reach the League Cup quarter-finals in midweek.
Still a hero among the United fans from his days as a prolific striker, the Dutchman's name was chanted in a buoyant pre-match atmosphere.
However, a Chelsea side that have lost once in nine Premier League games, provided a much stiffer test for a United side still short on confidence.
Both sides hit the woodwork in a disappointing first 45 minutes as Noni Madueke hit the post from a corner before Marcus Rashford clipped the bar with a cushioned volley in the last action of the half.
Pedro Neto's burst of pace nearly gave Chelsea a dream start to the second period but the Portuguese winger's shot flew just past the far post.
However, the Blues remain overly reliant on Cole Palmer for inspiration.
The boyhood United fan, who began his career at Manchester City before joining Chelsea last year, was kept quiet by Van Nistelrooy's pairing of Casemiro alongside Manuel Ugarte in midfield.
'Results will come'
Before kick-off only Southampton had scored fewer than United's eight Premier League goals in nine games this season.
The lack of efficiency in front of goal that Ten Hag often bemoaned was in evidence again.
"Over the last games we played we lost some points where we didn't convert our chances," said Van Nistelrooy, who is set to take charge of two more games.
"When you create a lot, all through the season so far, I'm not too worried the players will get in a flow with results and confidence growing. Results will come."
Alejandro Garnacho wasted a glorious chance to open the scoring from Fernandes' cut-back with a tame effort that looped into the arms of Robert Sanchez.
United were gifted the chance to break the deadlock when Sanchez clipped Rasmus Hojlund inside the box.
Fernandes kept his cool to send the Spanish international goalkeeper the wrong way and spark wild celebrations from Van Nistelrooy.
United's joy was short-lived as Chelsea hit back within four minutes.
A corner was only cleared to the edge of the box, where Caicedo connected a sweet strike to fire into the bottom corner.
Caicedo cost a British transfer record £115 million ($149 million) when he joined Chelsea just over a year ago and is beginning to deliver on that investment after a difficult first season.
"Since we arrived Moi is doing fantastic," said Chelsea boss Enzo Maresca.
"The problem with Moi is because the club paid big money, people expect them to be the best. It is normal to be a bit up and down. We are trying to improve him and he is showing how good he is."
Chelsea should have struck again moments later when substitute Enzo Fernandez spooned over with Andre Onana stranded.
But it was United who came closest in a frantic finale when Garnacho's acrobatic effort flew just over.