Furious Medvedev knocked out by Khachanov in Monte Carlo
Medvedev was hit with a point penalty to start the final game after berating chair umpire Carlos Bernardes and then became embroiled in a heated discussion with the tournament supervisor.
The world number four had already been warned after hurling his racquet when he double-faulted to lose his service game and leave Khachanov a game away from victory.
Medvedev, who took issue with two line calls in his second-round win on Wednesday, felt aggrieved a Khachanov forehand was not called out during a rally that resulted in the latter earning two break points at 5-5.
The arrival of the physio to treat an apparent bleed at the ensuing changeover tipped Medvedev over the edge, with the Russian yelling "did I ask for the physio!" at the umpire.
The 28-year-old Russian then demanded of the supervisor "who will take responsibility?" for the decisions.
Television replays appeared to show the shot in question from Khachanov had landed wide of the court.
"It's second day in a row. Guys, open your eyes. Do something. It's out," raged Medvedev.
"The mark is out. They don't know how to referee anymore. Who will take action?
"Yesterday the ball is out, it's called in. Who will take action. This ball is out there. Who will take responsibility? It's not my responsibility to referee the matches."
Khachanov and Medvedev traded breaks in the first four games of the match before the former landed the crucial blow at 4-3 en route to taking the opening set.
The second set followed serve and Khachanov saved a set point down 5-4 before Medvedev imploded in the next game, losing both his serve and temper, and eventually the match.
Khachanov's reward is a quarter-final Friday against Alexander Zverev or Stefanos Tsitsipas.
"I think lately my results have been better and better," Khachanov, the 15th seed, said after his first top-five win of the season.
"Then to think how to beat the top guys and be at the top. These are the things that we have to keep in mind and keep working on. The win today gives extra confidence in my pocket to continue this tournament and to step into the clay swing."
Alex de Minaur moved into the last eight with a 6-3, 6-4 victory against fellow Australian Alexei Popyrin.
Grigor Dimitrov, last month's Miami Open runner-up, continued his good form with a 6-4, 6-3 win over Miomir Kecmanovic in a second-round match that was suspended on Wednesday because of rain.
Novak Djokovic is due to play Lorenzo Musetti later Thursday in a repeat of their last-16 clash a year ago won by the Italian.
World number two Jannik Sinner takes on Jan-Lennard Struff.