Nicklas Backstrom had a goal and an assist, and Eric Fehr scored the tiebreaking goal with 5:09 left to lift the Capitals over the Rangers 3-2.
Moments after New York defenseman Dan Girardi's drive hit the post, Fehr took a pass from Backstrom during 4-on-4 play, skated in, and beat Talbot from the right circle for his sixth goal.
"It was just a quick release that beat me," Talbot said. "It's (a save) I've got to make, especially in a situation like that where it's a tie game with five minutes to go and we're pressing for the win."
Mike Green added a goal, and Grubauer had 37 saves for Washington, which had dropped two straight before taking the finale of a three-game homestand.
"It's huge," Grubauer said. "Every win is, because I haven't played many games. Every win is a great feeling for me."
Grubauer improved to 5-1-1 in eight games since being called up to Washington.
"Right now, it's only been eight games, and goalies -- like everybody -- have their waves," Capitals coach Adam Oates said. "Right now it looks like he's playing with confidence, and he's giving the players confidence."
Benoit Pouliot and Carl Hagelin scored for New York, which had won two straight. It was the Rangers' first game after a nine-game homestand.
"It's disappointing, obviously. I thought we played a pretty strong game," Girardi said. "That last (goal) is tough. I hit the post then they come right back down and score."
Talbot, making his third straight start in place of No. 1 goalie Henrik Lundqvist, stopped 31 shots in the opener of a five-game road trip. He came in with an 8-2 record and a 1.60 goals against average in 12 games, and won the final two games of the homestand.
Backstrom continued his impressive December and has 17 points in his last eight games. He began the night fourth in the NHL in points and third in assists,
"He's handling the puck with confidence, and he's making great plays," Fehr said. "It's what we expect of him, but it's pretty unbelievable to watch."
Hagelin tied the game with a short-handed goal 17 seconds into the third period. He took a feed from Ryan McDonagh, skated in alone on Grubauer, and scored with a backhanded shot.
The Rangers outshot Washington 14-7 in the opening period and had two power plays to one for the Capitals, but trailed 1-0.
With Rick Nash off for tripping, Martin Erat sent a pass to Green at the point for a one-timed shot that got past Talbot on the glove side at 6:55 for his third goal.
New York lost defenseman Anton Stralman to an undisclosed injury in the period and played the remainder of the game with five defensemen.
"You go from six to five defensemen and you're top three have to log a tremendous amount of minutes," Rangers coach Alain Vigneault said, "Might make it a little bit more challenging in terms of defending and attacking, but that's not an excuse. Those are things that happen."
The Rangers tied it on a power play when Pouliot deflected Brad Richards' shot from the point past Grubauer at 1:37 of the second period. It was Pouliot's sixth goal of the season and third in four games.
The Capitals appeared to retake the lead midway through the second, but a replay review showed that Fehr's shot bounced off the post, went behind Talbot, but didn't cross the goal line.
After a flurry of saves by Grubauer late in the period, the Capitals went ahead 2-1 when Steve Olesky pushed a loose puck ahead to Backstrom. Talbot could only get a piece of the shot from the right circle that produced Backstrom's 10th goal with 1:05 left in the period.
"He's playing great, he's obviously in a groove," Oates said. "We count on him so much for so many touches in a game."
Lundqvist, a three-time All-Star, signed a seven-year, $59.5 million contract extension on Dec. 4 to become the highest paid NHL goalie.
Since then, he is 2-4-2 with a 3.41 goals-against average in eight games. Lundqvist lost his most recent appearance 5-3 to the New York Islanders last Friday, giving up four goals on 19 shots.
Alex Ovechkin failed to score a goal for the ninth straight game against the Rangers, including the playoffs. ... Capitals C Brooks Laich (groin), who returned Monday after missing 11 games, was out of the lineup. ... RW Erat has 14 assists in 35 games, but hasn't scored a goal.
Copyright by STATS LLC and The Associated Press
WASHINGTON, DC - DECEMBER 27: Alex Ovechkin #8 of the Washington Capitals warms up prior to playing an NHL game...
(Photo by Patrick McDermott/NHLI via Getty Images)