The win over the title-contending Cobb gave Norton another shot at a potential title-fight, and on May 11, 1981. at Madison Square Garden he stepped into the ring with top contender Gerry Cooney, who, like Cobb, was undefeated entering the fight. Very early in the fight it became clear that Norton was no longer the caliber of fighter he once was, as Cooney's first punch caused Norton's legs to buckle. Norton continued to take shots from Cooney in his corner for nearly a full minute before Perez, who refereed his last fight, stepped in to stop the bout 54 seconds in, as Norton was slumped in his corner. Norton decided to retire following the match and turned his attention to charitable pursuits. Norton's enduring legacy as a fighter is that he is considered second to Joe Frazier as Ali's main nemesis and toughest opponent. Norton fought Ali to three decisions and was never hurt or knocked down. All three bouts were close and subject to controversy. Unfortunately, Norton was less successful against three of the greatest punchers of all time, losing by KO to Foreman and Shavers and by TKO to Cooney. Norton was considered past his prime in boxing from 1979 to 1981.