LAS VEGAS (AP) — One point on one card, a couple of points on some others. Ken Norton fought the greats, but the decisions he needed to be great never seemed to go his way.

He busted Muhammad Ali’s jaw to hand him only his second defeat. But he lost two narrow decisions to Ali the next two times they’d meet, including their final 1976 fight at Yankee Stadium.

And after he lost by just one point to Larry Holmes in their 1978 heavyweight title fight, Norton’s career was all but over.

“Kenny was a good, good fighter. He beat a lot of guys,” said Ed Schuyler Jr, who covered many of Norton’s fights for The Associated Press. “He gave Ali fits because Ali let him fight coming forward instead of making him back up.”

Norton, who died Wednesday at the age of 70, was forever linked to Ali for their trio of fights. Ali was beating everyone around him at the time but he always had trouble with Norton, even in the two fights he won.

Norton did briefly become the heavyweight champion, but he didn’t do it in the ring. His title was given to him after winning an elimination contest when Leon Spinks vacated the belt after deciding to fight Ali in a rematch instead of facing his mandatory challenger.

In his first defence, he and Holmes waged a bruising battle that went 15 rounds and could have gone to either man. It went Holmes’ way, and Norton would fight just five more times before finally retiring.

While boxing fans still talk about the bruising battle he waged with Holmes for the title, it was his first fight with Ali that made the former Marine a big name and the two fights that followed that were his real legacy.

Few gave Norton, who possessed a muscular, sculpted body, much of a chance against Ali in their first meeting, held at the Sports Arena in San Diego, where Norton lived. But his awkward style and close-in pressing tactics confused Ali, who fought in pain after his jaw was broken.

“Ali tore up his ankle while training and we were going to call the fight off but didn’t,” former Ali business manager Gene Kilroy said. “Ali said it’s not going to be that tough.”

It was, with Norton breaking Ali’s jaw in the early rounds and having his way with the former champion for much of the night. The loss was even more shocking because Ali had only lost to Joe Frazier in their 1971 showdown and was campaigning for the title he would win again the next year against George Foreman in Zaire.

“Norton was unorthodox,” Kilroy said. “Instead of jabbing from above like most fighters he would put his hand down and jab up at Ali.”

Kilroy said after the fight Norton visited Ali at the hospital where he was getting his broken jaw wired. Ali, he said, told him he was a great fighter and he never wanted to fight him again.

Ken Norton Jr, a coach with the Seattle Seahawks, confirmed his father’s death to the AP before handing the phone to his wife, too distraught to talk.

Norton had been in poor health for the last several years after suffering a series of strokes, Kilroy said.

“He’s been fighting the battle for two years,” he said. “I’m sure he’s in heaven now with all the great fighters. I’d like to hear that conversation.”

Norton didn’t have long to celebrate his big win over Ali. They fought six months later, and Ali won a split decision.

They met for a third time on September 28, 1976, at Yankee Stadium and Ali narrowly won to keep his heavyweight title.

Norton finished with a record of 42-7-1 and 33 knockouts. He would later embark on an acting career, appearing in several movies, and was a commentator at fights.

Former heavyweight champion Mike Tyson, who visited Norton at the veteran’s hospital in the Las Vegas suburb of Henderson, tweeted: “Ken Norton was always nice to me even when I was just an amateur fighter. He always treated me like I was somebody. Remarkable man.”

Ken Norton Jr. was a linebacker for 13 years in the NFL, playing for Dallas and San Francisco, and coaches the position for the Seahawks. He and his father were estranged for a time in the 1990s before finally reconciling.

Norton always gave his father credit for his career, saying he learned how to train hard by watching him go for early morning runs when he was a child.

“It’s been noted that my father and I are on speaking terms and everything’s back together now,” Norton Jr. said in 1995. “It’s part of what I do. No matter what I do, I can’t get away from boxing.”

Norton, born August 9, 1943, in Jacksonville, Illinois, started boxing when he was in the Marines, and began his pro career after his release from duty in 1967. He lost only once in his early fights but had fought few fighters of any note when he was selected to meet Ali.

At the time, Ali was campaigning to try to win back the heavyweight crown he lost to Joe Frazier in 1973.

After that first bout, they faced off two more times, including the final fight at Yankee Stadium on a night when police were on strike and many in the crowd feared for their safety. The fight went 15 rounds and Ali won a decision.

Kilroy said Ali and Norton never had any animosity toward each other and became good friends over the years. Still, Norton always thought he had won all three fights.

Norton would come back in 1977 to win an eliminator against Jimmy Young and was declared champion by the WBC when Spinks was stripped of the title.

His fight against Holmes in 1978 at Caesars Palace was his last big hurrah, with the two heavyweights going back and forth, trading huge blows inside a steamy pavilion in the hotel’s back lot. The fight was still up for grabs in the 15th round and both fighters reached inside themselves to deliver one of the more memorable final rounds in heavyweight history.

Norton was badly injured in a near fatal car accident in 1986. He recovered but never regained his full physical mobility.

“The doctors said I would never walk or talk,” Norton said at an autograph session in 2011 in Las Vegas, lifting his trademark fedora to show long surgical scars on his bald head.

Kilroy said Norton was visited at the hospital by former fighters, including Tyson, Earnie Shavers and Thomas Hearns.

Norton’s final fight came November 5, 1981, when he was knocked out in the first round by Gerry Cooney at Madison Square Garden.

Information on services and other survivors was not immediately released by the family.