Sweet was born in New York City, New York. His father was an auto mechanic and his first ambition was playing football. In 1939, he attended the University of Alabama; however, he was called away from his education for a tour of duty in World War II with the US Army Air Force, serving as a navigator on B-24 Liberator bomber aircraft. During his Service, he was shot down over Romania while flying on Operation Tidal Wave, and subsequently spent two years as a POW.
His first major film role was in the motion picture The Young Doctors in 1961. He went on to make numerous appearances in films such as You're a Big Boy Now (1966), A Lovely Way to Die (1968), The Swimmer (1968) and Finian's Rainbow (1968) as the Sheriff, and on television through the 1960s and 1970s, including roles on The Defenders, The Edge of Night, Another World as Police Chief Gil McGowan, and Dark Shadows. In his personal life he married and had a son. This marriage ended in divorce before the mid-1970s; Sweet later remarried.
Through the 1970s he was much in demand, with roles in films such as Colossus: The Forbin Project (1970), The Out-of-Towners (1970), The New Centurions (1972), Fear Is the Key (1972), Sisters (1973), Cops and Robbers (1973), The Lords of Flatbush (1974), Amazing Grace (1974), The Bad News Bears in Breaking Training (1977), Which Way Is Up? (1977), Go Tell the Spartans (1978), Heaven Can Wait (1978) and The Wanderers (1979). In addition to film roles, he also had guest starring roles on Little House on the Prairie and Mrs. Columbo. He had a notable role as FBI Director J. Edgar Hoover in the 1978 television miniseries King, based on the life of civil rights leader Martin Luther King, Jr.
He was well known for his role as policeman Gil McGowan, third husband of Ada Davis (later Hobson), on the soap opera Another World (1972–1977). He also voiced the character of Manhattan Subway Transit Police Captain Costello in the 1974 version of The Taking of Pelham One Two Three.
As the 1980s began, Sweet worked steadily in such films as Below the Belt (1980) and Reds (1981), and television series like Hill Street Blues and Hart to Hart.
Sweet is best remembered for his role as police chief and father Carl Kanisky, who was constantly at odds with housekeeper Nell Carter on the sitcom Gimme a Break!, a role he played from 1981 until his death. Sweet was diagnosed with cancer during the series' fourth season, but continued to work. Dolph Sweet died on May 8, 1985; his final appearance on Gimme a Break! aired 3 days later on the day of his funeral. He was survived by his wife, Iris Braun, and a son, Jonathan.
