Seneca was born Joel McGhee, Jr. in Cleveland, Ohio. Prior to his acting career, he belonged to the R&B singing group The Three Riffs, which was active from the late 1940s and performed at upscale supper clubs in New York City. He was also a Songwriter and had big hits with "Talk to Me" which was sung by Little Willie John, and "Break It to Me Gently," which was a smash hit twice, once by Brenda Lee in 1962 and once by Juice Newton in 1982.