Margolis was born in Philadelphia, Pennsylvania, the son of Fanya (neé Fried) and Isidore Margolis. He is from a Jewish immigrant family from Europe. He attended Temple University briefly before moving to New York City, where he studied drama with Stella Adler and at the Actors Studio. He is noted for his supporting roles in Scarface (1983) and the films of Darren Aronofsky: π (1998), Requiem for a Dream (2000), The Fountain (2006), The Wrestler (2008), Black Swan (2010), and Noah (2014).
In 1991, Margolis played Helmut Dieter in the soap opera Santa Barbara. He has had recurring roles on numerous other TV shows, including The Equalizer, Quantum Leap, Oz, Law & Order, Crossing Jordan, Californication, Breaking Bad, and Better Call Saul. In January 2015, he portrayed Felix Faust in the Constantine episode "Quid Pro Quo". For his role in Breaking Bad and Better Call Saul, Margolis received a nomination for a Primetime Emmy Award for Outstanding Guest Actor in a Drama Series.
Margolis is also a stage actor. In the Berkeley Repertory Theater's 2014 season, he appeared as Gus in Tony Kushner's The Intelligent Homosexual's Guide to Capitalism and Socialism with a Key to the Scriptures.
