He hosted the CBC summer radio program Road Dot Trip in 2000 and has contributed to shows like Dispatches and Outfront. In 2000, his career received a boost after he was selected to work on Ira Glass' popular public radio program This American Life. Goldstein relocated to Chicago to work as a Producer on the show. Many of Goldstein's pieces have been featured on This American Life where he is a contributing Editor. From 2000 to 2002 he was also a Producer of the show.
In 2001, Goldstein's debut novel, Lenny Bruce Is Dead, was published by Coach House Books. Goldstein also co-authored Schmelvis: In Search of Elvis Presley's Jewish Roots with Max Wallace, an account of a Hasidic Elvis impersonator and rabbi's quest to trace the Jewish roots of Elvis Presley. Goldstein has also been published in The New York Times Magazine, Saturday Night, The New York Times, The Walrus, GQ, the Journey Prize Anthology and the National Post. He has also self-produced a number of small publications, most notably carwash the size of a peach.
In 2002, Goldstein returned to Montreal and started work on several projects for CBC Radio One. He hosted WireTap, which debuted in 2004 and ended in 2015. The program featured stories told over the phone.
Goldstein was in a relationship with the author Heather O'Neill that ended in 2007.
In May 2014, Goldstein played an "expert witness" in humorist John Hodgman's comedy/court show podcast Judge John Hodgman.
Goldstein married fellow radio Producer Emily Condon in 2015. Their son was born in December 2016.
In September 2016, Goldstein began a new podcast, Heavyweight, with podcast network Gimlet Media.
