Caan was born on March 26, 1940, in the Bronx, New York, the son of Sophie (née Falkenstein; June 24, 1915 – January 18, 2016) and Arthur Caan, Jewish immigrants from Germany. His father was a meat dealer and butcher. One of three siblings, Caan grew up in Sunnyside, Queens, New York City. He was educated in New York City, and later attended Michigan State University. He later transferred to Hofstra University in Hempstead, New York, but did not graduate. His classmates at Hofstra included Francis Ford Coppola and Lainie Kazan.