Drew was born in 1904 into an African-American middle-class family in Washington, D.C. His father, Richard, was a carpet layer and his mother, Nora Burrell, was a Teacher. Drew and his siblings grew up in D.C.'s Foggy Bottom neighborhood and he graduated from Dunbar High School in 1922. Drew won an athletics scholarship to Amherst College in Massachusetts, where he graduated in 1926. An outstanding athlete at Amherst, Drew also joined Omega Psi Phi fraternity. He attended medical school at McGill University in Montreal, Quebec, Canada, receiving his MDCM in 1933, and ranked 2nd in his class of 127 students. A few years later, Drew did graduate work at Columbia University, where he earned his Doctor of Medical Science degree, becoming the first African American to do so.