He was educated at Whitestown Seminary, and then at Hamilton College in Clinton New York. At Hamilton, he was noted for his skills in oratory and debate. After his 1878 graduation, he remained at Hamilton for a for a year to study law, then continued his studies at the Utica office of Beardsley, Cookingham and Burdick, which included his brother in law Henry J. Cookingham as a partner. He was admitted to the bar in 1880, and practiced with Cookingham in the firm of Cookingham & Martin. Sherman was also President of the Utica Trust & Deposit Co. and the New Hartford Canning Company. He became mayor of Utica at age twenty-nine.