Herrmann attended Purdue University, where he played for coach Jim Young's Purdue Boilermakers football team from 1977 to 1980. Herrmann had an impressive college career; in 1980 he was recognized as a consensus first-team All-American, he was selected as the Big Ten Conference's Most Valuable Player, and he finished fourth in voting for the Heisman Trophy. His 9,946 career passing yards set an NCAA record (which has since been broken). He is one of only three Purdue quarterbacks to start in three consecutive bowl games (Drew Brees did the same, and Kyle Orton started four straight). Herrmann won all three of his bowl games, and was selected Most Valuable Player in each of them: the 1978 Peach Bowl, the 1979 Bluebonnet Bowl, and the 1980 Liberty Bowl. He also holds the Liberty Bowl record for passing touchdowns.