Scrimm had several minor supporting roles in the early 1970s before being cast as the Tall Man, the chief villain in Don Coscarelli's independent horror film Phantasm (1979). Scrimm stood approximately 6 feet 4 inches (1.93 m). To appear even taller when playing the Tall Man, he wore suits that were several sizes too small and platform shoes. His Phantasm role led to a steady acting career in theater and television. Scrimm had a recurring role as an SD-6 agent on Alias.