Block grew up on Long Island, New York. He went to college to study English in Chapel Hill, North Carolina, but relocated to New York City, where he began his music career as a subway busker. In 1996, Block self-released his debut album, Lead Me Not Into Penn Station, which was well received by critics. In reviewing the album, Laurel Bowman of Alternative Press wrote that Block "practices the alchemy of melodious discord: pretty enough to draw you in, rough enough to keep you listening." The CD included CD-ROM content, including video clips, song lyrics, photos and a New York City guide. Block released a music video for the album song "Rhinoceros," which was broadcast on cable network MTV2.