On June 1, 1935, he lost the title to Baltasar "Sangchili" Belenguer Hervás of Spain, by a fifteen round decision, at Valencia, Spain. They had a rematch on March 4, 1938, with Brown avenging his earlier loss with a fifteen round decision, but by then, Sixto Escobar of Puerto Rico had already taken the world Bantamweight championship. His rematch win over Sangchili is believed to be his last great night and, bowing to Cocteau's wishes, Brown vowed to retire after one more fight. That came in 1939 against Valentine Angelmann in Paris (Brown stopped him in eight rounds).