Avildsen's first wife was Marie Olga Maturevich (Melissa McCall). After they divorced, he married Actress Tracy Brooks Swope in 1987; they divorced in 2003. He had four children. His estranged son Ash (born November 5, 1981), founded Sumerian Records. Another son, Jonathan Avildsen, appeared in the films The Karate Kid Part III and Rocky V. His eldest son was named Anthony and he had a daughter, Bridget.