Joseph was the son of Rev. James R. and Mourning Winn Gunn. His grandparents were Thomas Gunn III and Susannah Burnet. Barbara Elizabeth Gunn was his daughter.
