With his wife, Lucinda Hamilton (1826-1899), and their unmarried daughter, Martha Ann Calhoun (1851-1926). They lived and died in Tarrytown, Georgia.