Summersby was the son of the Reverend B J Summersby, a Congregational minister from Oxfordshire. He was educated locally until the age of fourteen when he left school and moved to London to serve a four-year apprenticeship in the drapery business. He and his wife had two sons and a daughter.