Ernest Barclay was born in Bridgetown, Barbados, as one of the twelve children of Sarah Ann Bourne Barclay, the daughter of London Bourne. His father Anthony Barclay Jr. led the Barbados Emigration Society that organized the 1865 exodus of 346 people from Bridgetown to Liberia. When his father died in 1866 soon afterwards, his mother Sarah took over leading the Barbadian immigrant community in Liberia while continuing to parent her 10 surviving children.