Gilder was born at Bordentown, New Jersey. He was the son of Jane (Nutt) Gilder and the Rev. William Henry Gilder, and educated at his father's seminary in Flushing, Queens. There he learned to set type and published the St. Thomas Register. Gilder later studied law at Philadelphia.