The Endowment House stood on the northwest corner of Temple Square. Initially, it was a two-story adobe building, 44 feet by 34 feet, with a single-story 20-foot extension on its north side. In 1856 another extension was added on its south side and a baptistry on its west side.