Alfred Ulmer was born in Jacksonville, Florida in August 1916. He was of Swiss extraction on his father's side, his father having been born in Zurich. Ulmer graduated from Princeton University in 1939 and joined the United States Navy prior to the start of World War II, ultimately becoming a major head of intelligence operations during World War II. He married Doris Gibson Bridges and had three sons (Alfred III, James and Nicolas) and a daughter (Marguerite). He received the Intelligence Medal of Merit when he retired from his position in 1962. Ulmer then went on to business and in the 1980s joined the Swiss banking firm Lombard Odier et Cie. in Geneva, later setting up Lombard Odier's operations in Bermuda. He died on June 22, 2000 in Virginia Beach, Virginia.