Werner was born into a musical family. His mother, Johanna Geisler, was a soprano and his father was the famous conductor and composer, Otto Klemperer. His father's cousin, Victor Klemperer, was the author of a couple of published diaries which talked about life in Nazi Germany. Werner was a "Halbjude" (1/2 Jew), but raised a Catholic. While growing up, he took lessons in playing the piano, trumpet and violin, while his father, Otto, worked as a conductor at the Cologne Opera House, and later at the Kroll Opera House in Berlin during the 1920s until the Nazi takeover in 1933. After the closing of the Kroll and then the disappearance of a physician whose only crime was being Jewish, Otto took his family out of Germany, stopping first in Vienna, Austria and then in Los Angeles, California, a