Ginzberg was born into a religious family whose piety and erudition was well known. The family traced its lineage back to the legendary Gaon of Vilna. In his own mind, Ginzberg emulated the Vilna Gaon’s intermingling of ‘academic knowledge’ in Torah studies under the label ‘historical Judaism’. In his book "Students, Scholars and Saints", Ginzberg quotes the Vilna Gaon instructing, “Do not regard the views of the Shulchan Aruch as binding if you think that they are not in agreement with those of the Talmud.”