A native of Scotland, Peter Fraser was born in Hill of Fearn, a small village near the town of Tain in the Highland area of Easter Ross. He received a basic education, but had to leave school due to his family's poor financial state. Though apprenticed to a carpenter, he eventually abandoned this trade due to extremely poor eyesight – later in life, faced with difficulty reading official documents, he would insist on spoken reports rather than written ones. Before the deterioration of his vision, however, he read extensively – with socialist activists such as Keir Hardie and Robert Blatchford among his favourites.