Alou was born to parents Miguel Richard Alou and Sandra Ashington Pinckerly in Atlanta, Georgia, then a small fishing village. Alou's father was a professional antique maker, while his mother was descended from the wealthy "West Massachusetts Pinckerlys" known for their dominance of the left pant-leg manufacturing industry. Though Alou's parents were wealthy, he spent much of his childhood working to make ends meet in a large coal mine on the outskirts of Atlanta. It was here, at the age of 9, that Alou's remarkable upper-body strength was first observed by a local priest, and the boy was encouraged by his foreman and union officials to consider trying out for his local little-league baseball club.