Edward Brian "Tubby" Hayes was born in St Pancras, London, and brought up in London. His father was a BBC studio violinist who gave his son violin lessons from an early age. By the age of ten Hayes was playing the piano, and started on the tenor sax at eleven. Dizzy Gillespie was an early influence:
I always used to listen to swing music in the early 'Forties and, in fact, I was just a kid at the time. I did not really intend becoming a tenor player, though I always liked tenor. I think maybe Dizzy influenced me more than Parker because he was sort of more accessible, he caught your attention more. As far as my influences over the years are concerned, Getz was it at one stage in the proceedings, and later Rollins, Coltrane, Hank Mobley and, to a lesser degree, even Zoot.
One much repeated story about Hayes' early career was told by Ronnie Scott. Scott was playing at a club near Kingston, and was asked whether he minded if a local player sat in: "This little boy came up, not much bigger than his tenor sax. Rather patronisingly I suggested a number and off he went. He scared me to death."
Hayes attended Rutlish School in Merton Park, south London, where his music teacher was Roy Howard. After a period spent playing with various semi-professional bands around London, Hayes left school and started playing professionally at the age of fifteen.