Amy was born in Houston, Texas. He learned how to play clarinet before joining the Army, and during his time in service, picked up the tenor saxophone. After his discharge, he attended and graduated from Kentucky State College. He worked as an educator in Tennessee while playing in midwestern jazz clubs. In the mid-1950s he relocated to Los Angeles and signed with Pacific Jazz Records, often playing with organist Paul Bryant. In the mid-60s he spent three years as musical director of Ray Charles' orchestra, together with his wife, Merry Clayton and Steve Huffsteter.
As well as leading his own bands and recording albums under his own name, Amy also did session work and played the solos on several recordings, including The Doors song "Touch Me", Carole King's Tapestry, and Lou Rawls' first albums, Black and Blue and Tobacco Road, coinciding with Dexter Gordon in the Onzy Matthews big band, as well as working with Marvin Gaye, Tammy Terrell and Smokey Robinson.
Up until his death he was married to singer and recording artist Merry Clayton