Green was born in St. Louis, Missouri. He first performed in a professional setting at the age of 12. His influences were Charlie Christian, Charlie Parker, Lester Young, and Jimmy Raney, he first played boogie-woogie before moving on to jazz. His first recordings in St. Louis were with tenor saxophonist Jimmy Forrest for the United label. The drummer in the band was Elvin Jones, later the powerhouse behind John Coltrane. Grant recorded with Elvin again in the early Sixties. Lou Donaldson discovered Grant playing in a bar in St. Louis. After touring together with Donaldson, Grant arrived in New York around 1959-60.
Lou Donaldson introduced Grant to Alfred Lion of Blue Note Records. Lion was so impressed with Grant that rather than testing Grant as a sideman, as was the usual Blue Note practice, Lion arranged for him to record as a group leader first. However, due to a lack of confidence on Green's part the initial recording session was only released in 2001 as First Session
Despite the shelving of his first session, Green's recording relationship with Blue Note was to last, with a few exceptions, throughout the Sixties. From 1961 to 1965, Grant made more appearances on Blue Note LPs, as leader or sideman, than anyone else. Grant's first issued album as a leader was Grant's First Stand. This was followed in the same year by Green Street andGrantstand. Grant was named best new star in the Down Beat critics' poll, in 1962. He often provided support to the other important musicians on Blue Note, including saxophonists Hank Mobley,Ike Quebec, Stanley Turrentine and organist Larry Young.
Sunday Mornin' , The Latin Bit and Feelin' the Spirit are all loose concept albums, each taking a musical theme or style: Gospel, Latin and spirituals respectively. Grant always carried off his more commercial dates with artistic success during this period. Idle Moments (1963), featuring Joe Henderson and Bobby Hutcherson, and Solid (1964), are thought of as two of Grant's best recordings.
Many of Grant's recordings were not released during his lifetime. These include McCoy Tyner and Elvin Jones (also part of the Solid group) performing on Matador (also recorded in 1964), and several albums with pianist Sonny Clark. In 1966 Grant left Blue Note and recorded for several other labels, including Verve. From 1967 to 1969 Grant was, for the most part, inactive due to personal problems and the effects of heroin addiction. In 1969 Grant returned with a new funk-influenced band. His recordings from this period include the commercially successful Green is Beautiful and the soundtrack to the film The Final Comedown.
Grant left Blue Note again in 1974 and the subsequent recordings he made with other labels divide opinion: some consider Green to have been the 'Father of Acid Jazz' (and his late recordings have been sampled by artists including US3, A Tribe Called Quest and Public Enemy), whilst others have dismissed them (reissue producer Michael Cuscuna wrote in the sleeve notes for the album Matador that "During the 1970s [Green] made some pretty lame records").
Grant spent much of 1978 in hospital and, against the advice of doctors, went back on the road to earn some money. While in New York to play an engagement at George Benson's Breezin' Lounge, Grant collapsed in his car of a heart attack in New York City on January 31, 1979. He was buried in Greenwood Cemetery in his hometown of St. Louis, Missouri, and was survived by six children. Since Green's demise, his reputation has grown and many compilations of both his earlier (post-bop/straight ahead and soul jazz) and later (funkier/dancefloor jazz) periods, exist.