Oliver Nelson's family was musical: his brother was also a saxophonist who played with Cootie Williams in the 1940s, and his sister sang and playedpiano. Nelson began learning to play the piano when he was six, and started on the saxophone at eleven. From 1947 he played in "territory" bands around Saint Louis, before joining the Louis Jordan big band from 1950 to 1951, playing alto saxophone and arranging.
In 1952 Nelson underwent military service in the Marines playing woodwinds in the 3rd Division band in Japan and Korea. It was in Japan that Nelson attended a concert by the Tokyo Philharmonic Orchestra and heard Maurice Ravel's Mother Goose Suite and Paul Hindemith's Symphony in E Flat. Nelson later recalled that this was the "First time that I had heard really modern music, for back in St. Louis I hadn't even known that negroes were allowed to go to concerts, I realised everything didn't have to sound like Beethoven or Brahms...It was then that I decided to become a composer". Nelson returned to Missouri to study music composition and theory at Washington and Lincoln Universities, graduating in 1958. Nelson also received private tutoring from composersElliott Carter, Robert Wykes and George Tremblay. While back in his hometown of St. Louis, he met and married Eileen Mitchell; the couple had a son, Oliver Nelson Jr., but soon divorced. After graduation, Nelson married Audrey McEwen, a union which lasted until his death; they had a son, Nyles. Audrey was a native of St. Louis, Missouri.
Nelson moved to New York, playing with Erskine Hawkins and Wild Bill Davis, and working as the house arranger for the Apollo Theater in Harlem. He also played on the West Coast briefly with the Louie Bellson big band in 1959, and in the same year began recording as leader with small groups. From 1960 to 1961 he played tenor saxophone with Quincy Jones, both in the U.S. and on tour in Europe