Lionel Frederick Cole was born on October 15, 1931, the youngest of Edward and Paulina Nancy Cole's five children. His three elder brothers, Eddie, Ike and Nat (twelve years Freddy's senior) were all musicians.
"I started playing piano at five or six," Freddy remembers. "Music was all around me." In the Chicago home of his youth, visitors included Duke Ellington, Count Basie, and Lionel Hampton. He also credits Billy Eckstine as a major influence.