In 1957, he began working with the Carl-Henrik Norin orchestra in Stockholm. Over the coming years, he would work frequently in Copenhagen with groups such as the Jazz Quintet 60 and the Danish Radio Jazz Orchestra. Back in Sweden, he continued his work with Gullin and also worked frequently with trumpet player Jan Allan. In 1966, he joined the Danish Radio Big Band (led by Ib Glindemann), spending the summer in Gothenburg helping to look after his mother's hotel in the city. There he became seriously ill and died on his way to the hospital on 17 August 1966.[1]
The liner notes to the CD Rare Danish Recordings 1956-1957 states that:
According to the most credible jazz witnesses in Denmark and Sweden...Billberg was an extroverted artist with a gluttonous appetite for life and all it has to offer – including intoxicants, which came to affect both his emotional and physical health - but never stopped his love of playing in all tempos and keys.[2]
Music broadcaster Peter H. Larsen has drawn comparisons between Billberg's lyricism and that of Lee Konitz.[2]