In the autumn, Håkon Meyer was fired as leader of the mega-union Forbundet for offentlige yrker. Olsen was installed as leader, serving for a short time. He was then fired, formally for alcohol abuse and economic irregularities, but he had also criticized Just Lippestad and Odd Fossum. In November 1942, Olsen had initiated his own campaign against "incompetence" in Nasjonal Samling, and posed the threat that if Lippestad did not remove incompetent elements, Olsen would make use of a ticket to London, claimed to be in his possession. Olsen tried to cooperate with Kåre Rein, who was of better standing, but Rein found Olsen and his supporters to "not hold water morally and character-wise". Olsen quarreled with Lippestad and Fossum during a study trip to Germany, and Fossum stripped him of the title as deputy leader of NS Faggruppeorganisasjon. Fossum even had Statspolitiet arrest Olsen, but he was released as Statspolitiet and Sikkerhetspolitiet found no grounds for incarceration.[4]
In January 1943, Olsen was hired in a minor job in the Directorate of Labour (not to be confused with the later Directorate of Labour). He worked here in 1943 and from July 1944 to May 1945. He lost his position at the end of the war, but had since long faded into obscurity.[1]