Essential reconstruction was made in 1780 with Marcin Knackfus project. Construction of the palace was started by the widow of Jerzy Abramowicz, Andrzej's brother, Marcjanna (née Dernalowicz).[1] In 1844 the palace became the property of the Orthodox consistory clergy. In 1853-1860 it was rebuilt under the direction of Tomasz Tyszecki and connected with St. Casimir's Church, then an Orthodox church.[1] In the interwar period, the building belonged to the Jesuit Order.[1] After World War II it was Juozas Tallat-Kelpša High School of Music, which later was transformed into conservatory.