After the Reds lost the war, Gylling fled to Sweden,[2] but later moved to the Soviet Union. He became one of the main leaders of the Karelo-Finnish ASSR as Chairman of Council of People's Commissars of the Karelo-Finnish SSR 1920–1935.
He was accused of nationalism, removed in 1935 and arrested in 1937 as a part of the Finnish Operation of the NKVD. There are some contradictions concerning Gylling's death. According to earlier Soviet sources, Gylling died in August 1944, but according to other sources he was actually executed earlier, 1940 or 1938. According to the most recent information, the most likely date of his execution was 14 June 1938. Gylling was posthumously rehabilitated by the Soviet authorities on 16 July 1955.