*Club domestic league appearances and goals, correct as of 06:55, 18 August 2024 (UTC) ‡ National team caps and goals, correct as of 23:59, 8 June 2021 (UTC)
Born in Lambeth, London,[4] Scarlett signed his first professional contract with Gillingham in summer 2019.[5] He made his professional debut for Gillingham on 4 September 2018 in their 4–0 EFL Trophy defeat to Portsmouth, coming on as a substitute for Alex Lacey after 80 minutes.[6]
In January 2019, Scarlett was sent out on work experience loan to Sheppey United,[7] where he went on to make seven first-team appearances.[8]
On 2 September 2019, Gillingham announced that Scarlett's contract with the club had been cancelled.[5] He later signed for Colchester United, first featuring for their under-23 side in November 2019.[9]
Scarlett made his Colchester United debut on 5 September 2020, starting in their 3–1 EFL Cup defeat to Reading.[10] He then made his English Football League debut in Colchester's 1–1 draw at Walsall on 10 October.[11]
Colchester announced that Scarlett was one of seven under-23 players who had their contract terminated by mutual consent on 1 February 2021.[12]