From When Benedict Arnold came to town:

Governor Thomas Jefferson had called out the Virginia militia, but they proved to be completely unequal to Arnold’s forces and quickly retreated when Arnold approached. Arnold and his men marched into Richmond on January 5, likely following Route 5 into the city. It was claimed by her descendants that Elizabeth Ege Welsh, owner of what is now the Poe Museum, watched from her front door as Arnold’s men marched down Main Street.

According to a 19th century account by Henry Barton Dawson, “About two hundred men had assembled, under Colonel John Nicholas, on the heights of Richmond Hill [Church Hill], near the venerable meeting-house of St. John’s Church; and Lieutenant-colonel Simcoe was ordered to dislodge them, but, without firing a shot, they fled in confusion when he reached the summit of the hill.”

[…]

Arnold, meanwhile, made himself comfortable at the City Tavern, on the northwest corner of 19th and East Main streets. His men were sent to Saint John’s Church to set up camp, in and around the church building. By this time, the weak colonial forces that had gathered on Church Hill had fled – along with Thomas Jefferson and most of the government, who had quickly removed to Charlottesville.

From Richmond, Arnold sent Jefferson a letter offering to keep Richmond safe in exchange for all of the city’s tobacco and other stores. Jefferson angrily refused, and Arnold then laid waste to Richmond by setting it ablaze on January 6.