Corbett married, but his wife died in childbirth. Following her death, he moved to Boston and continued working as a hatter. He became a reborn, evangelical Christian and changed his name to Boston. Trying to imitate Jesus, he began to wear his hair very long.
On July 16, 1858, in order to avoid the temptation of prostitutes, Corbett castrated himself with a pair of scissors. Afterward, he went to a prayer meeting and ate a meal before going for medical treatment.
Corbett joined the Union army at the outbreak of the American Civil War. He re-enlisted three times. His final rank was sergeant in the 16th New York Cavalry. He was captured by the Confederate Army on June 24, 1864, and was held captive at Andersonville Prison. He was eventually released and returned to his unit
On April 24, 1865, he was selected as one of the 26 cavalrymen to pursue John Wilkes Booth after his assassination of Abraham Lincoln. On April 26, they cornered Booth and fellow conspirator David Herold in a tobacco barn on the Virginia farm of Richard Garrett. The barn was set on fire. Herold surrendered, but Booth remained inside. Corbett was positioned by a large crack in the barn wall. He saw Booth moving about inside and shot him with a Colt revolver from a distance of several yards. Booth died a few hours later as Corbett's shot had hit his spinal cord.