First, it's jumping Jehosaphat.

Jehosaphat was a wealthy king of Jerusalem. One day an army from the east marched on Jerusalem to take his gold. Jehosaphat told the people not to worry that the battle would be his on the following day. The people let out a joyous cry "Great Jumping Jehosaphat" . It was total joy at not having to go to war!

Jehosaphat as an interjection, first recorded in 1857, was probably a euphemism for Jehovah. This would be like many other oaths that take, but coyly alter, a holy name. Thus oh God! and Jesus! become the oaths egad! and jeez!

Jumping had been added to other oaths, as in Jumping Geraniums!, Jumping Jellybeans!, Jumping Juleps!, Jumping Jupiter!, and Jumping Jiminy Cricket! (Notice that Jiminy Cricket plays off the initials of Jesus Christ.) Within a decade the verbal Jehosaphat was sometimes one who jumped (Mayne Reid, 1866¹: By the jumpin’ Geehosofat).

But at the time other forms were used as well, including an alliterative one. (George Washington Harris 1867: by the jinglin' Jehosephat), and the prevailing form seems to have been simply Jehosephat! or Great Jehosephat!, without jumping. No one has explained why jumping Jehosaphat ultimately prevailed.