After more than 100 hours of coding over three months, Mike Soule was finally ready to switch on his project. He didn’t know what to expect. If things went right, he could be on his way to financial success. If things went wrong, he could lose his savings.

His creation wasn’t a new mobile app or e-commerce store. It was a computer program that would buy and sell currencies 24 hours a day, five days a week.

DIY’s newest frontier is algorithmic trading. Spurred on by their own curiosity and coached by hobbyist groups and online courses, thousands of day-trading tinkerers are writing up their own trading software and turning it loose on the markets.

“It’s definitely one of those things where you are like, ‘Is this going to work?’” said Mr. Soule, who is a student at University of Nevada, Reno, and a network administrator at Tahoe Forest Health system. “When it finally started trading, wow, wow. I don’t know if that is what I expected, but I did it.”

Interactive Brokers Group Inc. actively solicits at-home algorithmic traders with services to support their transactions. YouTube videos from traders and companies explaining the basics have tens of thousands of views. More than 170,000 people enrolled in a popular online course, “Computational Investing,” taught by Georgia Institute of Technology professor Tucker Balch. Only about 5% completed it, but at an algorithmic trading event in New York in April, three people asked him for his autograph.