ROTTERDAM, the Netherlands—One recent afternoon at a Shell-branded station on the outskirts of this Dutch city, the price of a gallon of unleaded gas started ticking higher, rising more than 3½ cents by closing time. A little later, a competing station 3 miles down the road raised its price about the same amount.

The two stations are among thousands of companies that use artificial-intelligence software to set prices. In doing so, they are testing a fundamental precept of the market economy.

...