The man who runs America's biggest bank can't stop worrying about tech startups.

In his annual letter to shareholders this week, JP Morgan CEO Jamie Dimon warned investors and those in the banking industry that "Silicon Valley is coming."

See also: 10 startups to watch in 2015

"There are hundreds of startups with a lot of brains and money working on various alternatives to traditional banking," Dimon wrote in the letter. "The ones you read about most are in the lending business, whereby the firms can lend to individuals and small businesses very quickly and — these entities believe — effectively by using Big Data to enhance credit underwriting."

Dimon may have been alluding to businesses like Lending Club and Prosper, which enable lending between users and are both have valuations in the billions. There are also a growing number of financial startups like Stripe, which handle payment transactions online and investing portfolio services like Wealthfront and Betterment.

"They are very good at reducing the 'pain points' in that they can make loans in minutes, which might take banks weeks," Dimon added in his letter. "We are going to work hard to make our services as seamless and competitive as theirs. And we also are completely comfortable with partnering where it makes sense."

"They all want to eat our lunch"

It's not the first time Dimon has issued a warning about Silicon Valley businesses.

"They all want to eat our lunch," he told investors a year ago. "Every single one of them is going to try."

In the latest shareholder letter, Dimon even went so far as to acknowledge the competitive threat posed by Bitcoin and other payment alternatives.

"You all have read about Bitcoin, merchants building their own networks, PayPal and PayPal look-alikes. Payments are a critical business for us — and we are quite good at it," he said. "But there is much for us to learn in terms of real-time systems, better encryption techniques, and reduction of costs and 'pain points' for customers."

Bitcoin has yet to achieve mainstream adoption and its price continues to fluctuate wildly, but a growing number of big-name investors have backed these startups, including the New York Stock Exchange, which recently backed Coinbase, a Bitcoin wallet application.

Dimon appeared to express some annoyance about apps and other "free riders" that don't have pay fees to move money, as banks must do. "Some payments systems, particularly the ACH system controlled by NACHA, cannot function in real time and, worse, are continuously misused by free riders on the system. There is a true cost to allowing people to move money," Dimon groused.

JP Morgan won't go down without a fight, Dimon assured shareholders. "Rest assured, we analyze all of our competitors in excruciating detail — so we can learn what they are doing and develop our own strategies accordingly."

Dimon also spared a few words for cybersecurity threats, an issue that has plagued JP Morgan and other financial institutions in the last year.

"Regarding privacy, I do not believe that most people fully understand what no longer is private and how their information is being bought, sold and used," he wrote. "As a bank, we are appropriately restricted in how we can use our data, but we have found many examples of our data being misused by a third party. We are going to be very aggressive in limiting and controlling how third parties can use JPMorgan Chase data."