Warren Buffett's Berkshire Hathaway added more than 35 million shares of J.P. Morgan Chase last quarter, according to the firm's latest filing with the Securities and Exchange Commission.

The "Oracle of Omaha" is now very much long the U.S. banking system. Berkshire's second- and third-largest holdings behind Apple are Bank of America and Wells Fargo. He increased the firm's Bank of America stake by 29 percent last quarter. Berkshire also added to its already large stakes in U.S. Bancorp, Goldman Sachs and Bank of New York Mellon last quarter, filings showed.

Shares of J.P. Morgan rallied 1.5 percent in after-hours trading following news of Buffett's new $4 billion stake, which was reported in the company's 13-F filing. The filing documents the company's holdings as of end of September. Berkshire slightly decreased its stake in Wells Fargo.

Buffett and J.P. Morgan CEO Jamie Dimon admire each other. Buffett has said he reads Dimon's annual shareholder letter for insights into the banking industry. Dimon is the longest-tenured of CEOs leading a major U.S. bank. The pair, along with Jeff Bezos of Amazon, in January announced a partnership to cut health-care costs and improve services for their U.S. employees.

Todd Combs, one of Buffett's investing lieutenants at Berkshire who first got the company involved in Apple, sits on the board of directors of J.P. Morgan.