Warren Buffett's Berkshire Hathaway increased its stake in Apple by 5 percent, according to a regulatory filing Tuesday that discloses positions through the end of the second quarter.

Berkshire also upped its holdings of Goldman Sachs by 21 percent, Delta by 18.8 percent and Southwest by 18.7 percent, the filing said, and it added another 6.7 percent to its Teva stake, which it had doubled in the first quarter of this year.

Buffett's conglomerate has been steadily building its Apple stake for two years to become its second-biggest shareholder. Apple recently won the race to become the first U.S. company to reach a market value of $1 trillion, and its shares have surged 23 percent this year.

"If you look at Apple, I think it earns almost twice as much as the second most profitable company in the United States," Buffett told CNBC in May.

Banks have been another focus. Berkshire added to Goldman in the second quarter as the stock has fallen nearly 10 percent this year. Goldman recently announced that Lloyd Blankfein, its CEO for the last 12 years, will step down from that role in September and retire at the end of the year, to be succeeded by David Solomon.

The company made its first bet on Goldman Sachs in 2008 after the collapse of Lehman Brothers. The move was seen as a vote of confidence in the investment bank as the financial crisis worsened.

Buffett told CNBC earlier this year that Berkshire had added 75 million shares of Apple and exited its stake in IBM in the first quarter. The company also increased its stake in St. Louis-based agricultural product giant Monsanto in the first quarter.

Berkshire cut its holdings in Phillips 66 by 24 percent, Wells Fargo by 1 percent, and Charter Communications by 8.7 percent, as of the end of June. It also cut shares in American Airlines by 2.8 percent, and United Continental by 3.7 percent.