Washington (CNN) In the wake of several high-profile scandals -- including allegations of sexual assault and cocaine use against Navy SEAL team members -- the admiral overseeing the Navy SEALs has ordered a series of immediate changes to address what he calls a drift from "Navy core values of Honor, Courage, and Commitment."

"Our Force has drifted from our Navy core values of Honor, Courage, and Commitment and the tenets of our Naval Special Warfare Ethos due to a lack of action at all levels of Leadership," Rear Adm. Collin Green, the commander of Naval Special Warfare Command, wrote in a memo to major subordinate commanders

A US Navy official says Green is trying to strongly emphasize to the SEALS their loyalty must be to the values of ethics and honor and not first to their SEAL teammates. It's about SEAL leadership, not a loyalty first to other SEALS. Serving the country is the first and only priority, the official said.

Green said a "portion" of Navy SEALS is "ethically misaligned with our Culture," saying that "the root of our problem begins with members who fail to correct this behavior within their sphere of leadership and prioritize this misalignment over the loyalty to Navy and Nation."

"This erodes the foundation of trust we have earned with our leaders and the American people. All Hands will address this issue with urgent, effective and active leadership. This drift ends now," he added.

Read More