Washington (CNN) John Kelly is marking a milestone : surviving one year as chief of staff to President Donald Trump. The job comes with the luxury of the second-best piece of real estate in the West Wing, but the biggest headaches.

But as Kelly on Saturday marks his one-year anniversary, the retired Marine Corps general is now just a shadow of the chief of staff who roamed the West Wing with sweeping authority at the start of his tenure.

Many of the protocols and processes he put in place to organize policymaking and rein in freewheeling access to the President have broken down. He no longer commands the same respect he once enjoyed from White House staff or the President.

And other aides -- notably new communications chief Bill Shine -- have assumed new stature as Trump increasingly looks to surround himself with a different set of voices.

As his influence and standing with the President have waned, Kelly's future has been the subject of intense speculation for months , with aides split on whether Kelly will depart before or after the fall elections. In the two weeks leading up to Trump's disruptive swing through Europe, senior aides predicted that Kelly had days or hours left. Those same aides now think the ensuing chaos may have helped Kelly hang on a little longer.

Kelly has held increasingly fewer senior staff meetings -- once daily occurrences which have been whittled down to weekly gatherings -- and has exerted less control over who talks to Trump, as well as how and when outsiders connect with the President.

Trump himself has not given Kelly a timeline for leaving and has told people it will be up to Kelly to set a departure date. But the question across the White House is less will Kelly leave, but when.

Lately, Trump has fluctuated between venting about Kelly to certain aides while reassuring others that he is pleased with him. He went on a profanity-laced tirade against his chief of staff in recent weeks, a source who heard the conversation told CNN, but the next day he was telling others Kelly was doing a fine job.

"It's no longer a question of the President blowing up and firing Kelly -- that would have happened already," one senior White House official said. "The status quo seems fine with the President, until he can find someone to replace him."

In conversations this week with more than 20 administration officials and people who talk to the White House on a regular basis, a portrait of a diminished Kelly emerges. He has little direct role in two of the biggest subjects hanging over the White House: the midterm elections and the Russia investigation.

The White House and Kelly declined CNN requests for comment.

Less and less present

A year after taking the job, Kelly often leaves his Manassas, Virginia, home later in the morning than he once did and occasionally arrives home earlier. His Secret Service detail has been spotted outside the White House gym during the lunchtime hour as Kelly takes a break from the office to work out.

Kelly still travels regularly with the President, including sometimes on Trump's weekend visits to his golf resort in New Jersey. He was spotted there one July weekend, dressed in shorts and a T-shirt, mounting an orange ladder to help extract an enormous American flag that had become entangled in wire.

With every passing month, officials say, Kelly has spent less and less private time with the President. And at least some days, Kelly sees Trump no more than some mid-level aides and far less than new advisers like Shine, the former Fox News executive who is at the President's side more than anyone.

When Trump was jetting home last week from Helsinki, the site of his maligned summit with Russian President Vladimir Putin, it was Shine at the head of the table in Air Force One's conference room managing discussions of how to triage the disaster. Kelly sat to his left.

"(Trump) doesn't want someone telling him what to do," one White House official said. "He wants someone to make him look good doing it."

That task has increasingly fallen to Shine, who has begun applying his knowledge of camera angles and lighting to Trump's events. After the President complained about the television lighting during speeches, suggesting it made his hair glow, Shine worked with aides to develop alternate set-ups. He also recently had aides install neutral-density filters on two windows in the Cabinet Room in order to modify the sunlight ahead of a video recording.

Unlike Kelly, who worked swiftly to impose changes when he entered the job a year ago, Shine has been more watchful. Some officials said he viewed Kelly's approach, which aggravated the President, as a cautionary tale.

If Kelly entered the West Wing with an iron fist, one official in the West Wing noted, Shine has taken the opposite approach, quietly observing the power structure first. But multiple sources close to Shine say there is no doubt he will make his mark when he feels the time has come. Shine's presence has already irritated some officials, such as Kellyanne Conway, who was once encouraged to take on a similar communications role, people who have heard her voice frustration told CNN.

The President doesn't hold animosity toward Kelly in the way he has for several other aides he has fired, several officials said, but he is often not at the center of the West Wing activity. In conversations with friends, Trump has praised Kelly's military background but noted that because he spent most of his career in the ranks of the Marine Corps, he has little sense of the political world. Himself a newcomer to politics, Trump has regularly questioned the political instincts of his senior-most aides, believing his own intuition surpasses that of seasoned operatives.

A person close to Kelly said the retired Marine general has no plans of stepping down until the President asks him to do so. Asked whether he still enjoys the job, this person, speaking on condition of anonymity, said: "I'm not sure he ever did."

Whether Kelly has ultimately been successful bringing order and discipline to an extraordinarily chaotic White House is an open question, but his firm grip has left an unmistakable mark: He saw to it that Omarosa Manigault , the former reality star who entered the White House as a top-level adviser, and other aides viewed as problematic, were removed.

Controversial tenure

His tenure has nevertheless been pocked with controversies, all of which he managed to survive. Weeks after taking over for Reince Priebus, his predecessor who was unceremoniously fired over Twitter while he sat on a rainy tarmac, Kelly was faced with the Charlottesville controversy. He was photographed looking grim-faced in the lobby of Trump Tower as the President declared there were "good people" on both sides of the racist violence.

Since then he's created some of his own dust-ups. He insulted Rep. Frederica Wilson, D-Florida, using inaccurate information, later declaring he would "never" apologize. He declared some of those eligible for DACA protections "lazy." He was quoted by NBC News calling the President an "idiot," an accusation the White House vehemently denied, and a senior administration official later told CNN that Kelly had also privately called Trump "unhinged."

Perhaps most damaging, he woefully bungled the situation involving former staff secretary Rob Porter, who was accused by two of his ex-wives of abuse. Kelly's shifting accounts of the matter after the fact caused his credibility inside the West Wing to plummet, and it never truly recovered, according to officials. Kelly's highly criticized handling of the Porter controversy was an inflection point in his tenure, and some of his internal relationships became strained in the months that followed the former staff secretary's ouster.

That included Kelly's ties to members of Trump's family, who saw him as attempting to limit their access to the President. When he started in the job, senior advisers Jared Kushner and Ivanka Trump made sure to announce publicly they would report to Kelly, and not directly to the President, who is Ivanka's father. Over the course of the year, however, the arrangement soured.

"They have the worst working relationship I have ever seen," an administration official said of the dynamic between Kelly and Ivanka Trump.

"John last lost the (Trump) family and John has lost the staff," another source close to the White House said.

One source close to the President said Trump has complained about Kelly's manner in clashing with other aides, calling him a "hothead."

"He uses all the terms that would equivocate that he's done with the guy," the source said.

Kelly still doesn't read Twitter unless shown one of the President's messages. And he still doesn't watch cable television in his office, gazing instead on the Abraham Lincoln portrait he hung in place of the large TV screen above his fireplace.

Departure date unknown

JUST WATCHED Kelly warned Trump about talking to witnesses Replay More Videos ... MUST WATCH Kelly warned Trump about talking to witnesses 02:47

It's not clear how Kelly's eventual departure -- whether it comes in a matter of days, weeks, or months -- might affect the structure he sought to impose. In recent weeks, Trump has told confidants he misses the chaos of the early days of his administration, which mirrored his style as a real estate impresario with its large cast of advisers and competing internal factions.

Still, Trump has looked to two experienced political hands as possible Kelly successors, polling friends and advisers about the possibility of tapping Office of Management and Budget Director Mick Mulvaney or Vice President Mike Pence's chief of staff Nick Ayers to take the reins. Mulvaney, whose sharp wit has impressed Trump, is seen as the strongest candidate, three sources inside the West Wing told CNN, though Trump has remarked that he likes Ayers' hair.

Yet there remains no obvious successor for Kelly, which is why many believe Trump has not replaced him yet. Even as Trump is mulling Kelly's successor, one Republican close to the White House stressed that Trump was not looking to replace Kelly imminently.

"The President polls people on everybody. He does that constantly," the Republican said. "It has nothing to do with job openings ... He's constantly taking a check on different people."

Kelly has raised eyebrows in the White House press and communications offices by occasionally tapping Zack Fuentes, his deputy, to handle some media issues related to himself. For example, Fuentes spoke on the record to some media outlets when, in late April, the chief of staff became the target of an explosive leak after NBC published allegations that Kelly had called Trump an "idiot" behind closed doors -- a claim Fuentes rushed to deny.

In the Rose Garden on Wednesday, as Trump made an announcement on tariffs, Kelly stood to the side, well out of camera view, and took in the scene. When the President finished speaking, ignoring shouted questions, Kelly followed him into the Oval Office.