At least four people were killed on Sunday and 63 injured, 11 of them critically, when a passenger train heading into New York City derailed in the Bronx.

President Barack Obama said his thoughts and prayers were with the friends and families of the victims of the derailment. A White House statement said the president was briefed on the accident on Sunday morning and would continue to stay in touch with New York officials throughout the day.

Two carriages were flipped onto their sides in the early morning incident, which occurred at 7.20am about 100 yards north of Spuyten Duyvil station on the Hudson line of the Metro-North Railroad. The New York Fire Department, which coordinated a rescue effort involving hundreds of firefighters, police and railway workers, said 63 people were hurt when the seven-carriage, southbound 5.54am service from Poughkeepsie to Manhattan's Grand Central Station left the rails on a sharp bend.

Three of the dead were killed as they were thrown from the train, according to FDNY chief of department Edward Kilduff, who said he believed all the passengers and train crew were accounted for after a rescue operation that included divers searching for survivors in the adjacent Harlem River, and that he thought the casualty count was unlikely to rise further. At a press conference three hours after the derailment, officials said numerous passengers had been taken away on stretchers to local hospitals.

"Three of the four fatalities were thrown as the train came off the track and was twisting and turning somewhat," Kilduff said. "The train is pretty beat up. There was substantial damage inside and a lot of personal possessions thrown around. We believe we've searched the entire area and we don't have any other victims we're aware of."

Eyewitnesses said they saw dozens of scratched and bloodied passengers leaving the wreckage, several holding ice packs to their heads. Frank Tatulli, a passenger in the front carriage, which came to rest just inches from the edge of the Harlem River, told a local TV station that he felt the train was travelling "a lot faster" than it usually would coming into the station.

An anonymous law enforcement official told journalists the train's driver had said he tried to apply the brakes before the bend but that they did not work.

State officials indicated that they had ruled out criminal activity or terrorism as a cause and that the incident appeared to be an accident. Andrew Cuomo, the governor of New York, visited the scene and told reporters that investigators from the National Transportation Safety Board would arrive later on Sunday.

"Four people lost their lives today, in the holiday season just after Thanksgiving, and they're in our thoughts and prayers," he said. "This is obviously a very tragic situation. The first order of business is to care for the people who were on the train. We are trying to save lives. New York is blessed with the best first responders I think anywhere in the country.

"In terms of causes, we don't know exactly what happened, The NTSB is on its way they'll do a thorough investigation and we'll wait to see what the NTSB say before speculating as to any causes."

Cuomo added that the train driver had been injured and was being cared for at a local hospital.

New York governor Andrew Cuomo arrives. Photograph: Andrew Gombert/EPA

Another passenger, Joel Zaritsky, told Associated Press he was travelling into the city for a dental conference.

"I was asleep and I woke up when the car started rolling several times," he said. "Then I saw the gravel coming at me, and I heard people screaming. There was smoke everywhere and debris. People were thrown to the other side of the train."

Thomas Prendergast, chairman of the Metropolitan Transportation Authority, said the speed of the train would be "one of the factors" investigators would look at, but that his first priorities were the injured passengers, then seeing if any service on the blocked line could be restored by the start of the working week tomorrow. Services were immediately suspended between Croton-Harmon and Grand Central. Amtrak services between New York City and Albany were also cancelled.

An MTA spokeswoman, Marjorie Anders, said the speed limit on the curve before Spuyten Duyvil is 30mph, compared with 70mph in the area before the curve. The train's data recorders should be able to tell how fast it was traveling, she said.

The stretch of line where Sunday's incident took place saw another derailment earlier this year, when a northbound freight train came off the track in July.

Dr David Listman of St Barnabas Hospital, one of four local medical facilities that treated victims, said the most seriously hurt passenger had a spinal cord injury. He also reported collarbone, leg and arm fractures, minor head injuries and many facial lacerations among those brought in. Railroad officials estimated later that between 100 and 150 people had been aboard the train, as witnesses and passengers who survived the crash came forward to share their experiences.

Hank Goldman, who lives close to the site, said: "I heard this horrible, whooshing sound. It was very disturbing, very loud. I got my binoculars and I couldn't believe my eyes that the train had jumped the tracks right here."

Passenger Amanda Swanson, from Harlem, said she was trapped in a carriage that landed on its side before she was pulled free by rescuers. "I was nodding off," she said. "I felt my body was at a 45-degree angle even though I was sat up in my seat, so I knew that didn't feel quite right. I could hear screeching metal over the music on my headphones.

"The windows broke out. The gravel came flying up in our faces. I really didn't know if I would survive. The train felt like it was on its side and dragging for a long time, the whole thing felt like slow motion."

She added: "I just kind of rode out the crash and tried to stay conscious and aware to do whatever I could to stay alive and uninjured. I could hear moaning and I could hear men communicating to each other that someone was stuck and they were trying to do what they could to make sure that person was all right.

"We couldn't get out, plus we didn't know exactly we were, how close to the river we were, even if there was any land for us to climb on to."

Westchester County officials warned that it would take at least two days for the track to be repaired, even after the wreckage had been removed. Aaron Donavan, a spokesman for the Metro-North Railroad, said the operation to clear the line would be extensive.

"The next step is to allow the NTSB to conduct a full and thorough investigation," he said. "When they give the all clear we can bring in heavy equipment. We need to get cars away from that area that will allow us to restore a service. Right now we don't have any trains running on any part of that track."

An injured man is evacuated from the scene. Photograph: Eric Thayer/Reuters

Donavan said it was "a black day" for the railroad, noting that Sunday's accident was the first passenger fatality in its three-decade history. It was also the region's second passenger train derailment in six months, after an eastbound train derailed in Bridgeport, Connecticut, on 17 May and was struck by a westbound train, injuring 73 passengers, two drivers and a conductor. Eleven days later, a track foreman was struck and killed by a train in West Haven, Connecticut.

Deborah Hersman, chairman of the NTSB, said a 15-strong team of investigators was due to arrive at Spuyten Duyvil by Sunday lunchtime.

"They will hit the ground running," she said. "They are focusing on operation, track, human factors, crash worthiness and survivability, any recorders, for example to see if we can identify any black box-type recorders that might be on board, looking at the signal system to see if there's any indication there.

"We've got a lot of work to do, we don't have a lot of daylight hours to do it but we'll do as much as we can as we get on scene to try to find out what happened so we can prevent it happening again."