Athens, Greece (CNN) They are images that will forever haunt Muhidin Hussein Muhamed: The boat he was about to board flipping over in seconds, trapping hundreds of its passengers inside.

Those who made it to the surface didn't know how to swim, he recalls. Many drowned, while others wearing life vests simply drifted away with the currents.

Muhamed, 32, was traveling with his six brothers. They were all lost.

"I survived. But I'm not going to forget my brothers," he tells CNN. "The survivors will not forget their families. It's not possible."

He was one of only 41 survivors -- 37 men, three women and a 3-year-old child -- from a boat believed to be carrying as many as 500 people, many of them asylum seekers from Somalia, before it capsized in the Mediterranean

Muhamed says his journey started on a boat that left the Libyan port of Tobruk on April 12 just before sunrise. He remembers the captain making a headcount: 200 people on board a 15-meter boat.

They were headed for Italy, but some 15 hours into the journey, the captain announced that the boat wasn't going to make it. He says their boat pulled up alongside a larger vessel overloaded with hundreds of people.

Panic

They threw ropes over to the larger boat, he says, and started to transfer passengers one by one. At first, it was an orderly process. But as night fell, people began to rush forward, fearful of losing a space on board.

The seas became rougher and the waves higher. They waited on the smaller boat with a group of women, afraid to move in the chaos.

"I told them 'Wait here. Let us wait. They are all shouting and in a hurry. It's dangerous.'"

He remembers the captain shouting at the crowds to stop rushing and balance the bigger boat. Then it flipped over.

"I saw it with my eyes," he says, using his hands to illustrate the capsized hull of the boat. "I'm watching the people. They can't swim. Watching them go up and down in the water."

The bigger boat was already overcrowded with about 500 people, including some from Muhamed's smaller vessel.

JUST WATCHED Lesbos: A Greek cemetery for the migrant dead. Replay More Videos ... MUST WATCH Lesbos: A Greek cemetery for the migrant dead. 02:24

'Muhidin, help me!'

Muhamed recalls seeing one of his brothers in the water.

"He shouted 'Muhidin, help me!' But I did not have the power to help him. I am only looking."

Muhamed says about 10 people managed to swim back to his boat. He believes there could have been more survivors, including his own brothers, but the boat captain restarted the engine and left them behind.

"I asked the captain 'Why you leave my brother there?' He said 'shut up' and he hit me. I want to kill him. But I can't kill him because who will drive us to Italy then?"

A short while later, Muhamed says, another empty boat arrived. The captain and the crew jumped aboard. They punched in a number on a satellite phone and then threw it to the remaining survivors.

"Call for help," was the last thing Muhamed remembers the captain saying before he sped away.

Adrift at sea

They drifted for three days at sea before they were rescued. Several ships passed them by, Muhamed says, before a commercial vessel from the Philippines responded to their distress call.

JUST WATCHED A year in the life of migrants Replay More Videos ... MUST WATCH A year in the life of migrants 01:47

"Without food and water we become so dry -- our skins, eyes - all of us. We were just hoping we were going to die. We waited for the minute the boat goes down."

The survivors were taken to Kalamata, Greece, and are now under the care of the UNHCR in a hotel in Athens.

The first thing Muhamed did was borrow five euros to call his wife and five children in Somalia. They cried when they heard his voice.

He then sent a text message to his mother informing her that his brothers had died.

Muhamed says he will apply for asylum in Greece and try to build a new life here. For now, he lives with the other survivors.

"We are like brothers now because we survived together. 41 out of 500," he says.

What about the Libyan captain who left him at sea?

"If I saw this man now, I would not wait to kill him," he says. "He killed us. He knew this boat could not carry all these people. I wish I could see him again with my own eyes."