I knew him as "Mr one-glove". The origins of his nickname were cloudy, but had to do with his legendary stinginess. He had just lost his company close to $1bn betting on mortgages. That company, facing massive losses from him and other traders, had only staved off bankruptcy because of the grace of the government.

It was late in 2008 and Mr one-glove had joined us at a bar, a group of disparate Wall Street traders united in an attempt to drink away a bad year. Near the end of the night, Mr one-glove leaned into the table of beers, and asked,

Do you think we will get paid well this year?

Mr one-glove was not somebody who trafficked in irony. Despite the massive loss, despite his company being bailed out, Mr one-glove didn't get fired, nor did he lose any of his wealth. No, Mr one-glove got paid well. Not by his standards, since he did not get a year-end bonus, just his salary of around $300,000. Mr one-glove was unhappy with that.

Nearly five years later, my life was very different. I left my Wall Street job to start a photo project documenting the lives of addicts in the Hunts Point section of the Bronx, New York City's poorest neighborhood.

A year ago I sat with one my new friends, Takeesha, at a small table in a visiting room in Rikers Island jail. Takeesha was in Rikers for possession of a needle and for intent to sell. Or maybe this time it was for prostitution. Takeesha is often in Rikers. Near the end of our conversation, Takeesha leaned into the table and asked,

Do you think I will get out this year?

Takeesha in her apartment in the Bronx section of New York City. Photograph: Chris Arnade

Takeesha did not end up getting out; she spent another two months, held a total of four months, for a variety of charges that almost all ended up being dropped.

Takeesha was raped at 11-years-old, by a family member, and pimped by another family member at 13. She ran away at 15. She has worked the last 25 years as a prostitute in Hunts Point and is addicted to both heroin and crack. Her story is not different from many homeless addicts I have gotten to know: childhood trauma, often sexual abuse, followed by rejection by the family, followed by addiction, and then, almost always for the women, followed by prostitution.

Mr one-glove is not that different from many whom I met on Wall Street over my 20 year career. He had upper middle class parents, was college educated, got an MBA, followed by a demanding high-paying job that became his identity.

What have I learned from knowing both Mr one-glove and Takeesha? Here's my one-line answer:

When you're wealthy you make mistakes. When you are poor you go to jail.

Yes, it is like comparing apples and oranges. That is the point though. We have built two very different societies with two very different sets of values. Takeesha was born into a world with limited opportunities, one where the black market has filled the void. In her world transgressions are resolved via violence, not lawyers. The law as applied to her is simple and stark, with little wiggle room.

Mr one-glove was born into a world with many options. The laws of his land are open for interpretation, and with the right lawyer one can navigate in the vast grey area and never do anything wrong. The rules are often written by and for Mr one-glove and his friends.

No, Mr one-glove did not break any laws. Not explicitly, although in 2008, he helped to bankrupt a company that helped to almost bankrupt the global economy. Rather, he spent his adult life moving numbers around on spreadsheets and betting on other numbers. Over his entire career, he has probably lost more money than he made, with the hole from 2008 swallowing any prior profits. For that he has been very richly rewarded. Tens of millions of dollars rewarded.

Takeesha has broken many laws, none open for interpretation. You usedrugs (well, not prescription drugs), you go to jail. You sell your body for sex, you go to jail. You can paint a narrative where young Takeesha shakes off her rape at 11, shakes off being sold on the streets at 13, and rallies. She finds the right foster family, takes advantage of the social services offered, and graduates from high school or at least gets her GED equivalent, then goes to college and moves well beyond her past.

Maybe she even ends up in banking, with her juvenile record forgiven. You can paint that story, but it's a fairy tale.

Mr one-glove would probably not approve of Takeesha. He felt everyone makes his own path in life, that raising his taxes to help the poor was encouraging a lifetime of sloth. To him, poverty was because of a lack of trying, a lack of working as hard as he had. Many successful professionals, who forget their benign youth, share that attitude.

Poverty and addiction have a thousand mothers, none of them sloth. Surviving the streets and hustling for the next fix is some of the hardest work around. Takeesha would probably say about Mr one-glove what another addict said with admiration, when hearing about my Wall Street life: "You made tricky money in a tricky world."

Mr one-glove eventually left his company. He is still working somewhere in finance, putting together another portfolio of mortgages using borrowed money.

Takeesha is still out on the streets, charging $50 for men to have sex with her. Or maybe she is in jail. I have to check weekly to see which it is.