Photo: Spencer Wells

Three years ago, Rory Ferreira was holed up in a converted storage shed in Echo Park, with a laptop and a space heater, trying to stave off panic attacks. The Chicago-born rapper, better known as Milo, was 22, and had moved to Los Angeles earlier that fall with three friends from Wisconsin, settling in Boyle Heights, just east of the L.A. River. Ferreira’s debut album, the contemplative A Toothpaste Suburb, was released that September through Hellfyre Club, the label and collective he’d been initiated into. The winter before, he’d toured with Open Mike Eagle, Nocando, and Busdriver, each a radically talented rapper in his own right, all a decade older than Ferreira. “If you really love rap, Hellfyre is where you wanted to be at the time,” he says today, sitting in a Cuban coffee shop just a few miles from that shed.

But plans went left. The rollout for A Toothpaste Suburb hit a series of hangups, and led to a falling-out with the label — one that might have been more painful on a personal level than a professional one. “You have to understand,” he says, “I’m a really dope rapper. But there are a lot of really dope rappers. To be a black kid in a predominantly white space, making this kind of rap, it’s like, how will I ever be heard by the people who need to hear me? And Hellfyre was that. They literally plucked me out of obscurity. And so I was down to ride with them forever. When that blew up, it messed with my head, it messed with my heart. Watching guys I admire have falling-outs over me felt horrible, man, just, like, a tremendous amount of guilt and shame.”

And so: panic attacks. Ferreira was trying to scrape by with little money and no safety net (“My parents don’t have a lot of money; I’d call them back and they’re like, “Hope you make it, dog!”) and would book sessions with Kenny Segal, the renowned L.A. producer, in the hopes that Kenny might make him lunch. He was physically hungry, and it began to seep into his music — he hit the beat a little harder, began to cuss more. “I felt like it matched my life,” he says of the tone he adopted immediately post-Suburb.

Even with the creative progression, L.A. was eroding Ferreira’s mental health. He and Safari Al, a close friend and collaborator who had made the move with him, found that shed in Echo Park, rented it from the homeowner, and converted it into a studio of sorts. It looked like a log cabin from the inside. With barely enough money and weed for one more week in L.A., they wrote, produced, and recorded a 20-minute record called (Boyle) and Piles EP, crediting themselves as RED WALL. Released less than three months after A Toothpaste Suburb, it marked a radical shift in tone for Ferreira: the self-deprecation melted away, he got sharper, more serrated. “We made that EP in less than a week — dropped it on the sixth day, got the overnight sales, dipped up to Oakland for a weekend, crashed at a homie’s spot, got those weekend sales, used that money to get back to Wisconsin,” he recalls. “That was our escape rocket.”

Ferreira was born in Chicago and grew up between there and New England. “Both my parents are real hustlers, real go-getters,” he says. “My mom is from 87th and Ada, the deep South Side, all the way, you-can’t-go-there South Side. That’s where my granny’s house is. My dad — I’ve seen him do shit that I can’t repeat, you know what I mean?” His parents divorced when he was young, but Rory describes the environment as stable, and he remains close with both of them. His mother, who works as the executive director of an anti-racism group, settled in Maine.

From an early age, rap was part of his vocabulary and worldview. His uncle is a noted battle rapper by the name of NIZM; Ferreira quickly gobbled up decades of hip-hop and internalized the rhythms and syntaxes. “Always, rapping has always been the language,” he says.

Ferreira eventually moved north, to Wisconsin, for high school and college— he was a philosophy student and remains a voracious reader, though, on his new album, he quips: “College is rote Zionism”— and began gravitating to hip-hop’s outer, avant fringes. By 2011, he was putting mixtapes online, beginning with I Wish My Brother Rob Was Here, an ode to a late friend. In 2013, a pair of EPs (Things That Happen at Day and Things That Happen at Night) and a mixtape, Cavalcade, made him something of a sensation in certain crevices of the internet. Milo records were celebrated for their academic tics and laconic freeform, and were frequently — in ways both loaded and not — compared to spoken-word. By the time he joined up with Hellfyre Club, he’d garnered a modest but cultish group of followers.

That style from the formative works reached its logical end on A Toothpaste Suburb, which was dense and expansive, and dealt with the disaffection of early, overeducated adulthood. At times, it sounds as if Ferreira is writing about himself during an out-of-body experience: a man watching his skeleton eat pizza off of an upturned Frisbee. Looking back at Suburb today, Ferreira speaks highly of the record, but recognizes certain threads of thought he would excise: “Humility services society. But if I live in a white supremacist society that hates me, it would never service me to be humble,” he says.

Hence the shift that came toward the end of his time in L.A. Along with the RED WALL EP, he began working frequently with Segal, whose approval helped Ferreira shed some of that self-effacing reflex. “I was starting to explore this really emotionally charged but stripped-down sound,” Segal says about the beginning of their working relationship. “Coming off of Toothpaste Suburb, which was a very sonically dense record, Rory was attracted to these beats that had a lot of space to play in, and it was one of those moments where it just clicked.”

They made a record together and decided, at the last minute, to bill it as the second official Milo LP, instead of as a collaborative side project. “I thought I was gonna jettison the Milo name and fade out,” says Ferreira, “but now I’m like, ‘Fuck that, I’m keeping my name.’ I wanna be a lifer in this. I want this catalogue that becomes undefinable because I’m just so stubborn.”

After the RED WALL EP, Ferreira and Al moved to Milwaukee and tried to decompress. “We knew Milwaukee because Al had gone to college there,” he says,” and I used to live at his apartment: he had a back closet room, and his roommates was cool with me living there for like a hundred bucks a month. We knew that back in Milwaukee the stakes were a centimeter high. We could get a whole crib for four hundred dollars. Let’s rebuild.”

The record with Segal, which was eventually dubbed So the Flies Don’t Come, got finished. It was Ferreira’s greatest work to that point. It’s lean; the writing is knotty and discursive, but delivered with a force he’d never before tapped into. The writing also excavating different parts of his psyche. It’s an exceptional record, augmented by appearances from Future Islands front man Samuel T. Herring, Mike Eagle, the legendary L.A. rapper and Freestyle Fellowship co-founder Myka 9, and Elucid, an avant-rapper and producer from New York. Flies ends with a dedication to Busdriver, whose tutelage, Rory says, was invaluable.

“Anything I’m even capable of doing in rap is because Busdriver has existed,” he says. When the pair toured together, Driver would take a structured approach to mining cities for inspiration. “When you’re with him in a new city, he’s like, ‘We’re gonna eat food we’ve never eaten, we’re gonna find new shit we’ve never been to; we’re only going to read; we’re only going to listen to this exclusive, unheard-of jazz thing; we’re going to watch these art-house movies; we’re going to meet these great musicians; we’re going to build on the inspiration of every night, these audiences loving us, and go home and make the best album we ever have.” Ferreira pauses, then summarizes the teachings: “You study all day, and then you get free at night.”

With time, Ferreira has come to the see the road as something other than thoughtful tourism: he sees it as a workshop. His latest record, Who Told You to Think??!!?!?!?!, is comprised of songs that he toured for months on end before he committed them to wax. (“I can’t remember the last time I did a set where the audience knew every song,” he says.) The process allows Ferreira — still morphing as a writer — to finely tune his cadences to their beats, to make phrasing sharper and more instinctive. That’s a process that’s also been helped by his growing hand in his own production. “When you make beats you start to unpack how you rap. As I get better at making beats, you see in my raps, I’m learning how to tuck everything in,” he says. Too Much of Life Is Mood, one in the series of self-produced records he releases under the name Scallops Hotel, was a turning point, he says. “That’s when I finally knew how to make a beat,” he says. “Before, beats got made, but I didn’t know how.”

That craftsmanship is also on display in his yet-to-be-released collaborative venture with Elucid. Under the name Nostrum Grocers, each artist raps and produces. Says Elucid: “We run on spontaneity and improvisation, and being free to let things settle naturally.” He goes on, “Milo has such an epic vision. He’s done or is doing everything that he said he would. Its hella exciting and inspiring to watch take form.”

What’s taking form is something that doesn’t have a particularly clear antecedent. Ferreira’s earliest work was airy and whimsical and almost unerringly polite, even when it dealt with heavy subject matter. Now his delivery is more vital and vibrant; Who Told You to Think??!!?!?!?! opens with the line: “Ghiath Matar is dead / roses are not armor,” Matar being a devoutly nonviolent Syrian activist who was tortured and murdered by his own government. Ferreira has absorbed the influence of his peers as well. There are pieces of Busdriver, the philosophical jewels that slip in while the artist is more obviously wrestling with form. But there’s also Open Mike Eagle’s willingness to warp tone, Serengeti’s Method-actor commitment to a given song’s premise, the caustic absurdity of mid-period Kool Keith. The new, still-unreleased Scallops Hotel record, Sovereign Nose of Your Arrogant Face, has a song (“fat tummy riff suite”) that seems to be structured almost entirely around its phonetics, yet finds time to exhale and ask, “Shit — who am I?”

After the Hellfyre situation fizzled, Ferreira and Al had formalized plans for a new record label: Ruby Yacht, named for the Rubáiyát of Omar Khayyám and for Rory’s grandmother, Ruby. It would be entirely self-owned and, in time, self-sustaining. Emboldened by his new music and encouraged by his father, Ferreira has released all his post-Suburb music through Ruby Yacht.

Toward the end of his time in Milwaukee, Ferreira got married. When he found out that his wife was pregnant, he realized it was time to leave: “Milwaukee is a majority-black city with an unemployment rate among African-Americans that’s like 56 percent, you know what I mean? We was one digit over from the Zip Code that has the highest incarceration rate of black males in America. That’s not a place I need my son coming up in. There’s a lot of black bodies, but not a lot of hope for black bodies there.”

And so they settled just off the coast of Maine, near where his mother lives. When he and his wife decided they didn’t want their son to be born in a hospital — “knowing the history of hospitals and black children and black women” — he got to work immediately, telling his booking agent he needed to make at least $10,000 before the baby boy was due. They made it happen, through a punishing and precise tour schedule. “I had a baby at 24 and paid for it with rap,” he says, beaming. “I felt and still do feel like I’m the fucking man.”

That ethos of art as trade has bled into Ferreira’s music. “My career has been making albums to accomplish life goals,” he says. “I need money, so I need to make a record.” On “poet (Black bean),” the opening, James Baldwin–interpolating song from Who Told You to Think??!!?!?!?!, he puts a finer point on it: “Fuck your notepad, wrote a poem with a tool kit.”

The craftsmanship continues. On top of the beats and label and owning his masters, Rory, Al, et al. are opening a record store in Biddeford, Maine. “Mom used to run an after-school program there,” Ferreira says. “Teach kids how to garden, keep ’em out of trouble.” Now, the town will have an independently (and black-) owned record shop, a little bit of brick and mortar built on the ephemeral.

“The thing I admire most about Rory is the way he forges his own path on his own terms,” Kenny Segal says. “Watching him take that same ethos as he quietly builds his indie rap empire, self-financed, beholden to no one, has been amazing.”

There are stakes to all of this. “To have a family — to have a black family in America today — I’m ready to die every day,” Ferreira says. “My son, my wife … man, if something happened to them … I’m not from a school of thought that’s: survival. There’s a lot of things I’d never live through.” After a beat of consideration: “I’m going to war, dog.”

But for the moment, things are still. There are no more panic attacks. Rory Ferreira and his family live on an island with a year-round population of about 800 people. “No bridge, bro,” he says, cackling. “You’ve gotta take a boat out to me.” His mother and sister each live on the island, too, and will come by the house to play with Rory’s 1-year-old son. “We’re about five or six miles across the harbor from America,” he says. “And we just look at the continent.”