I was born on 148th Street in 1965, and from then until the late 1990s it never dawned on me to live anywhere other than New York City. When I lived on 14th Street in the late 1980s, I paid $140 a month to share an apartment with a bunch of other odd and dysfunctional musicians and artists. AIDS, crack and a high murder rate kept most people away from New York back then. But even though it was a war zone, or perhaps to some extent because it was a war zone, Manhattan was still the cultural capital of the world. Of course everything's changed since. New York has, to state the obvious, become the city of money. People say your rent should be 30% of your salary; in Manhattan today, at least for many people, it feels like it hovers around 300%.

The gradual shift in New York's economic fortunes and mores reminds me of the boiling frog theory. If you take a frog and throw it in a pot of boiling water, the frog will do everything in its power to escape. But if you place a frog in room-temperature water and slowly raise the heat, it will boil to death without realizing it's dying. (I truly hope this theory will never actually be tested.) That's what happened to me in New York. I was so accustomed to the city's absurd cult of money that it took me years to notice I didn't have any artist friends left in Manhattan, and the artists and musicians I knew were slowly moving farther and farther east, with many parts of Brooklyn even becoming too pricey for aspiring or working artists.

During the 1990s, thanks to the cessation of the crack epidemic, New York became increasingly safer and more affluent, and less artist-friendly, but it was still the place I wanted to call home. What happened next reminded me of Gremlins: you're not supposed to feed the gremlins after midnight or they metastasize. Gremlin midnight came to New York sometime in the mid-1990s. I realized then that most people I met in New York were happily observing and talking about culture, but not necessarily contributing to it. It seemed New York had entered the pantheon of big cities that people visit and observe and patronize and document, but don't actually add to, like Paris. No one goes to Paris imagining how they can contribute to the city. People go to Paris thinking, "Wow, I want to get my picture taken with Paris in the background." That's what New York became, a victim of its own photogenic beauty and success.

And, to again state the obvious, New York is exclusively about success – it's success that has been fed steroids and vitamin B. There's a sense that New Yorkers never fail, but if they do, they're exorcised from memory, kind of like Trotsky in early pictures of the Soviet Communist Politburo. In New York, you can be easily overwhelmed by how much success everyone else seems to be having, whereas in LA, everybody publicly fails at some point – even the most successful people. A writer's screenplay may be turned into a major movie, but there's a good chance her next five screenplays won't even get picked up. An actor may star in acclaimed films for two years, then go a decade without work. A musician who has sold well might put out a complete failure of a record – then bounce back with the next one. Experimentation and a grudging familiarity with occasional failure are part of LA's ethos.

Maybe I'm romanticizing failure, but when it's shared, it can be emancipating and even create solidarity. Young artists in LA can really experiment, and if their efforts fall short, it's not that bad because their rent is relatively cheap and almost everyone else they know is trying new things and failing, too. There's also the exciting, and not unprecedented, prospect of succeeding at a global level. You can make something out of nothing here. Take Katy Perry. She's a perfectly fine singer who one minute was literally couch surfing and the next was a household name selling out 50,000-capacity stadiums. Or Quentin Tarantino, one minute a video clerk, the next minute one of the most successful writer/directors in history. Los Angeles captures that strange, exciting and at times delusional American notion of magical self-invention.

I don't want to create a New York-LA dichotomy, because both cities are progressive and wonderful, and there are clearly many other great American cities. Artists aren't just leaving New York for LA – they're also going to Portland, Minneapolis, Miami, Atlanta, Philadelphia and countless other places. And, as an aside, I don't know why they aren't moving to Newark. It's 15 minutes away from Manhattan and remarkably cheap. I think it's the unwarranted New Jersey stigma that unfortunately keeps people from crossing the Hudson. People would rather move to the worst part of Brooklyn and still have the magical "NY" in their address. That single consonant on their mail – "Y" as opposed to "J" – seems to keep people from making that 15-minute trek to Newark.

Plenty of other cities in the United States and abroad are, of course, interesting and beautiful, but I moved to LA due to its singular pre-apocalyptic strangeness. It seems equally baffled and baffling, with urban and suburban and wilderness existing in fantastic chaos just inches away from one another. There's no center to L.A, and in many ways it's kind of a fantastically confused Petri dish of an anti-city. If you're in New York, Brussels, London or Milan, you're surrounded by a world that has been subdued and overseen by humans for centuries, sometimes for millennia. They're stable cities; and when you're in an older city you feel a sense of safety, as if you're in a city that's been, and being, well looked after. You feel like most well-established and conventional cities know what they're doing. LA, on the other hand, is constantly changing and always seemingly an inch away from some sort of benign collapse.

If you look at some of LA's patron saint artists, like Robert Irwin and James Turrell, their work is about the vast, unknowable and at times uncaring strangeness of the world we live in – not the human world, but the natural world. And it makes sense: nature, with all its empty, otherworldly expanses, is the constant, hulking neighbor to Los Angeles. The moment you leave LA, you're in a desert that would most likely kill you if you left your water bottle at home. For southern California, humanity is the weird exception, not the rule.

LA's strange environment and contradictions have also shaped the sound of my recent music. My last album, innocents, is a fairly quiet and domestic record, almost like whistling in the dark in the face of the vast maw. And if I were more of a weird, brave artist – and maybe I'll do this in the future – I would move out into the desert and let its vastness and uncaringness inform what I'm doing. So far I have made quiet sounds as something of a retreat into my home.

I should admit I have an ulterior motive in promoting LA. I'm so outspoken about my love for the city because I want my friends to move here. When friends from New York ask me why I moved here, I say, somewhat elusively, "David Lynch lives here, there's the Museum of Jurassic Technology, rents are relatively cheap, and I can run around outside 365 days of the year. Oh, and there are still recording studios in LA." And I'm always sending them real estate listings, especially when they complain about the cost of real estate in New York (in other words: constantly). If the weather is bad in New York in February, I'll also be a clichéd Angeleno and send them a picture of me outside by the pool. Not just because I'm an asshole and I like shameless Schadenfreude, but also because I think they'd be happier here, especially those who are trying to start families.

Even friends of mine who are making very good salaries of $150,000 a year feel dirt-poor when they picture raising kids in New York. My friends who are trying to start families in New York have given up on simple things, like ever having a 50-square-foot backyard for their kids. A good domestic life is simply more attainable here, as LA has both invented and perfected that strange balance between the suburban and the apocalyptic. But let's be clear, I have an agenda: I want my friends to join me here so I can sit with them by my pool in February and look at the weather updates for the rest of the Western world and feel smug together.

• This piece was originally published on 3 February 2014 on Creative Time Reports. Read it here.