In Sofia Coppola's 2006 film of the life of Marie Antoinette, there is a scene where an entourage of palace jeunes filles sweeps through a ball at which the set and costumes are period, but the music and manners are straight out of a modern dance club. The proposition seems to be that an elite few were able to put a toe into the future to experience what is ordinary today.

Something like that went on in the Silicon Valley I knew in the 1980s. The debates and dilemmas that occupy a generation today appeared in miniature before there was an internet. We took our anticipation of the internet deadly seriously, to the point where it seemed already real. Thus I have experienced the internet age twice.

Experiencing the internet in reality is different - and even bizarre, because although it seemed reasonable to expect the thing to come about, it is still uncanny that the reasoning was right. It feels as though we got away with something we shouldn't have done.

The internet arrived from two directions, one top-down and the other bottom-up. Initially computers and computer networking were both developed in military and government labs. The way you experienced computation from the 1960s often reflected this point of origin, with early computer companies such as IBM exuding a grey, regimented stoniness in order to appear seductive to their patrons.

In the 1970s, a small market emerged for hobbyist computers. You could build your own little box with blinking lights that you could program by flipping lines of switches on the front panel. That's all you could do at first, but oh, the ecstasy to be able to touch your own computer, if you had an inkling of where it all could lead.

A culture grew up around these hobbyist machines centred in Silicon Valley, and spawned the personal computer market - with Microsoft launching in 1975 and Apple in 1976. The centre of gravity split: the stony grey opposite delirious hippies and faux revolutionaries.

The turbulent confluence between top-down and bottom-up continues to this day. Internet start-ups sprout like garage bands. Most die, but a few explode into national-scale empires, as in the case of Facebook. Dreary top-down institutions such as wireless carriers maintain their lofty entitlements, though occasionally they drain away, like the old music business. I used to be partisan, favouring the bottom-up approach, but now I appreciate the balance of tides, because all kinds of power should be checked.

My first encounter with Silicon Valley was at the end of my teens, which was also the end of the 1970s. The world seemed carved into zones according to the degree of magic available. The highest magic was found in nexuses of hippie exuberance such as the beach town of Santa Cruz, California, where pearlescent rainbows covered everything and even the most mediocre musicians could effortlessly invent melodies superior to almost anything heard since. Young, creative people with any sense of ambition tended to be drawn to these places like weight to gravity, but by the time I arrived the magic was receding.

The overwhelming explanation we held of our time and place was that we had been born too late to experience the one true orgasm of meaning, the 1960s. Young people who felt jilted by life because of a slight error in timing found solace in a twisted calculus of punk humour. An alternative to the Santa Cruz-type El Dorados of bohemia were the zones of brazen, barren reality: remote and violent desert towns, impoverished villages in Mexico, or tenements in New York City.

The most deficient places - condemned by hippies and punks alike - were the suburbs, the places of the conventional parent: an artificial world ruled by Disney and McDonald's.

I did not arrive at this suspect ontology naturally, having grown up in a way that was both gritty and bohemian. My father and I couldn't afford a home at one point, when I was 11, so we lived in tents on cheap land while building a crazed, geometric, spaceship-like house in a rough corner of southern New Mexico. I adapted to the flight from the suburbs because this seemed the ticket into the social world of my peers in that era. I well remember how my heart sank when I later realised that eco­nomic circumstances left me no choice but to force my old jalopy over the mountain pass that insulated dewy, arousing Santa Cruz from soul-killing, blandifying Silicon Valley, which was situated in, of all places, a suburb.

The mountain ridge that separates Silicon Valley and the town of Palo Alto from the ocean keeps out the famed fog of northern California in the summer. This has always made it an elite getaway from San Francisco, but to me Silicon Valley's light looked incomplete and made me feel remote and depressed - so close to the ocean, but without its full light.

I despaired at the time that I had failed to earn enough to be able to remain at the fulcrum of hippie truth, but I was to learn, slowly, that I was moving from one narcissistic category war to another. Instead of hippies v suburbs, I enlisted in the turf war between nerds and - well, the opposite doesn't have a name. A sort of muggle: the fool who doesn't realise that he lives in a cocoon and serves only as a battery to power the action; a person who fails to understand that the world is an information system, and that life is programming.

Having moved from one kind of nonsense to another eventually helped me learn to be sceptical of both.

Palo Alto was nicknamed "Shallow Alto" by the hippie hackers, who felt that living there was a sell-out, a sign of failure. And yet, one by one, we gave in and entered an alternate, infinitely better-funded elite club. The place was much more than a suburb, naturally. A little more than a century earlier, there had been a Native American culture there, but it was murdered and erased, so little more can be said. Layers of mutually indifferent histories were then overlaid on to this, awaiting the final washout by Silicon Valley culture.

A trace of the Spanish colonial period remained in the odd old adobe mansion; evidence of black immigration from earlier in the 20th century lay in the shocking, violent twin to Palo Alto, East Palo Alto; fruit orchards swept to the horizon in some directions and utilitarian grids of simple wooden buildings testified to the well-ordered conception of railroad towns and military bases.

But the hackers would take over. What a strange society nerds make. In 1996 Oliver Sacks published a book called The Island of the Colour-blind, about a place where so many people cannot see colour that it becomes the norm. In the same way, the society of computer nerds is nerdy not in comparison to a centre, but as a centre. Our nerdy world, which from an outsider's perspective might seem slightly askew, even tilted a touch into Asperger's syndrome, was and is our centre. The rest of the world seemed hysterical, irrational and confused by the surface aesthetics of things, somehow failing to grasp the numerical, causal, core truth underpinning events and the problem-solving purpose of reality.

I kept my concerns about the light of Palo Alto to myself and "passed", which was, happily, not hard for me. Certain kinds of math and programming come on strongest when you're young, and I could program the hell out of a computer in those days. Then and now, technical credibility is the ultimate membership card in Silicon Valley, and it is one of the reasons I still love the place. The billionaire company starters - and I won't name names because it's all of them - still get a little insecure and feel a need to preen when they're around top hackers.

The overlap between the late stages of hippie bohemia and the early incarnations of Silicon Valley was often endearing. There was a sense of justice in the way that males who had been at the bottom of the social ladder in high school were on track to run the world. Greasy cottages with futons on the floor, with dustings of pot and cookie crumbles rubbed into cheap oriental rugs, a carnage of forgotten dirty clothes in the corner, empty refrigerators and tangles of thick grey cables leading to the huge computer monitors and the hot metal cabinets where the silicon chips crunched. Asymmetrical, patchy beards, shirts part tucked, prescriptions for glasses powerful enough to find life on a distant planet. This was the new model of hippie nerd, supplanting the ascetic fellow with the pocket protector.

There were precious few girl nerds at the time. There was one who programmed a hit arcade game called Centipede for the first video game company, Atari, and a few others. There were, however, extraordinary female figures who served as the impresarios of social networking before there was an internet. It still seems wrong to name them, because it isn't clear if I would be talking about their private lives or their public contributions: I don't know how to draw a line.

These irresistible creatures would sometimes date alpha nerds, but mostly brought the act of socialising into a society where it probably would not have occurred otherwise. A handful of them had an extraordinary, often unpaid degree of influence over what research was done, which companies came to be, who worked at them and what products were developed.

That they are usually undescribed in histories of Silicon Valley is just another instance of what a fiction history can be. The advent of social networking software and oceans of digital memories of bits exchanged between people has only shifted the type of fiction we accept, not the degree of infidelity.

In retrospect, I cringe to think how naive and messianic the tech scene became amid all the post-1960s idealism. The two poles of San Francisco Bay Area 1960s culture - psychedelic hippies and leftist revolutionaries - became the poles of early computer culture.

In 1974, the philosopher Ted Nelson, the first person to propose and describe the programming of something like the web, published a large-format book composed of montages of nearly indecipherable small-print snippets flung in all directions, called Computer Lib/Dream Machines. If you turned the book one way, it was what Che Guevara would have been reading in the jungle if he had been a computer nerd. Flip it upside down, and you had a hippie-wow book with visions of crazy, far-out computation.

In fact, the very first description of the internet in any detail was probably E M Forster's The Machine Stops from 1909, decades before computers existed: "People never touched one another. The custom had become obsolete, owing to the Machine." It might still be the most accurate description. How Forster did it remains a mystery. Later, in the 1940s, the engineer Vannevar Bush wrote "As We May Think", an essay imagining a utilitarian experience with a computer and internet of the future. Bush's essay is often cited as a point of origin, and he even delved a little into how it might work, using such pre-digital components as microfilm.

But Ted Nelson was the first person, to my knowledge, to describe how you could implement new kinds of media in digital form, share them and collaborate. Ted was working so early - from 1960 onwards - that he couldn't invoke basic notions such as storing images, and not just text, because computer graphics had not been described yet. (The computer scientist Ivan Sutherland saw to that shortly.)

Ted was a talker, a character, a Kerouac. He was more writer than hacker, and didn't always fit into the nerd milieu. Thin, lanky, with a sharp chin and always a smile, he looked good. He came from Hollywood parents and was determined to be an outsider because, in the ethics of the times, only the outsiders were "where it's at". He succeeded tragically, in that he is not as well known as he ought to be, and it's a great shame he was not better able to influence digital architecture directly. He lives today on a houseboat in Sausalito, California, one of the other luminous, numinous nodes of Bay Area geo-mythology.

The hippest thing in the late 1970s and early 1980s was to form a commune, or even a cult. I remember one around the Haight-Ashbury neighbourhood of San Francisco which fashioned itself as the Free Print Shop. Members printed lovely posters for "movement" events in the spectral, inebriated, neo-Victorian visual style of the time. (How strange it was to hear someone recommended as "part of the movement". This honorary title meant nothing beyond aesthetic sympathy, but there was an infantile gravity to the word "movement", as though our conspiracies were consequential. They never were, except when computers were involved, in which case they were more consequential than almost any others in history.)

The Free Print Shop made money doing odd jobs, it included women and it enacted a formal process for members to request sex with one another through intermediaries. This was the sort of thing that seemed the way of the future and beckoned to computer nerds: an algorithm leading reliably to sex! I remember how reverently dignitaries from the Free Print Shop were welcomed at a meeting of the Homebrew Club at Stanford and other such venues where computer hobbyists shared their creations.

Ted had a band of followers or collaborators; it would have been uncool to specify what they were. They sometimes lived in a house here or there, or vagabonded about. They broke up and reconciled repeatedly, and were perpetually on the verge of presenting the ultimate software project, Xanadu, in some formulation that would have been remembered as the first implementation of the internet. Xanadu was a manifesto that never quite manifested.

If my tone has not been consistently reverent, please know that I am not cynical when it comes to my praise of Ted Nelson's ideas. As the first person on the scene, he benefited from an uncluttered view. Our huge collective task in finding the best future for the internet will probably turn out to be like finding our way back to where Ted was at the start.

In his conception, each person would be a free agent in a universal online market. Instead of separate stores of the kind run by Apple or Amazon, there would be one universal store, and everyone would be a first-class citizen, both buyer and seller. You wouldn't have to keep separate passwords or accounts for different online stores. That's a pain, and it guarantees that there can't be too many stores, thereby re-creating the kind of centralisation that shouldn't be inherited from physical reality.

This is an example of how thinking in terms of a network can strain intuition. It might seem as though having only one store would reduce diversity, yet it would increase it. When culture is privatised, as has happened recently online, you end up with a few giant players - the Googles and Amazons. It's better to put up with the rancour and pain of a single community, of some form of democracy, than to live in a world overseen by a few forces you hope will be benevolent. The stress of accommodation opens cracks from which brilliance emerges.

Ah, there it is - my idealism, still in your face after all these years. Silicon Valley remains idealistic, if sometimes narcissistic. We refer to uprisings in the Middle East as "Facebook revolutions" as if it's all about us. And yet, look. We code and scheme through the night, and then genuinely change the whole world within a few short years, over and over again. What other bunch of oddballs can say that?

Much has changed. Silicon Valley now belongs to the world. In a typical nerd cabal you will find recently arrived Indians, Chinese, Brits, Israelis and Russians. What is strangest in the recent waves of young arrivals in Silicon Valley is that they tend no longer to be downtrodden geniuses rejected in the playing of social status games, but sterling alpha males. Legions of perfect specimens seem to have grown up in manicured childhoods, nothing scrappy about them. When children started to be raised perfectly in the 1990s, chauffeured from one play date to the next, I wondered what world they would want as adults. Socialism? Facebook and similar designs seem to me continuations of the artificial order we gave children during the boom years.

Now we are entering a period of diminishing middle classes and economic dimming. What will Silicon make of this? Poorly conceived computer networks played central roles in many of our more recent troubles, particularly the 2008 financial crisis. Such tactics as high-frequency trading just pluck money out of the system using pure computation and without giving anything back.

Can we adjust the world, make it happier, merely by reprogramming computers? Perhaps. We continue to twiddle with human patterns from our weird suburb. Maybe, if we are able to echo the ancient idealism of those early days, we will do some good as the software grows.

Jaron Lanier is the author of "You Are Not a Gadget: a Manifesto" (Penguin, £9.99)