Tag: technology

Some notes on the semiconductor industry

~ elityre ~ Leave a comment

In Spring of 2024, Jacob Lagerros and I took an impromptu trip to Taiwan to glean what we could about the Chip supply chain. Around the same time, I read Chip War and some other sources about the semiconductor industry.

I planned to write a blog post outlining what I learned, but I got pseudo-depressed after coming back from Taiwan, and never finished or published it. This post is a lightly edited version of the draft that has been sitting in my documents folder. (I had originally intended to include a lot more than this, but I might as well publish what I have.)

Interestingly, reading it now, all of this feels so basic, that I’m surprised that I considered a lot of it worth including in a post like this, but I think it was all new to me at the time.

  • There are important differences between logic chips and memory chips, such that at various times, companies have specialized in one or the other.
  • TSMC was founded by Morris Chang, with the backing of the Taiwanese government. But the original impetus came from Taiwan, not from Chang. The government decided that it wanted to become a leading semiconductor manufacturer, and approached Chang (who had been an engineer and executive at Texas instruments) about leading the venture. 
    • However, TSMC’s core business model, being a designerless fab that would manufacture chips for customers, but not designing chips of its own, was Chang’s idea. He had floated it to Texas instruments while he worked there, and was turned down. This idea was bold and innovative at the time—there had never been a major fab that didn’t design its own chips.
      • There had been precursors on the customer side: small computer firms that would design chips and then buy some of the spare capacity of Intel or Texas Instruments to manufacture them. This was always a precarious situation, for those companies, because they depended on companies who were both their competitors and their crucial suppliers. Chang bet that there would be more companies that would prefer to outsource fabbing, and that they would prefer to depend on a fab that wasn’t their competitor. 
      • This bet proved prescient. With the advent of chip design software in the 80s, the barriers to chip design fell. And at the same time, as transistor sizes got smaller and smaller, the difficulty of running a cutting edge fab went up. Both these trends incentivized specialization in design and outsourcing of manufacture.
  • Chang is sometimes described as “returning to Taiwan” to start TSMC, but this is only ambiguously correct. He grew up in mainland China, and had never been to Taiwan before he visited to set up a Texas Instruments factory there. He “returned” to start TSMC, only in the sense that the government of Taiwan was descended from the pre-revolutionary government of mainland China.
  • TSMC is the pride of Taiwan. TSMC accounts for between 5 and 25% of Taiwan’s GDP. (that’s a big spread. Double check!) The company is referred to as “the Silicon shield”, meaning that TSMC preempts an invasion of Taiwan by China, because China, like the rest of the world, depends on TSMC-produced chips. My understanding is that the impact of this defense is overstated, but it’s definitely part of the Zeitgiest.
  • Accordingly, the whole of Taiwanese society backs TSMC. Socially, there’s pressure for smart people to go into electrical engineering in general, and to work at TSMC in particular. Politically, TSMC pays very little taxes, and when it needs something from the government (zoning rights, additional power), it gets it.
    • Chip War quotes Shang-yi chang, head of R&D at TSMC:

“People worked so much harder in Taiwan,” Chiang explained. Because manufacturing tools account for much of the cost of an advanced fab, keeping the equipment operating is crucial for profitability. In the U.S., Chiang said, if something broke at 1 a.m., the engineer would fix it the next morning. At TSMC, they’d fix it by 2 a.m. “They do not complain,” he explained, and “their spouse does not complain” either.

  • Chips that have more transistors packed more densely, are better—able to do more computations. The “class” of a chip is called a “node.” 
  • A production process—all the specific machines and specific procedures, embodied physically in a fab used to make a class of chips. “The leading node” is the production process that produces the cutting edge chips to date (which have the most processing power and most efficient energy consumption). A new node rolls out about once every 2 years. Typically the old fabs continue operating, manufacturing now-less than cutting edge chips. 
  • Nodes are referred to by the size of an individual transistor on a chip, measured in nano meters. eg the in 1999 we were at the 130 nm node. But around 2000, we started running into physical limits to making semiconductors smaller (for instance the layers of insulation were only a few atoms thick, which meant that quantum tunneling effects started to interfere with the performance of the transistor). To compensate, chips started using a 3D design, instead of a 2D design. Since then the length of the transistor stopped being a particularly meaningful measure. Nodes are still referred to by transistor length (we’re currently on the 4 nm node), but it’s now more of a marketing scheme rather than a description of physical reality.
  • No one has ever caught up to the leading node. There used to be dozens of companies that could produce chips on the smallest scale allowed by the technology, but over the decades more and more companies have fallen back to fabbing chips that are somewhere behind the cutting edge. My understanding is that no one in history has ever overtaken the leaders from behind. Currently, TSMC is the only company that can produce leading node chips.
  • Semiconductor manufacturing is a weird mix of hyper competitive and a monopoly.
    • On the one hand, my impression is that semiconductors, along with hedge funds, are the most competitive industries in the world, in the sense that very tiny improvements on an “absolute” scale, translate into billions of dollars in profit. TSMC employs hundreds (?) of thousands of engineers working 12 or 14 hours a day, day in and day out, to squeeze out tiny process improvements. (I was told that everyone at TSMC universally says that it’s a very hard place to work.)
    • On the other hand, the winner of that brutal race to stay at the front of the pack effectively has monopoly pricing power. No company in the world, except TSMC, can produce leading node chips, and so can effectively charge monopoly profits for their manufacture. (From what I read in the TSMC museum, their actual profit margins appear to be around 50%.)
    • On the other hand, there’s unusually high levels of vertical coordination between companies. The supply chain is extremely complex, and each step depends on specifications both upstream and downstream.  Many of the inputs to chip production processes are distinctly not commodities. Very often, a crucial component of a sub process will be produced by only one supplier and/or used by only one customer.For this reason, the companies in the chip industry are unusually well coordinated. ASML can’t make a secret bet on an improved lithography mechanism, because it needs to be compatible with TSMC’s process flows.
      • So the industry as a whole decides which technological frontiers to invest in, so that they can all move together. 
      • Further, major companies in the supply chain are often substantial investors in their suppliers, because they are depending on those suppliers to do the R&D to develop components that will be crucial to their business 3, 5, or 10 years down the line.
        • For instance, very early EUV lithography R&D was researched by Intel, and Intel, Samsung, and TSMC all invested heavily in ASML, to make sure it could develop working EUV tech. ASML, in turn, manages a network of suppliers producing crucial high precision components, including investing in those suppliers to make sure they have the funding they need, and doing corporate takeovers if ASML decides it can manage a company’s production better than it can itself.
  • Jacob compared the chip industry to “a little bit of dath ilan on earth”. That sounds right to me. (Ironically, the semiconductor industry is the one industry on dath ilan that is not functioning like a dath ilani industry.
  • Robin Hanson claims that the rejection of prediction markets is because executives don’t really want the company to know the truth, because it undermines their ability to spin a motivating narrative. But this industry might be the one where results, and accurate predictions, matter enough, that the companies involved would embrace prediction markets.
  • From looking at videos of the inside of the fabs that were displayed in the TSMC museum, it looks like the whole process is automated. The videos don’t show workers operating machines. They show machines operating on their own—presumably with process engineers monitoring and adjusting their operation from a nearby room. Metal boxes, presumably containing wafers, are periodically lifted from the machines, transferred around the fab by robots attached to tracks on the ceiling, and then deposited in another machine.
  • The chip industry of every country that has a major chip industry does or did massively benefit from government intervention. 
  • As a rule of thumb, it takes 10 years to go from a published paper of technological process, to a usable scalable version. The papers published at conferences describe the manufacturing technology of 10 years in the future.

That no one rebuilt old OkCupid updates me a lot about how much the startup world actually makes the world better

~ elityre ~ Leave a comment

The prevailing ideology of San Francisco, Silicon Valley, and the broader tech world, is that startups are an engine (maybe even the engine) that drives progress towards a future that’s better than the past, by creating new products that add value to people’s lives.

I now think this is true in a limited way. Software is eating the world, and lots of bureaucracy is being replaced by automation which is generally cheaper, faster, and a better UX. But I now think that this narrative is largely propaganda.

It’s been 8 years since Match bought and ruined OkCupid and no one, in the whole tech ecosystem, stepped up to make a dating app even as good as old OkC is a huge black mark against the whole SV ideology of technology changing the world for the better.

Finding a partner is such a huge, real, pain point for millions of people. The existing solutions are so bad and extractive. A good solution has already been demonstrated. And yet not a single competent founder wanted to solve that problem for planet earth, instead of doing something else, that (arguably) would have been more profitable. At minimum, someone could have forgone venture funding and built this as a cashflow business.

It’s true that this is a market that depends on economies of scale, because the quality of your product is proportional to the size of your matching pool. But I don’t buy that this is insurmountable. Just like with any startup, you start by serving a niche market really well, and then expand outward from there. (The first niche I would try for is by building an amazing match-making experience for female grad students at a particular top university. If you create a great experience for the women, the men will come, and I’d rather build an initial product for relatively smart customers. But there are dozens of niches one could try for.)

But it seems like no one tried to recreate OkC, much less creating something better, until the manifold team built manifold.love (currently in maintenance mode)? Not that no one succeeded. To my knowledge, no else one even tried. Possibly Luna counts, but I’ve heard through the grapevine that they spent substantial effort running giant parties, compared to actually developing and launching their product—from which I infer that they were not very serious. I’ve been looking for good dating apps. I think if a serious founder was trying seriously, I would have heard about it.

Thousands of funders a year, and no one?!

That’s such a massive failure, for almost a decade, that it suggests to me that the SV ideology of building things that make people’s lives better is broadly propaganda. The best founders might be relentlessly resourceful, but a tiny fraction of them seem to be motivated by creating value for the world, or this low hanging fruit wouldn’t have been left hanging for so long.

This is of course in addition to the long list of big tech companies who exploit their network-effect monopoly power to extract value from their users (often creating negative societal externalities in the process), more than creating value for them. But it’s a weaker update that there are some tech companies that do ethically dubious stuff, compared to the stronger update that there was no startup that took on this obvious, underserved, human problem.

My guess is that the tech world is a silo of competence (because competence is financially rewarded), but operates from an ideology with major distortions / blindspots, that are disconnected from commonsense reasoning about what’s Good. eg following profit incentives, and excitement about doing big things (independent from whether those good things have humane or inhumane impacts) off a cliff.

Small cashflow software businesses might be over soon?

~ elityre ~ Leave a comment

[Epistemic status: half-baked musing that I’m writing down to clarify for myself]

For the past 15 years there’s been an economic niche, where a single programer develops a useful tool, utility, or application, and sells it over the internet to a few thousand people for a small amount of money each, and make a decent (sometimes passive or mostly-passive) living on that one-person business.

In practice, these small consumer software businesses are on the far end of a continuum that includes venture-backed startups, and they can sometimes be the seed of an exponentially scaling operation. But you only need to reach product-market fit with a few thousand users for a business like this to sustainable. And at the point, it might be mostly on autopilot, and the entrepreneur has income, but can shift most of their attention to other projects, after only two or three years.

Intend (formally complice), is an example of this kind of business from someone in my circles.

I wonder if these businesses will be over soon, because of AI.

Not just that AI will be able to do the software engineering, but that AI swarms will be able to automate the whole entrepreneurial process from generating (good) ideas, developing early versions, shipping them, getting user-feedback, and iterating.

The discourse already imagines a “one person-unicorn”, where a human CEO coordinates a company of AIs to provide a product or service. With half a step more automation, you might see meta-entrepreneurs overseeing dozens or hundreds of separate AI swarms, each ideating, prototyping, and developing a business. Some will fail (just like every business), but some will grow and succeed and (just like with every other business venture) you can invest more resources into the ones that are working.

Some questions:

  • How expensive will inference be, in running these AI entrepreneurs? Will the inference costs be high enough that you need venture funding to run an AI entrepreneur-systems?
    • Estimating this breaks down into roughly “how many tokens does it take to run a business (per day?)?” and “How much will an inference token cost in 2028?”
  • What are the moats and barriers to entry here? What kind of person would capture the gains to this kind of setup.
  • Will this eat the niche of human-ideated software businesses? Will there be no room left to launch businesses like this and have them succeed, because the space of niche software products will be saturated? Or is the space of software ideas so dense, that there will still be room for differentiation, even if there are 1000x as many products of this type, of comparable quality, available?

. . .

In general, the leverage of code is going to drop over the next 5 years.

Currently, one well-placed engineer will write a line of code that might be used by millions of users. That because there’s 0-marginal cost to replicating software and so a line of code written once might as well be copied to a million computers. But it’s also representative of the relative expense of programming labor. Not many people can write (good) code and so their labor is expensive. It’s definitely not worth paying $100 an hour for an engineer to write some software when you can buy existing off the shelf software that does what you need (or almost what you need) for $50 a month.

But, as AI gets good enough that “writing code” becomes an increasingly inexpensive commodity, the cost-benefit of writing custom software is going to shift in the “benefit” direction. When writing new software is cheap, you might not want to pay the $50 a month, and there will be more flexibility to write exactly the right software for your particular usecase instead of a good-enough off the shelf-version (though I might be overestimating the pickiness of most of humanity with regards to their software). So more people and companies will write custom software more of the time, instead of buying existing software. As that happens the number of computers that run a given line of code will drop, in the process.