Build Your Startup With AI

Ben Horowitz
It is kind of the darkest side of capitalism. When a company is so greedy, though, they're willing to destroy the country and maybe the world to like just get a little extra profit. And they do it like the really kind of nasty thing is they claim, oh, it's for safety. You know, we've created an alien that we can't control, but we're not going to stop working on it. We're going to keep building it as fast as we can, and we're going to buy every freaking GPU on the planet.

But we need the government to come in and stop it from being open. This is literally the current position of Google and Microsoft right now. It's crazy. The content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investor or potential investors in any a 16 z fund. Please note that a 16 z and its affiliates may maintain investments in the companies discussed in this podcast.

Marc Andreessen
For more details, including a link to our investments, please see a 16 hey, folks, welcome back. We have an exciting show today. We are going to be discussing the very hot topic of AI. We are going to focus on the state of AI as it exists right now in April of 2024. And we are focusing specifically on the intersection of AI and company building.

So hopefully this will be relevant to anybody working on a startup or anybody at a larger company. We have, as usual, solicited questions on X, formerly known as Twitter. And the questions have been fantastic. So we have a full lineup of listener questions and we will dive right in. So, first question.

So three questions. Three questions on the same topic. So Michael asks, in anticipation of upcoming AI capabilities, what should founders be focusing on building right now? Gwen asks, how can small AI startups compete with established players with massive compute and data scale advantages? And Alistair McLeay asks, for startups building on top of OpenAI, etcetera.

What are the key characteristics of those companies that will benefit from future exponential improvements in the base models versus those that will get killed by them? So let me start, I'll just start with one point, Ben, and then we'll jump, jump right to you. So, Sam, Sam Altman recently gave an interview, I think maybe Alex Friedman or one of these, one of the, one of the podcasts. And he actually said something I thought was actually quite helpful. Or let's see, Ben, if you agree with it, he said something along the lines of, you want to assume that the big foundation models coming out of the big AI companies are going to get a lot better.

So you want to assume they're going to get like a hundred times better. And as a startup founder, you want to then think, okay, if the current models, if this current foundation models get 100 times better, is my reaction, oh, that's great for me and for my startup because I'm much better off as a result, or is your reaction the opposite? Is it, oh, shit, I'm in real trouble. So let me, let me, let me just stop right there, Ben, and see what you think of that as general advice. Well, I think, like, generally that's right.

Ben Horowitz
But there's some nuances to it, right? Like, so I think that, you know, from Sam's perspective, he was probably, you know, discouraging people from building foundation models, which I don't know that I would entirely agree with that, in that a lot of the startups building foundation models are doing very well. And there's, there's many reasons for that. One is there are architectural differences which lead to, so there, there's how smart is a model? There's how fast is the model?

There's how good is a model in a domain? There's, you know, and that goes for not just text models, but, you know, image models as well. Like there are different domains, different kinds of images that responds to prompts differently. Like if you ask mid journey and ideogram the same question, they react very differently, you know, depending on the use cases that they're tuned for. And then there's this whole field of distillation where, you know, Sam can go build the biggest, smartest model in the world, and then you can walk up as a startup and kind of do a distilled version of it and get a model very, very smart at a lot less cost.

So there are things that, yes, the big company models are going to get way better, you know, kind of way better at what they are. But if you're building, so you need to deal with that. So if you're trying to go head to head, full frontal assault, probably have a real problem just because they have so much money. But if you're doing something that's different enough, then, or like different domain and so forth, for example, at databricks, they've got a foundation model, but they're using in a very specific way in conjunction with their leading data platform. So, okay, now, if you're an enterprise and you need a model that knows all the nuances of how your enterprise data model works and what things mean and needs access control and what needs to use your specific data and domain knowledge and so forth, then it doesn't really hurt them if Sam's model gets way better.

Similarly, eleven labs with their voice model has kind of embedded into everybody, everybody uses it as part of the AI stack and so it's got kind of a developer hook into it and then they're going very, very fast what they do and really being very focused in their area. So there are things that I would say like extremely promising that are kind of ostensibly, but not really competing with OpenAI or Google or Microsoft. So I think it's kind of, I think it's just too a little, it sounds a little more coarse grain than I, than I would interpret it if I was building a startup. Right, well, let's start, let's dig into this a little bit more. So let's start with the question of like do we think the big models, the God models are going to get 100 times better?

I, well, I think so. I kind of think so. And then I'm not sure. So like if you think about the language models, let's do those, because those are probably the ones that people are most familiar with it.

I think if you look at the very top models, Claude and OpenAI and Mistral and Lama, the only people who I feel like really can tell the difference as users amongst those models are the people who study them. They're getting pretty close.

You know, you would expect if we're talking 100 x better that they would be like one of them might be separating from each other a lot more, but the improvement. So 100% better. In what way? Like for the normal, for us and using it in a normal way, like asking it questions and finding out stuff. Let's say some combination, let's say some combination of just like breadth of knowledge and capability.

Yeah, yeah. Like I think in some of them you're. Yeah, yeah, yeah, right. But then also just like combined with like sophistication of the answers, you know, sophistication of the output, the quality of the output. Sophistication, the output, you know, lack of, you know, lack of hallucination, factual grounding.

Well, that I think is for sure gonna get a hundred times better like that. Yeah, I mean they're on a path for that. The things that are so against that. Right. Like the alignment problem where, okay, yeah, they're getting smarter, but they're not allowed to say what they know.

And then that alignment also kind of makes them dumber in other ways. And so they, you do have that thing, the other kind of question that's come up lately, which is kind of, do we need a breakthrough to go from what we have now, which I would categorize as artificial human intelligence as opposed to artificial general intelligence, meaning it's kind of the artificial version of us. Like, it's. We've structured the world in a certain way using our language and our ideas and our stuff, and it's learned that very well. Amazing.

And it can do kind of a lot of the stuff that we can do. But are we then the asymptote, or you need a breakthrough to get to some kind of higher intelligence, more general intelligence? And I think if we're the asymptote, then in some ways, it won't get 100 times better because it's already, like, pretty good relative to us. So that's, you know, but, yeah, like, it'll know more things. It'll hallucinate less on all those dimensions.

It'll be 100 times better. You know, there's this graph floating around. I forget exactly what. What the axes are, but it's basically shows the improvement across the different models, and it basically shows an asymptote to your point. It shows an asymptote against the current tests that people are using that sort of, like, at or slightly above human levels, which is what you would think if you're being trained on entirely human data.

Marc Andreessen
Now, the counterargument on that is, are the tests just to the test too simple. Right. It's a little bit like the question. People ever run the SAT, which is if you have a lot of people getting 800s, you know, on both math and verbal. On the SAT, you know, is the scale.

Is the scales too constrained? Do you need a test actually, that can actually test for Einstein? Right, right. Summarize the test that we have, and it's great at it. Right.

But, like, you could imagine, and you can imagine sat that, like, really can detect gradations of people who have, like, ultra high iqs who are, you know, ultra good at math or something. You could imagine tests for AI that, you know, you could imagine tests that test for reasoning, you know, above human levels, one assumes. Yeah, yeah. Well, maybe the AI needs to write the test. Yeah.

Straight. The test. Yeah. And then there's a related question that comes up a lot. It's an argument we've been having internally, which is also, I'll start to take some sort of more provocative and probably more bullish, or, as you would put it, sort of science fiction predictions on some of this stuff.

So there's this question that comes up, which is like, okay, you take an LLM, you train it on the Internet. What is the Internet data? What is the Internet data corpus? Like? It's an average of everything, right?

It's sort of, it's a representation of sort of human activity. Representation of human activity is going to kind of, you know, because of the sort of distribution of intelligence in the population. You know, most of it's somewhere in the middle. And so you're sort of, you're sort of. The data set on average sort of represents the average human.

Ben Horowitz
You're teaching it to be very average. Yeah, yeah. You're teaching it to be very average. It's just because most of the, most of the content created on the Internet is created by average people. And so kind of the content on average, you know, as a whole, on average is average.

Marc Andreessen
And so therefore, the answer is our average, right? Like, you're going to get back an answer that sort of represents the kind of thing that an average 100 iq. By definition, the average human is 100 iq. Its iq is indexed to 100 at the center of the bell curve. And so by definition, you're kind of getting back the average.

I actually argue that's not, I mean, that may be the case for the default prompt today. That may be, if you just ask the thing, does the earth revolve around the sun or something, you get the average answer to that. And maybe that's fine. This gets to the point. This gets to the point.

It's like, well, okay, the average data might be of an average person, but the data set also contains all of the things written and thought by all the really smart people. All that stuff is in there. And all the current people who are like that, their stuff is in there. And so then it's sort of like a prompting question, which is like, how do you prompt it in order to get basically, in order to basically navigate to a different part of what they call the latent space, to navigate to a different part of the data set. That basically is like the super genius part.

The way these things work is if you craft the prompt in a different way, it actually leads it down a different path inside the dataset and gives you a different kind of answer. Well, here's another example of this. If you ask it, write code to do x write code to sort a list or whatever, render an image, it will give you average code to do that. If you say, write me secure code to do that, it will actually write better code with fewer security holes, which is very interesting because it's accessing a different corpus of training data, which is secure code. And if you ask, write this image generation thing the way John Carmack would write it, you get a much better result because it's tapping into the part of the latent space represented by John Carmack's code, who's the best graphics programmer in the world.

And so you could imagine prompting crafts in many different domains such that you're kind of unlocking the latent super genius, even if that's not the default answer. Yeah, I think that's correct. I think there's still a potential limit to its smartness. So we had this conversation in the firm the other day where you have, like, there's the world, which is very complex, and intelligence kind of is, you know, how well can you understand, describe, represent the world? But our current iteration of artificial intelligence consists of humans structuring the world and then feeding that structure that we've come up with into the AI.

Ben Horowitz
And so the AI kind of is good at predicting that how humans have structured the world as opposed to how the world actually is, which is something more probably complicated, maybe the irreducible or what have you. And so do we just get to a limit where, like, it can be really smart, but its limit is going to be the smartest humans as opposed to smarter than the smartest humans, and then kind of related? Is it going to be able to figure out brand new things, new laws of physics and so forth? Now, of course, there are, like, one in 3 billion humans that can do that or whatever. Like, that's a very rare, rare kind of, kind of intelligence.

So it still makes the AI's extremely useful, but it kind of, you know, they play a different role if they're kind of artificial humans than if they're, like, artificial, you know, super duper mega humans. Yeah. So let me make the bulk, let me make the sort of extreme bullcase for the hundred because. Okay, so, so the cynic would say that, of course. Like, the cynic would say the Sam Altman would be saying they're going to get 100 times better.

Marc Andreessen
Precisely. If they're not going to. Yeah, yeah, yeah, yeah, yeah. Right. Because he'd be saying that basically in order to scare people into not competing.

Ben Horowitz
Well, I think that whether or not they are going to get 100 times better, Sam would be very likely to say that. Like, Sam, for those who don't know him, is, he's a very, very smart guy, but for sure, he's a competitive genius. There's no question about that. So you have to take that into account. Yeah, right.

Marc Andreessen
So if they weren't going to get a lot better, he would say that. But of course, if they were going to get a lot better, to your point, he would also say that, yes, why not? Why not, right? And so let me make the bull case that they are going to get 100 times better or maybe even 1000, or even on an upper curve for a long time. And it goes as follows, which is like, and there's like enormous controversy, I think, on every one of the things I'm about to say.

But you can find very smart people in the space who believe basically everything I'm about to say. So one is, look, there is, there is generalized learning happening inside the neural networks. And we know that because we now have introspection techniques where you can actually go inside and look inside the neural networks to look at the neural circuitry that is being evolved as part of the training process. And these things are evolving general computation functions. There was a case recently where somebody trained one of these on a chess database, and just by feeding it, by training lots of chess games, it actually imputed a world model of a chessboard inside the neural network, and that was able to do original moves.

The neural network training process does seem to work. And then specifically, not only that, but meta and others recently have been talking about how so called overtraining actually works, which is basically continuing to train the same model against the same data for longer, putting more and more compute cycles against it. I've talked to some very smart people in the field, including there, who basically think that actually that works quite well. And the diminishing returns, people were worried about, about more training, and they proved. It in a new Lahman release.

Ben Horowitz
That's the primary technique they use. Yeah, exactly. One guy in the space basically told me, basically, he's like, yeah, we don't necessarily need more data at this point to make these things better. We maybe just need more compute cycles. Just train it 100 times more and it may just get actually a lot better.

So one day, the labeling, it turns out that supervised learning ends up being a huge boost to these things. Yeah, so we've got that. We've got all of the kind of, you know, let's say rumors and reports of various kinds of self, you know, sort of self improvement loops, you know, that are kind of underway. And most of the sort of super advanced practitioners in the field think that there's now some form of self improvement loop that works, which basically is you basically get an AI to do what's called chain of thought. You get it to basically go step by step to solve a problem.

Marc Andreessen
You get it to the point where it knows how to do that, and then you basically retrain the AI on the answers. On those answers, you're basically doing a forklift upgrade across cycles of the reasoning capability. A lot of the experts think that things starting to work now. Then what else is starting to work? Oh, synthetic data.

There's still a raging debate about synthetic data, but there's quite a few people who are actually quite bullish on that. Then there's even this trade off. There's this dynamic where llms might be. They might be okay at writing code, but they might be really good at validating code. They might actually be better at validating code than they are writing it.

Ben Horowitz
That would be a big help. Yeah, but that also means they can. Validate their own code. Yeah, they can validate their own code. And we have this anthropomorphic, the anthropomorphic bias is very deceptive with these things because you think of the model as an item.

Marc Andreessen
It's like, how could you have an it that's better at validating code than writing code? But it's not an it. What it is is it's this giant latent space. It's this giant neural network. And the theory would be there are totally different parts of the neural network for writing code and validating code, and there's no consistency requirement whatsoever that the network be equally good at both of those things.

And so if it's better at one of those things, so then the thing that it's good at might be able to make the thing that it's bad at better and better and better. Right, right, sure, sure, right. Sort of a self improvement thing. And so anyway, and then, yeah, and then, look, there's. And then on top of that, there's all the other things coming.

Right. Which is, it's everything is all these practical things, which is there's an enormous chip constraint right now. So every AI that anybody uses today is, its capabilities are basically being gated by the availability of chips. But like that, that will resolve over time. There's also data.

To your point, I like data labeling. Like, you know, there is a lot of data in these things now, but there is a lot more data out in the world. And there's, at least in theory, you could generate, some of the leading AI companies are actually paying generate new data. And by the way, even, like, the open source data sets are getting much better. And so there's a lot of data improvements that are coming, and then there's just the amount of money pouring into the space to be able to underwrite all this.

And then, by the way, there's also just the systems engineering work that's happening. Right. Which is a lot of the current systems basically were built by scientists, and now they're really world class engineers are showing up and tuning them up and getting them to work better. And maybe that's not, maybe that's not. A, which makes training, by the way, way more efficient as well.

Ben Horowitz
Not just inference, but also training. Yeah, exactly. And then even another improvement area is basically it was at the, Microsoft released their phi small language model yesterday. And apparently it's competitive. It's a very small model, competitive with much larger models.

Marc Andreessen
And the big thing they say that they did was they basically optimized the training set. So they deduplicated the training set. They took out all the copies, and they really optimized on a small amount of training data, on a small amount of high quality training data, as opposed to the larger amounts of low quality data that most people train on. And so anyway, it's like, you know, you add all these up and you've got like eight or ten different combination of sort of practical and theoretical improvement vectors, you know, that are, that are all in play. And, you know, it's just, it's, it's, it's as.

It's hard for me to imagine that some combination of those doesn't lead to, like, really dramatic improvement from here. Yeah, no, I definitely agree. I think that's, that's for sure gonna happen. Right. Like, if you were so back to Sam's question, or the Sam's proposition, I think if you were a startup and you were like, okay, in two years, I can get as good as GPT four.

Ben Horowitz
You shouldn't do that. Right. That would be a bad mistake. Right, right. Well, this also goes, you know, this also goes to, you know, a lot of what's happening in the, you know, a lot of what entrepreneurs are afraid of is it's just like, okay, I'll give you an example.

Marc Andreessen
So a lot of, a lot of entrepreneurs, here's something they're trying to figure out, which is like, okay, I really think I know how to use, I know how to build a SaaS app that makes harnesses an LLM to do really good marketing collateral. Let's just make a very similar, a very similar thing. And so I build a whole system for that. Will it just turn out to be that the big models in six months will be even better in making marketing collateral just from a simple prompt such that my apparently sophisticated system is just irrelevant because the big model just does it. Yeah, let's talk about that.

Apps. Another way you can think about it is the criticism of a lot of current AI app companies is their quote, unquote, GPT wrappers. There's sort of thin layers of wrapper around the core model, which means the core model could commoditize them or displace them. But the counterargument, of course, is it's a little bit like calling all old software apps database wrappers. Wrappers around a database.

It turns out, actually, wrappers around a database is like most modern software, and a lot of that actually turned out to be really valuable. And it turns out there's a lot of things to build around the core engine. Yeah. So, Ben, how do we think about that when we run into companies thinking about building apps? Yeah, you know, it is a.

Ben Horowitz
It's a very tricky question because there's also this, this correctness gap. Right? Like, so, you know, why do we have copilots? Where are the pilots? Right?

Where are the. There's no AI pilots. There are only AI copilots. There's a human in the loop on absolutely everything. And that really kind of comes down to this.

You know, you can't trust the AI to be correct and drawing a picture or writing a program or, you know, even like, writing a court brief without making up citations. You know, all these things kind of require a human and kind of turns out to be, like, fairly dangerous to not. And then I think that. So what's happening a lot with the application layer is people saying, well, to make it really useful, I need to turn this copilot into a pilot, and can I do that? And so that's an interesting and hard problem.

And then there's a question of, is that better done at the model level or, like, at some layer on top that kind of teases the correct answer out of the model by doing things like using code validation or what have you, or is that just something that the models will be able to do? I think that's one open question. And then as you get into kind of domains and potentially wrappers on things, I think there's a different dimension than what the models are good at, which is what is the process flow of, which is kind of in database relatives. So on the database kind of analogy, there is like, the part of the task in a law firm that's writing the brief, but there's like 50 other tasks and things that have to be integrated into the way a company works, like the process flow, the orchestration of it. And maybe there are on a lot of these things, like if you're doing video production, there's many tools, like so, or music even, right?

Like, okay, who's going to write the lyrics, which AI, I'll write the lyrics and which AI, like figure out the music and then like how does that all come together and how do we integrate it and so forth? And those things tend to just require a real understanding of the end customer and so forth in a way. And that's typically been how applications have been different than platforms in the past is like there's real knowledge about how the customer using it wants to function that doesn't have anything to do with the kind of Intelli or is just different than what the platform is designed to do. And to get that out of the platform for a kind of company or person turns out to be really, really hard. And so those things I think are likely to work, especially if the process is very complex and it's something, it's funny.

As a firm, we're a little more hardcore technology oriented and we've always struggled with those in terms of, oh, this is like some process application for plumbers to figure out this. And we're like, well, where's the technology? But a lot of it is how do you encode some level of domain expertise and kind of how things work in the actual world back into the software? I often think I have until founders that you can think about this in terms of price. You can work backwards from pricing a little bit, which is to say business value and what you can charge for, which is the natural thing for any technologist to do is to say, I have this new technological capability and I'm going to sell it to people and what am I going to charge for?

Marc Andreessen
It is going to be somewhere between my cost of providing it and then whatever markup I think I can justify. And if I have a monopoly providing it, maybe the markup is infinite, but it's our technology forward, supply forward know, pricing model. There's a completely different pricing model for kind of business value backwards and or sort of, you know, so called value pricing, value based pricing. And, and that's, you know, to your point, that's basically a pricing model that says, okay, what's the business value to the customer of the thing? And if the business value is, you know, a million dollars, then can I charge 10% of that and get $100,000, right, or whatever?

And then, you know, why is it cost $100,000 as compared to $5,000 is because, well, because to the customer it's worth a million dollars and so they'll pay 10% for it. Yeah, actually, so a great example of that. Like, we've got a company in our portfolio, crest AI, that does things like debt collection. Okay, so if I can collect way more debt with way fewer people, with my, you know, it's a copilot type solution, then what's that worth? Well, it's worth a heck of a lot more than just buying an OpenAI license because an OpenAI license is not going to easily collect debts or kind of enable your debt collectors to be massively more efficient or that kind of thing.

Ben Horowitz
So it's bridging that gap between the value. And I think you had a really important point. The test for whether your idea is good is how much can you charge for it? Can you charge the value or are you just charging the amount of work? It's going to take the customer to put their own wrapper on top of OpenAI.

That's the real test to me of how deep and how important is what you've done. Yeah. To your point on the kinds of businesses that technology investors have had a hard time with, kind of thinking about maybe accurately, is sort of, it's the company that is, it's a vendor that has built something where it is a specific solution to a business problem, where it turns out the business problem is very valuable to the customer, and so therefore, they will pay a percentage of the, of the value provided in the terms for price for the software. And that actually turns, that actually turns out you can have businesses that are not very technologically differentiated, that are actually extremely lucrative, because that business is so lucrative, they can actually afford to go think very deeply about how technology integrates into the business, what else they can do. This is the story of a salesforce.com, for example.

And by the way, there's a kind of a chance, a theory that the models are all getting really good. There are open source models, there are like, that are awesome, you know, llama Mistral, like, these are great models. And so the actual layer where the value is going to accrue is going to be like tools, orchestration, that kind of thing, because you can just plug in whatever the best model is at the time, whereas the models are going to be competing in a death battle with each other and be commoditized down to the cheapest one wins and that kind of thing. So you could argue that the best thing to do is to kind of connect the power to the people. Right?

Marc Andreessen
So that actually takes us to the next question, and this is a two in one question. So Michael asks, these are, and I'll say these are diametrically opposed, which is why I paired them. So Michael asks, why are VC's making huge investments in generative AI startups when it's clear these startups won't be profitable anytime soon? Which is a loaded question, but we'll take it. And then Kaiser asks, if AI deflates the cost of building a startup, how will the structure of tech investment change?

And of course, Ben, this goes to exactly what you just said. So it's basically, the questions are diametrically opposed because if you squint out of your left eye, what you see is basically the amount of money being invested in the foundation model, companies going up into the right at a furious pace. These companies are raising hundreds of millions, billions, tens of billions of dollars. And it's just like, oh my God, look at these sort of capital, sort of, I don't know, infernos that hopefully will result in value at the end of the process. But my God, look at how much money is being invested in these things.

If you squint through your right eye, you think, wow, that now all of a sudden it's much easier to build software. It's much easier to have a software company. It's much easier to have a small number of programmers writing complex software because they've got all these AI copilots and all these automated software development capabilities that are coming online. And so on the other side, the cost of building an AI like application startup might crash. And it might just be that like Salesforce, the AI, Salesforce.com might cost a 10th or a hundredth or a thousandth the amount of money that it took to build the old database driven salesforce.com dot.

What do we think of that dichotomy, which is you can actually look out of either eye and see either cost to the moon for startup funding or cost actually going to zero? Well, so it is interesting, we actually have companies in both camps, probably the companies that have gotten to profitability the fastest maybe in the history of the firm have been AI companies. There have been AI companies in the portfolio where the revenue grows so fast that it actually kind of runs out ahead of the cost. And then there are people who are in the foundation model race who are raising hundreds of millions, even billions of dollars to keep pace and so forth. They also are generating revenue at a fast rate.

Ben Horowitz
The headcount in all of them is small. So I would say where AI money goes. And even if you look at OpenAI, which is the big spender in startup world, which we are also investors, headcount wise, theyre pretty small against their revenue. It is not a big company headcount. Like, if you look at the revenue level and how fast they've gotten there, it's pretty small.

Now, the total expenses are ginormous, but they're going into the model creation. So it's an interesting thing. I mean, I'm not entirely sure how to think about it, but I think, like, if you're not building a foundation model, it will make you more efficient and probably get to profitability quicker. Right. So this is a very bullish counterargument, but the counter argument to that would be basically that falling costs for, like building new software companies are a mirage.

Marc Andreessen
And the reason for that is this thing in economics called the Jevons paradox, which I'm going to read from Wikipedia. So the Jevons paradox occurs when technological progress increases the efficiency with which a resource is used, right? Reducing the amount of that resource necessary for any one use. But the falling cost induces increases in demand elasticity enough that the resource use overall is increased rather than reduced. That's certainly possible, right?

You see versions of this, for example, you build in your freeway and it actually makes traffic jams worse because basically what happens is, oh, it's great, now there's more roads, now we can have more people live here, we can have more people, we can make these companies bigger. And now there's more traffic than ever, and now the traffic's even worse. Or you saw the classic example is during the industrial revolution, coal consumption, as the price of coal dropped, people use so much more coal that actually the overall consumption actually increased, people getting a lot more power. But the result was the use of a lot more coal in the paradox. And so the paradox here would be, yes, the cost of developing any given piece of software falls, but the reaction to that is a massive surge of demand for software capabilities.

And so the result of that actually is, although it looks like starting software companies, the price is going to fall. Actually it's going to happen is it's going to rise for the high quality reason that you're going to be able to do so much more with software. The products are going to be so much better, and the roadmap is going to be so amazing of the things you can do, and the customers are going to be so happy with it that they're going to want more and more and more so the result of it, and by the way, another example of Jevons Paradox playing out in other related industries in Hollywood, CGI, in theory, should have reduced the price of making movies, in reality has increased it because audience expectations went up. And now you go to a Hollywood movie and it's wall to wall CGI. And so movies are more expensive to make than ever.

And so the result of it, but the result in Hollywood is at least much more, let's say, visually elaborate movies. Whether they're better or not is another question, but much more visually elaborate, compelling, kind of visually stunning movies through CGi. The version here would be much better software, radically better software to the end user, which causes end users to want a lot more software, which causes actually the price of development to rise. If you just think about a simple case like travel, okay, booking a trip through Expedia is like, complicated. You're likely to get it wrong.

Ben Horowitz
You're clicking on menus and this and that and the other. And like, you know, da da da da. An AI version of that would be like, you know, send me to Paris, put me in a hotel I love at the best price, you know, send me on the best possible kind of airline, an airline ticket. And then, you know, like, make it like, really special for me. And like, maybe you need a human to go, okay, like we're going to, you know, or maybe the AI gets more complicated and says, okay, well, we know the person loves chocolate and we're going to life, you know, FedEx in the best chocolate in the world, from Switzerland into this hotel in Paris and this and that and the other.

And so, like, the quality you can, the quality could get to levels that we can't even imagine today just because, you know, the software tools aren't, aren't what they're going to be. So. Yeah, that's right. Yeah, I kind of buy that, actually. I think I buy the argument.

You're both nice. How about yeah, or how about I'm going to land in, whatever, Boston at 06:00 I want to have dinner at seven with a table full of, like, super interesting people. Yeah, right, right, right. You know? Yeah, right.

Like, yeah, yeah. You know, no travel agent would do that for you today, nor would you want them to. No, no, right. Well, and then you think about it. It's got to be integrated into my personal AI.

And like, and this is, you know, there's just like, unlimited kind of ideas that you can do. And I think this is one of the kind of things that's always been underestimated about humans is like, our ability to come up with new things we need. Like that has been unlimited. And there, there's a very kind of famous case where John Maynard Keynes, who, the kind of prominent economists in the kind of first half of last century, had this thing that he predicted, which is like, nobody, because of automation, nobody would ever work a 40 hours workweek, you know, like, good, because once their needs were met, needs being like shelter and food and, you know, I don't even know if transportation was in there. Like, that was it.

It was over. And, like, you would never work past the need for shelter and food. Like, why would you, like. There's no reason to. But of course, needs expanded.

So then everybody needed a refrigerator. Everybody needed not just one car, but a car for everybody in the family. Everybody needed television set. Everybody needed, like, glorious vacations, everybody, you know, so what are we going to need next? I'm quite sure that I can't imagine it, but, like, somebody's going to imagine it, and it's quickly going to become a need.

Marc Andreessen
Yeah, that's right. By the way, as Keynes famously said, it was his essay, I think, was economic prospects for our grandchildren, which was basically that. Yeah. You just articulated. So Karl Marx had another version of that.

I just pulled up the quote. So society, when, you know, when, when the marxist utopia socialism is achieved, society regulates the general production, thus makes it possible for me to do blah, blah, blah. To hunt in. This is a quote. To hunt in the morning, fish in the afternoon, rear cattle in the evening, criticize after dinner.

What a glorious life. What a glorious life. Like, if I could just list four things that I do not want to do, it's hunt, fish, rear cattle, and criticize. Right. And by the way, it says a lot about Marx that those were his four things.

Ben Horowitz
Well, criticizing being his favorite thing, I think, is basically communism in a nutshell. Exactly. I don't want to get too political, but. Yes, yes, 100%. And so, yeah, so it's just this, yeah, what they had when Keynes and Marks had in common is just this incredibly constricted, it's incredibly constricted view of what people want to do.

Marc Andreessen
And then, and then correspondingly, you know, the other thing is just like, you know, people, people who want, people who want to have a mission. I mean, probably some people just want to fish and hunt. Yeah. But, you know, a lot of, a lot of people want to have a mission. They want to have a cause.

They want to have a purpose. They want to be useful. They want to be productive, actually a good thing in life. It turns out. It turns.

It turn, it turns out, yeah. In the startling turn of events. Okay, so, yeah, so, yeah, I think that I've long felt, you know, a little bit of the software eats the world thing. A decade ago, I've always thought that. I've always thought that basically demand for software is sort of perfectly elastic, possibly to infinity.

And the theory there basically is if you just continuously bring down the cost of software, you know, which has been happening over time, that basically demand, you know, basically is like. Basically. Basically perfectly correlates upward. And the reason is because, as we've been discussing. But it's.

There's always something else to do in software. There's always something else to automate. There's always something else to optimize. There's always something else to improve. There's always something to make better.

And in the moment, with the constraints that you have today, you may not think of what that is, but the minute you don't have those constraints, you'll imagine what it is. I'll just give you an example. So I'll give an example. Playing out with AI right now, we have companies that do this. There have been companies that have made AI or that have made software systems for doing security cameras forever.

For a long time, it was a big deal to have software that would have different security camera feeds and store them on a DVR and be able to replay them and have an interface that lets you do that. Well, it's like AI security cameras all of a sudden can have actual semantic knowledge of what's happening in the environment. They can say, hey, that's Ben. And then they can say, hey, that's Ben, but he's carrying a gun. Yeah.

Right, right. And by the way, that's Ben. And he's carrying a gun. But that's because, like, he hunts on, you know, Thursdays and Fridays as compared to. That's Mary, and she never carries a gun.

And like, you know, like, something is wrong and she's really mad. Right. She's got a. Yeah. Really steamed expression on her face, and we should probably be worried about it.

Right? And so there's like, an entirely new set of capabilities. You can do, just as one example, for security systems that were never possible pre AI. And the security system that actually has a semantic understanding of the world is obviously much more sophisticated than the one that doesn't and might actually be more expensive to make. Right, right.

Ben Horowitz
Well, and just imagine healthcare, right? Like you could wake up every morning and have a complete diagnostic, you know, like, how am I doing today? Like, what are all my levels of everything? And, you know, how should I interpret them? You know, better than, you know.

This is one thing where AI is really good, is, you know, medical diagnosis because it's a super high dimensional problem. But if you can get access to, you know, your continuous glucose reading, you know, maybe sequence your blood now and again, this and that and the other. Yeah. You've got an incredible kind of view of things. And who doesn't want to be healthier, you know, like, now we have a scale that's basically what we do, you know, maybe, maybe check your heart rate or something, but like, pretty primitive stuff compared to where we could go.

Marc Andreessen
Yeah, that's right. Okay, good. All right, so let's go to the next topic. So on the topic of data. So Major Tom asks, as these AI models allow for us to copy existing app functionality at minimal cost, proprietary data seems to be the most important moat.

How do you think that will affect proprietary data value? What other moats do you think companies can focus on building in this new environment? And then Jeff Weishaupt asks, how should companies protect sensitive data? Trade secrets, proprietary data, individual privacy in the brave new world of AI. So let me start with a provocative, let me start with a provocative statement, Ben.

See if you agree with it, which is, you hear a lot. This statement, or cliche, is like, data is the new oil. And so it's like, okay, data is the key input to training AI, making all this stuff work. And so therefore, data is basically the new resource. It's the limiting resource.

It's the super valuable thing. And so whoever has the best data is going to win. And you see that directly in how you train AI's. And then you also have a lot of companies, of course, that are now trying to figure out what to do with AI. And a very common thing you'll hear from companies is, well, we have proprietary data.

So I'm a hospital chain or I'm whatever, any kind of business, insurance company or whatever, and I've got all this proprietary data that I can apply, that I'll be able to build things with my proprietary data. With AI, that won't just be something that anybody will be able to have. Let me argue that basically. Let's see. Let me argue in almost every case like that.

It's not true. It's basically what the Internet kids would call cope. It's simply not true. And the reason it's just not true is because the amount of data available on the Internet and just generally in the environment is just a million times greater. And so while it may not, you know, while it may not be true that I have your specific medical information, I have so much medical information off the Internet for so many people in so many different scenarios that it just swamps the value of, quote, your data.

You know, just, it's just, it's just like overwhelming. And so your, your proprietary data, as, you know, company x, will be a little bit useful on the margin, but it's not actually going to move the needle, and it's not really going to be a barrier to entry in most cases. And then let me cite as proof for my belief that this is mostly cope is there has never been, nor is there now any sort of, basically any level of sort of rich or sophisticated marketplace for data. Market for data. There's no large marketplace for data.

In fact, what there are, is there are very small markets for data. So there are these businesses called data brokers that will sell you large numbers of, like, information about users on the Internet or something. And they're just small businesses. They're just not large. It just turns out information on lots of people is just not very valuable.

And so if the data actually had value, it would have a market price and you would see it transacting, and you actually very specifically don't see that, which is sort of a sort of quantitative proof that the data actually is not nearly as valuable as people think it is. Where I agree. So I agree that the data, like, just as here's a bunch of data and I can sell it without doing anything to the data is like massively overrated. Like, I definitely agree with that. And like, maybe I can imagine some exceptions, like some, you know, special population genomic databases or something that are, that were very hard to acquire, that are useful in some way that, you know, that's not just like living on the Internet or something like that.

Ben Horowitz
I could imagine where that's super highly structured, very general purpose, and not widely available. But for most data in companies, it's not like that in that it tends to not, it's either widely available or not general purpose. It's kind of specific. Having said that, right, like, companies have made great use of data. For example, a company that you're familiar with, meta, uses its data to kind of great ends itself, feeding it into its own AI systems, optimizing its products in incredible ways.

And I think that us, Andreessen Horowitz, actually, so we just raised $7.2 billion, and it's not a huge deal but we took our data and we put it into an AI system and our LP's were able, there's a million questions investors have about everything we've done, our track record, every company we've invested and so forth. And for any of those questions, they could just ask the AI, they could be wake up at 03:00 in the morning, go, do I really want to trust these guys and go in and ask the AI question, and boom, they'd get an answer back instantly, they'd have to wait for us and so forth. So we really kind of improved our investor relations product tremendously through use of our data. And I think that almost every company can improve its competitiveness through use of its own data. But the idea that it's collected some data, that it can go like cell or that is oil or what have you, that's, yeah, that's probably not true, I would say.

And you know, it's kind of interesting because a lot of the data that you would think would be the most valuable would be like, your own code base, write your software that you've written. So much of that lives in GitHub. Nobody is actually, I don't know of any company we work with, you know, whatever, a thousand software companies. And do we know any that's like building their own programming model on their own code? Like, or, and would that be a good idea?

Probably not, just because there's so much code out there that the systems have been trained on. So like, that's not so much of an advantage. So I think it's a very specific kind of data that would have value. Well, let's ask, let's make it actionable then, if I'm running a big company, like if I'm running an insurance company or a bank or a hospital chain or something like that. Like how, or, you know, a consumer packaged goods company, Pepsi or something, like what, how should I validate, like how should I validate that I actually have a valuable proprietary data asset that I should really be focusing on using versus maybe versus in the alternate, by the way, maybe there's other things that maybe I should be taking all the effort I was spending trying to optimize use of that data, and maybe I should use it entirely trying to build things using Internet data instead.

Yeah, so I think, I mean, look, if you're right, if you're in the insurance business, then, like, all your actuarial data is both interesting and that, I don't know that anybody publishes their actual actuarial data. And so, like, I'm not sure how you would train the model on stuff off of the Internet. You know, similarly, that's a good, let. Me, can I challenge that one? So that would be a good thing.

Marc Andreessen
That'd be a good test case. I'm an insurance company. I've got records on 10 million people and the actuarial tables. And when they get sick and when they die, okay, that's great. But there's lots and lots of actuarial, general actuarial data on the Internet for large scale populations because governments collect the data and they process it and they publish reports, and there's lots of third party, there's lots of academic studies.

And so, like, is your, is your large data set giving you any additional actuarial information that the much larger data set on the Internet isn't already providing you? Like, are your insurance clients actually actuarially any different than just everybody? I think so. Because on intake, on the, you know, when you get insurance, they give you like a blood test. They've got all these things.

Ben Horowitz
They know if you're a smoker and so forth. And in the, I think in the general data set, like, yeah, you know, who dies, but you don't know what the fuck they did coming in. And so what you really are looking for is like, okay, for this profile of person with this kind, with these kinds of lab results, how long do they live? And that's, that's where the value is. And I think that, you know, interesting.

Like, you know, I was thinking about, like, a company like Coinbase, where, right. They have incredibly valuable assets in the terms of money. They have to stop people from breaking in. They've done a massive amount of work on that. They've seen all kinds of break in types.

I'm sure they have tons of data on that. It's probably weirdly specific to people trying to break into crypto exchanges. And so I think it could be very useful for them. I don't think they could sell it to anybody, but I think every company's got data that, if fed into an intelligent system, would help their business. And I think almost nobody has data that they could just go sell.

And then there's this kind of in between question, which is, what data would you want to let Microsoft or Google or OpenAI or anybody get their grubby little fingers on? And that, I'm not sure that, I think, is a question that enterprises are wrestling with more than, it's not so much should we go, like, sell our data, but it should we train our own model just so we can maximize the value. Or should we feed it into the big model? And if we feed it into the big model, do all of our competitors now have the thing that we just did and, you know, or could we trust the big company to not do that to us? Which I kind of think the answer on trusting the big company not to f, with your data is probably, I won't do that.

If your competitiveness depends on that, you probably shouldn't do that. Yeah. Well, there are at least reports that certain big companies are using all kinds of data that they should be using to train their models already, so. Yep. I think, like, I think those reports are very likely true.

Right. Or they'd have open data. Right. Like, this is, you know, we've talked about this before, but you have the same companies that are saying they're not stealing all the data from people, you're taking it in an unauthorized way, refuse to say, open their data. Like, why not tell us where your data came from?

And in fact, they're trying to shut down all openness. No open source, no open weights, no open data, no open nothing. And go to the government and try and get to do that. If you're not a thief, then why are you doing that? Right.

Marc Andreessen
Right. What are you hiding? By the way? There's other twists and turns here. So, for example, in the insurance example, I kind of deliberately loaded it because you may know it's actually illegal to use genetic data for insurance purposes.

Right. So there's this thing called the Geno law Genetic Information non Discrimination act of 2008, and basically, it basically bans health insurers in the US from actually using genetic data for the purpose of doing health assessment, actuarial assessment, which, by the way, because now the genomics are getting really good. That data probably actually is among the most accurate data you could have if you were actually trying to predict when people are going to get sick and die and they're literally not allowed to use it. Yeah, it is. I think that this is a interesting, like, weird misapplication of good intentions in a policy way that's probably going to kill more people than ever, get saved by every kind of health, FDA, et cetera, policy that we have, which is, you know, in the world of AI, having access to data on all humans, why they get sick, what their genetics were, et cetera, et cetera, et cetera, is the most, that is, you don't talk about data being the new oil.

Ben Horowitz
Like that is the new oil. That's the healthcare oil, is, if you could match those up then we'd never not know why we're sick. You could make everybody much healthier, all these kinds of things. But to kind of stop the insurance company from kind of overcharging people who are more likely to die, we've kind of locked up all this data. A kind of better idea would be to just go, okay, for the people who are likely to, like, we subsidize health care, like massively for individuals anyway, just like differential, differentially subject, you know, subsidize and, you know, and then, like, you solve the problem and you don't lock up all the data.

But, yeah, it's typical of politics and policy. I mean, most of them are like that, I think. Yeah, well, there's this interesting question. It's like in insurance, like, basically one of the questions people have asked about insurance is, like, if you had perfectly predictive information on, like, individual outcomes, does the whole concept of insurance actually still work? Right, because the whole theory of insurance is risk pooling, right.

Marc Andreessen
It's precisely the fact that you don't know what's going to happen in the specific case. That means you build these statistical models and then you risk pool, and then you have variable payouts depending on exactly what happens. But if you literally knew what was going to happen in every case, because, for example, you had all this predictive genomic data, then all of a sudden it wouldn't make sense to risk pool because you just say, well, no, this person's going to cost x, that person's going to cost y. There's no health insurance already. Doesn't make sense in that way.

Ben Horowitz
Right? Like, insurance, the idea of insurance is kind of like the, it started with crop insurance where like, okay, you know, my crop fails. And so we all put money in a pool in case, like, my crop fails so that, you know, we can cover it. It's kind of designed for, to risk pool for a catastrophic unlikely incident. Like, everybody's got to go to the doctor all the fucking time.

And some people get sicker than others and that kind of thing. But, like, the way our health insurance works is like, all medical gets paid for through this insurance systems, which is this layer of loss and bureaucracy and giant companies and all this stuff when, like, if we're going to pay for people's health care, just pay for people's health care, like, what are we doing, right? Like, and if you want to disincent people from, like, going for nonsense reasons and just up the copay, like, it's like, what are we doing? Just, well, and then from a. Yeah.

Marc Andreessen
From a justice standpoint, from a fairness standpoint, like, would it make sense for me, you know, would it make sense for me to pay more for your healthcare if I knew that you were going to be more expensive than me? Like, you know, I'm directly, you know, if you. If everybody knows what future healthcare cost is per person. Yeah. There has a very good predictive model for it.

You know, societal willingness to all pool in the way that we do today might really diminish. Yeah. Yeah. Well, and then, like, you, you could also, if you knew, like, there's things that you do genetically, and maybe we give everybody a pass on that. It's like you can't control your genetics, but then, like, there's things you do behaviorally that, like, dramatically increases your chance of getting sick.

Ben Horowitz
And so maybe, you know, we incentivize people to stay healthy instead of just, like, paying for them not to die. There's a lot of systemic fixes. We could use the healthcare system. And it couldn't be designed in a more ridiculous way, I think. Well, it could be designed a more ridiculous way.

It's actually more ridiculous than some other countries, but it's pretty crazy. Here, Nathan Odie asks, what are the strongest common themes between the current state of AI and Web 1.0? And so let me start there. Let me give you a theory, Ben, and see what you think. So I get this question because of my role.

Marc Andreessen
And, Ben, you with me at Netscape, we get this question a lot because of our role early on with the Internet. And so there's an Internet boom was like a major, major event in technology, and it's still within a lot of people's memories. And so people like to reason from analogy. So it's like, okay, the AI boom must be like the Internet boom. Starting an AI company must be like starting an Internet company.

And so what is this like? And we actually got a bunch of questions like that that are kind of analogy questions like that, I actually think. And, you know, and then, Ben, you and I were there for the Internet boom. So we lived through that. And the bust and the boom and the bust.

So I actually think that the analogy doesn't really work for the most part. It works in certain ways, but it doesn't really work for the most part. And the reason is because the, the Internet. The Internet was a network, whereas AI is a computer. Yep.

Okay. Yeah. So people understand what we're saying. Like the pc boom or the pc boom. Or even, I would say, the microprocessor, like my best analogy is to the microprocessor or even to the original computers, like back to the mainframe era.

And the reason is because, yeah, look, what the Internet did was the Internet obviously was a network, but the network connected together many existing computers. And then of course, people built many other new kinds of computers to connect to the. But fundamentally, the Internet was a network. And that's important because most of the industry dynamics, competitive dynamics, startup dynamics around the Internet had to do with basically building either building networks or building applications that run on top of networks. The Internet generation of startups was very consumed by network effects.

And all these positive feedback loops that you get when you connect a lot of people together, things like so called Metcalfe's law, which is the value of a network expands. The way it expands is you add more people to it. And then there were all these fights, these fights, all the social networks or whatever, fighting to try to get network effects and try to steal each other's users. Because of the network effects, it's dominated by network effects, which is what you'd expect from a network business. There are some networks effects in AI that we can talk about, but it's more like a microprocessor, it's more like a chip, it's more like a computer in that it's a system that basically, data comes in, data gets processed, data comes out, things happen.

That's a computer, it's an information processing system. It's a new kind of computer. We like to say the sort of computers up until now have been what are called von Neumann machines, which is to say they're deterministic computers, which is they're hyper literal and they do exactly the same thing every time. And if they make a mistake, it's the programmer's fault. But they're very limited in their ability to interact with people and understand the world.

We think of AI and large language models as a new kind of computer, a probabilistic computer, a neural network based computer that, by the way, is not very accurate and doesn't give you the same result every time, and in fact, might actually argue with you and tell you that it doesn't want to answer your question. Yeah. Which makes it very different in nature than the old computers. And it makes you get kind of composability, you know, the ability to build things, big things, out of little things, more complex. Right.

But the capabilities are new and different and valuable and important because it can understand language and images and, you know, all these, all these things that you. See when you use never solve with deterministic computers, we can now go after. Right, right, yeah, exactly. And so I think. I think, Ben, I think the analogy and I think the lessons learned are much more likely to be drawn from the early days of the computer industry or from the early days of the microprocessor than the early days of the Internet.

Does that sound right? I think so, yeah, I definitely think so. And that doesn't mean there's no, like, boom and bust and all that, because that's just the nature of technology. You know, people get too excited and then they get too depressed. So there will be some of that.

Ben Horowitz
I'm sure there will be overbuild outs, potentially, of eventually of chips and power and that kind of thing. We start with the shortage. But I agree, I think networks are fundamentally different in the nature of how they evolved in computers. And just the adoption curve and all those kinds of things will be different. Yeah.

Marc Andreessen
This goes to how I think the industry is going to unfold. And so this is my best theory for what happens from here. There's this, this giant question of, like, is the industry going to be a few God models or a very large number of models of different sizes and so forth? So the computer, like, famously the original computers, like the original IBM mainframes, the big computers, they were very, very large and expensive, and there were only a few of them. And the prevailing view actually, for a long time was that's all there would ever be.

And there was this famous statement by Thomas Watson Sr. Who was the creator of IBM, which was the dominant company for the first, like 50 years of the computer industry. And he said, he said, he said, I believe this actually true hood. He said, I don't know that the world will ever need more than five computers. And I think the reason for that, it was literally, it was like the government's going to have two, and then there's like three big insurance companies, and then that's it.

Ben Horowitz
Yeah. Who else would need to do all that math? Exactly. Yeah. Who else would need to.

Marc Andreessen
Who else needs to keep track of huge amounts of numbers? Who else needs that level of calculation capability? It's just not a relevant, you know, it's just not. Not a relevant concept. And by the way, they were like big and expensive, and so who else can afford them, right?

And who else can afford all the headcount required to manage them and maintain them? I mean, this is in the days, I mean, these things were big. These things were so big that you'd have an entire building that got built around a computer right? And they'd have, like, they'd famously have all these guys in white lab coats, literally, like taking care of the computer because everything had to be kept super clean or the computer would stop working. And so, you know, it was this thing where, you know, today we have the idea of an AI God model, which is like a big foundation model that, you know, then we had the idea of like a God mainframe.

Like there, there would just be a few, a few of these things. And by the way, if you watch old science fiction, it almost always has this sort of conceit. It's like, okay, there's a big supercomputer. And it either is like doing the right thing or doing the wrong thing. And if it's doing the wrong thing, that's often the plot of the science fiction movies, is you have to go in and try to figure how to fix it or defeat it.

So it's sort of this idea of like a single top down thing, of course. And that held for a long time. Like, that held for the first few decades. And then even when computers started to get smaller. So then you had so called mini computers was the next phase.

That was a computer that didn't cost $50 million. Instead it costs $500,000. But even still, $500,000 is a lot of money. People aren't putting minicomputers in their homes. It's like mid sized companies can buy mini computers, but certainly people can't.

Then, of course, with the pc, they shrunk down to dollar 2500. Then with the smartphone, they shrunk down to dollar 500. Then sitting here today, obviously, you have computers of every shape, size, description all the way down to computers that cost a penny. You know, you've got a computer in your thermostat that basically controls the temperature in the room. And it probably cost a penny.

And it's probably some embedded arm chip with firmware on it. And there's many billions of those all around the world. You buy a new car today, it has something new cars today have something on the order of 200 computers in them, maybe more at this point. And so you just basically assume with the chip today, sitting here today, you just kind of assume that everything has a chip in it. You assume that everything, by the way, draws electricity or has a battery because it needs to power the chip.

And then increasingly, you assume that everything's on the Internet because basically all computers are assumed to be on the Internet, or they will be. As a consequence, what you have is the computer industry today is this massive pyramid and you still have a small number of these supercomputer clusters or these giant mainframes that are the God mainframes. Then you've got a larger number of minicomputers, you've got a larger number of PCs, you've got a much larger number of smartphones, and then you've got a giant number of embedded systems. It turns out the computer industry is all of those things.

Size of computer do you want is based on what exactly are you trying to do and who are you and what do you need? And so if that analogy holds, it basically means actually we are going to have AI models of every conceivable shape, size, description, capability based on, trained on lots of different kinds of data, at running at very different kinds of scale, very different privacy, different policies, different security policies. You're just going to have enormous variability and variety and it's going to be an entire ecosystem and not just a couple of companies. Yeah. Let me see what you think of that.

Ben Horowitz
Well, I think that's right. And I also think that the other thing that's interesting about this era of computing, if you look at priors of computing from the mainframe to the smartphone, a huge source of lock in was basically the difficulty of using them. So, you know, nobody ever got fired for buying IBM because, like, you know, you had people trained on them. You know, people knew how to use the operating system like it was, you know, it was just kind of like a safe choice due to the massive complexity of like dealing with a computer. And then even with the smartphone, like the re, you know, why is the Apple computer smartphone so dominant?

You know, what makes it so powerful as well? Because, like, switching off of it is so expensive and complicated and so forth. It's an interesting question with AI, because AI is the easiest computer to use by far. It speaks English. It's like talking to a person.

And so, like, what is the lock in there? And so are you completely free to use the size, price, choice, speed that you need for your particular task or are you locked into the God model? And, you know, I think it's still a bit of an open question, but it's, it's pretty interesting in that that thing could be very different than prior generations. Yeah, yeah, that makes sense. And then just to complete the question, what would we say?

Marc Andreessen
So, Ben, what would you say are lessons learned from the Internet era that we live through that would apply that people should think about? I think a big one is probably just the boom bust nature of it that look, you know, the demand and the interest in the Internet, the recognition of what it could be, was so high that money just kind of poured in in buckets. And then the underlying thing was in Internet age was the telecom infrastructure and fiber and so forth got just unlimited funding and unlimited fiber was built out. And then eventually we had a fiber glut and all the telecom companies went bankrupt. And that was great fun, but we ended in a good place.

Ben Horowitz
And I think that something like that's probably pretty likely to happen in AI, where every company is going to get funded. We don't need that many AI companies, so a lot of them are going to bust. There's going to be a huge investor losses. There will be an overbuild out of chips for sure at some point, and then we're going to have too many chips. Some chip companies will go bankrupt for sure.

And then, you know, and I think probably the same thing with data centers and so forth. Like, we'll be behind, behind and then we'll overbuild at some point. So that that'll all be very interesting. I think that, and that's kind of the, that's every new technology. So Carlotta Perez has a great kind of has done, you know, amazing work on this where, like, that is just the nature of a new technology is that you overbuild, you underbuilding, then you overbuild, and, you know, and there's a hype cycle that funds the build out, and a lot of money is lost, but we get the infrastructure, and that's awesome because that's when it really gets adopted and changes the world.

I want to say, you know, with the Internet, the other, the other kind of big kind of thing is the Internet went through a couple of phases, right? Like, it went through a very open phase, which was unbelievably great. It was probably one of the greatest booms to the economy. It, you know, it certainly created tremendous growth and power in America, both, you know, kind of economic power and soft cultural power and these kinds of things. And then, you know, it became closed with the next generation architecture with, you know, kind of discovery on the Internet being owned entirely by Google and, you know, kind of other things, you know, being owned by other companies.

And, you know, AI, I think, could go either way. So it could be very open or like, you know, with kind of misguided regulation. You know, we could actually force our way from something that, you know, is open source, open weights. Anybody can build it. We'll have a plethora of this technology will be like, use all of american innovation to compete, or we'll, you know, we'll cut it all off.

We'll force it into the hands of the companies that kind of own the Internet today. And, you know, and we'll put ourselves at a huge disadvantage, I think, competitively against China in particular. But, but everybody in the world. So, so I think that's, that's something that definitely, you know, that we're involved with trying to make sure it doesn't happen, but it's a real possibility right now. Yeah.

Marc Andreessen
The sort of irony is that networks used to be all proprietary when they opened up. Yeah, yeah, yeah, right. Landman, Appletalk, Netbuoy, Netbios. Yeah, exactly. And so these are all the early proprietary networks from all individual specific vendors.

And then the Internet appeared in kind of TCP IP and everything opened up. The AI is trying to go the other, I mean, the big companies trying to take AI the other way. It started out as, like open, just like, basically, just like everything was open source in AI. Right? Right.

And now they're trying to, they're trying to lock it down. So it's, it's a, it's a, it's a fairly nefarious turn of events. Yeah, yeah, very nefarious. And, you know, I can, it's remarkable to me. I mean, it is kind of the darkest side of capitalism.

Ben Horowitz
When a company is so greedy, though, they're willing to destroy the country and maybe the world to, like, just get a little extra profit. But, you know, and they do it like the really kind of nasty thing is they claim, oh, it's for safety. You know, we've created an alien that we can't control, but we're not going to stop working on it. We're going to keep building it as fast as we can, and we're going to buy every fricking GPU on the planet. But we need the government to come in and stop it from being open.

This is literally the current position of Google and Microsoft right now. It's crazy. And we're not going to secure it. So we're going to make sure that, like, chinese spies can just, like, steal our chip plans, take them out of the country, and we won't even realize for six months. Yeah, it has nothing to do with security.

It only has to do with monopoly. Yes. The other, you know, just. Ben, going back on your point of speculation, so there's this critique that we hear a lot, right? Which is like, okay, you idiots.

Marc Andreessen
Basically, it's like, you idiots, you idiots. Entrepreneurs, investors, you idiots. It's like there's a speculative bubble with every new technology. Like, basically, like, when are you people going to learn to not do that? Yeah, and there's an old joke, there's an old joke that relates to this, which is the four most dangerous words in investing are this time is different.

The twelve most dangerous words in investing are the four most dangerous words in investing are this time is different. Right? Like, so, like, does history repeat, does it not repeat the my sense of it? And you reference Carlotta Perez's book, which, which I agree is good, although I don't think it works as well anymore. We can talk about sometime, but is a good, at least background piece on this.

It's just incontrovertibly true. Basically, every significant technology advance in history was greeted by some kind of financial bubble, basically, since financial markets have existed. And this, by the way, this includes everything from radio and television, the railroads, lots and lots of prior, by the way, there was actually a so called, there was an electronics boom bust in the sixties. It was called tronics. Every company had the name tronics.

And so, you know, there, there was that. So there, you know, there was like a laser boom bust cycle there, there were all these like, boom bust cycles. And so basically it's like any new tech, any new technology, that's what economists call a general purpose technology, which is to say something that can be used in lots of different ways. Like it inspires sort of a speculative mania. And, you know, and look, the critique is like, okay, why do you need to have this speculative mania?

Why do you need to have a cycle? Because, like, you know, people, you know, people, some people invest in the things, they lose a lot of money. And then there's this bus cycle that, you know, causes everybody to get depressed. Maybe it delays the rollout. And it's like two things.

Number one is like, well, you just don't know, like, if it's a general purpose technology like AI is, and it's potentially useful in many ways. Like, no, nobody actually knows upfront, like what the successful use cases are going to be or what successful companies are going to be like. You actually have to, you have to learn by doing. You're going to have some misses. That's venture capital.

Yeah, exactly. Yeah, exactly. So, yeah, the true venture capital model kind of wires this in, right? We basically, in core venture capital, the kind that we do, we sort of assume that half the companies fail, half the projects fail. And if any of us, if we.

Ben Horowitz
Or any of our fail, completely lose money. Lose money. Exactly. Yeah. And of course, if we or any of our competitors could figure out how to do the 50% that work without doing the 50% that don't work, we would do that.

Marc Andreessen
But here we sit, 60 years into the field, and nobody's figured that out. So there is that unpredictability to it. And then the other kind of interesting way to think about this is like, okay, what would it mean to have a society in which a new technology did not inspire speculation? And it would mean having a society that basically is just, like, inherently, like, super pessimistic about both the prospects of the new technology, but also the prospects of entrepreneurship and people inventing new things and doing new things. And, of course, there are many societies like that on planet Earth that just fundamentally don't have the spirit of invention and adventure that a place like Silicon Valley does.

And are they better off or worse off? And generally speaking, they're worse off. They're just less future oriented, less focused on building things, less focused on figuring out how to get growth. And so I think there's at least my sense there's it comes with the territory thing. We would all prefer to avoid the downside of a speculative boom bust cycle, but it seems to come with the territory every single time.

And at least I have not, no society I'm aware of has ever figured out how to capture the good without also having the bad. Yeah. And, like, why would you? I mean, it's kind of like, you know, the whole western United States was built off the gold rush. And, like, every kind of treatment in, like, popular culture of the gold rush kind of focuses on the people who didn't make any money.

Ben Horowitz
But there were people who made a lot of money, you know, and found gold. And, you know, in the Internet bubble, which, you know, was completely ridiculed by, you know, kind of every, every movie, if you go back and watch any movie between, like, 2001 and 2004, they're all like, how only morons did.com and this and that and the other. And there were all these funny documentaries and so forth. But, like, that's when Amazon got started. You know, that's when eBay got started.

That's when Google got started. You know, these companies, you know, that were started in the bubble. Well, in the kind of time of this great speculation, there was gold in those companies. And if you hit any one of those, like, you funded, you know, probably the next set of companies, you know, which included things like, you know, Facebook and X and, you know, snap and all these things. And so, yeah, I mean, like, that's just the nature of it.

I mean, like, that's what makes it exciting. And, and, you know, it's just a mix. It's an amazing kind of thing that, you know, look, the transfer of money from people who have excess money to people who are trying to do new things and make the world a better place is the greatest thing in the world. Like, and if we, some of the people with excess money lose some of that excess money in trying to make the world a better place, like, why are you mad about that? Like that.

That's the thing that I could never understand. Like, why would you be mad at, you know, young, ambitious people trying to improve the world, getting funded and some of that being misguided. Like, why is that bad? Right. Right.

Marc Andreessen
As compared to. Yeah, as a compared, as, especially as compared to everything else in the world and all the people who are not trying to. Money. So you'd rather like we just buy, like, you know, lots of mansions and boats and jets. Right?

Like, what are you talking, right, exactly. Or donate money to ruinous causes. Right. Such as ones that are on the news right now. Okay.

So. All right, we're at a minute 20. We made it all the way through four questions. We're doing good. We're doing great.

So let's call it here. Thank you, everybody, for joining us. And I believe we should do a part two of this, if not parts three through six because we have a lot more questions to go. But thanks, everybody, for joining us today. All right.

Ben Horowitz
Thank you.

Build Your Startup With AI

Primary Topic

Episode Summary

Main Takeaways

Episode Chapters

1: Introduction

2: Deep Dive into AI Strategies

3: Ethical and Competitive Considerations

Actionable Advice

About This Episode

People

Companies

Books

Guest Name(s):

Content Warnings:

Transcript