The nature of work in the Age of AI

The work we do will change a lot. Jobs like “prompt engineer” have left as quickly as they have arrived, but others, like “AI doom-poster on X” are here to stay. Products like Intercom’s Fin agent have shown us how capable AI already is and how it can be orchestrated to do real work. In this piece, I will examine the changing world of work, looking at how human jobs are evolving alongside the increasing capabilities of AI.

AI systems and humans work in fundamentally different ways, but it can be very tempting to interact with them in the same way, which would be a mistake. AI has limitations that are not intuitive or immediately obvious. AI responses are also generated very quickly, which can lead to laziness in prompting and planning. However, if people spent the same amount of time planning, prompting, and reviewing AI work as they do human work, I believe the results would be as good, if not better than those of a junior employee.

Over time, expert humans find their workday comprises planning and reviewing with little execution; at least, this has been my experience with software development. I now delegate the vast majority of software development tasks to my collection of agents. AI's exhaustive memory, attention to detail, and speed of iteration mean that I operate in a supervisor role, ensuring that the agent stays on task and helping it evaluate more nuanced architectural decisions. My agents run in parallel, and my time is now spent directing them or checking their work.

This trend away from execution and towards verification raises important questions about the nature of verification. Once we better understand verification for a piece of work, we can start to understand what the future of this work might look like. Some questions we can ask are:

• How do we verify this piece of work?
• Is a human required to do the verification?
• Is the verification expensive?
• Is it even possible to verify this piece of work?

How do we verify this piece of work?

Imagine a piece of code is written by a human or by an AI. How can we autonomously check if the code is correct? We might do code analysis to spot errors, simulate how a real user would use the code with an “end-to-end” test, and check if the code compiles. If all of these checks pass, we can be highly confident this is a valid snippet of code – our confidence comes from the diversity of ways to analyse and test the code. The success and adoption of AI coding agents are a result of the excellent code that they write, but more importantly, their ability to autonomously check that what they produce is valid.

Is a human required to do the verification?

This will be particularly salient in the coming years, given the popularity of AI in regulated fields like medicine, law, and finance. Here, the stakes are high, and mistakes are heavily penalised. As AI systems become more capable, though, I think it is highly likely we will see a new class of insurance products emerge specifically for AI agents. Humans, like AI, make mistakes, and that is why in high-stakes jobs, we mandate insurance to protect themselves and the people they harm.

Insurance for AI will become popular soon given that is seems to be a straightforward calculation. As a risk underwriter, the factors I care about are: what is the chance of a given event, the cost of the event, and the propensity of my competitors to offer insurance for that event at a given price.

Let’s look at an example to illustrate what I mean. Imagine I want to insure my house against flooding. Historically, there is a 10% chance of flooding in my neighbourhood, and if it floods, it will cost $100,000 to repair my house. This means that to break even on this, an insurer will need to make at least $10,000 (100,000 x 0.1 = 10,000). The most the insurer could charge is $100,000.

In this hypothetical scenario, imagine there is a competing insurance firm in the market. The other firm refuses in principle to insure houses against flooding, though, which means that the insurer is free to charge what they like.

With this in mind, let’s return to our AI insurance idea. Companies which build AI doctors and AI lawyers will want to insure against mistakes and bad outcomes, but because of regulatory or philosophical aversions to insuring AI, there will be fewer firms which offer this. The ones who do, can charge a premium.

This is particularly interesting given that some AI systems are already safer than humans, and will only get more accurate, like when AI doctors had 91% accuracy in differential diagnosis, compared to 75% for humans, Goh E, Gallo R, Hom J, et al. 2024. These accurate AI systems will result in fewer insurance payouts, all while paying inflated insurance premiums.

This is all to say that AI adoption in regulated fields will be even slower than one might expect otherwise.

Is Verification Expensive?

Some verification, like running a code compile check, will cost microcents of energy, and other verification, like drug discovery trials, will cost billions. As verification methods mature, qualitative checks will be used to verify a piece of work. This might look like a seasoned writer reading over an AI-generated essay, a doctor analysing an AI diagnosis, or a lawyer sifting through AI-generated legal arguments. Here, we are constrained by expert human labour.

The cost of verification is part of a broader class of problems which relate to bottlenecks. In some cases, the bottleneck in an industry is intellectual labour, but often there are regulatory, physical, and sociological barriers which prevent adoption.

Is it even possible to verify this piece of work?

There is a particular class of work which relates to art, taste, and strategy, which is difficult or impossible to verify. In each of these cases, it is hard to see how society will interact with the decision that was made or the sculpture that was produced. The litany of terrible movies, music, and advertisement campaigns attests to this. I'm sure we all remember when Kendall Jenner solved racism with a can of Pepsi.

Although verification may be difficult, it is still possible to produce art and strategy which are opinionated and differentiated. Consider for a moment some of history’s famous artists and founders. Often, they have a singular and unique view of how the world should be, and they make non-concensus decisions. Steve Jobs famously refused to do market surveys of new product features. Netflix cannibalised its DVD shipping business in favour of nascent web streaming. Hallowed artists like Vincent Van Gogh were considered unremarkable in their time.

As difficult as it sounds to be a visionary, I believe AI can produce a similar unwavering taste if given the right strategy:

First, we ask AI, “Create a rubric to evaluate highly opinionated decisions, like something that Steve Jobs or Marc Randolph would have used”.

Next, we instruct AI to “Produce 20 highly opinionated strategies”.

Finally, we request AI to “Rank all 20 ideas according to the rubric”.

In other words, we can get AI to search over the text space of possible opinionated strategies and autonomously evaluate them according to a predefined metric.

While the system described above may give promising strategies, verification will still be challenging given the long time horizons associated with questions of strategy and taste. This uncertainty, combined with high opportunity costs will prevent AI from being the ultimate decision maker in this area until we have more reliable methods of verification.

Despite AI's limitations in this area, it will be used by decision makers, artists, and founders to iterate, prototype, and research significantly faster than before. It will reduce barriers to entry and enable new kinds of expression and experimentation.

Ultimately, we can predict the rate at which AI will change a given job or industry by how easily its output can be verified. Our work will shift from doing to verification. Expert humans will have the greatest leverage, as they can most quickly review complex outputs in their field.

In medicine, education, and law, informal uses of AI will proliferate quickly while institutional adoption will take decades. For readers interested in thriving in the age of AI, I recommend becoming intimately familiar with AI to deeply understand its limitations and strengths. Use it daily on problems in your professional and personal life, and think about how you would have responded differently if you were the AI. It will also be valuable to pick an area where verification is difficult, expensive, or forbidden.