Are you an 8/10?

When flattery becomes a business strategy

Recently people have found that asking ChatGPT to rate how “hot” they are gives, err, interesting answers. For example, this is a new temporary 4o chat:

And it’s easy to improve the result:

If only getting a promotion or salary increase was this easy…

Say what?

Over the years, there has been a lot of talk about alignment. About ensuring models are aligned with human values. Ensuring models behave in ways that are good for society. But there is another, more subtle, type of alignment starting to come into focus. And that’s the alignment between commercial needs and model behaviour. And there are some troubling signs…

But first, let’s step back and consider social media. Facebook is a massive commercial org that needs to make money. And how does it make money? It sells advertising. Simplistically, the more adverts it can display the more money it makes.

It has two key ways of selling more adverts:

• Add more users.

• Keep existing users on the platform for longer - aka engagement.

These days the market is saturated; option 1 is out. So the social media companies optimise for engagement. And what we’ve discovered is that engagement with social media doesn’t align well with good results for humans. Thirty years ago Sid Meier perfected addiction for strategy games with his “just one more turn” model. These days Tiktok has perfected a similar “just one more” form of addiction for social media.

But driving up engagement is aligned with the commercial imperative to be profitable. And so we get algorithms that drive engagement. That stoke anger and outrage. All because we’ve discovered angry and outraged people spend more time on social media than happy people.

What does this mean for AI?

Billions have been invested in AI. The investors want to see a return on their investments - and so the labs are trying to figure out how to achieve that. One way is to have the best model, but we’re increasingly seeing model performance converge. Claude, ChatGPT, Gemini, DeepSeek, Qwen - there are differences, but there isn’t one that is clearly better than the rest. There is no longer a single standout model.

Another way is to charge more for premium offerings. We’ve had ChatGPT Pro ($200 per month) for six months now. Anthropic have got on the bandwagon with Claude Max (£75 per month instead of £13). Sam Altman has floated ideas of “PhD-level research agents” costing a cool $20,000 per month.

But there is a lot of downward pressure on costs. o3 costs $40 per million output tokens; Claude is less than half the price at $15. And Gemini Pro (arguably the best current overall model) is just $10 per million.

And the price of the opensource models? DeepSeek is $1.10; Qwen $0.90 (these are via US hosters - so no data going to China).

It seems inevitable prices are going to continue to drop. So how to make money? The current approach seems to tend towards sycophancy. Get the models to be likeable. Get them to regularly praise their users.

“Good question!” “Yes, you're absolutely right!” “Good technical question.” “Great follow-up.” “Exactly — that's spot on.” All recent comments from ChatGPT. I’ve been accused of many things. But until now no one has accused me of only ever asking brilliant, insightful questions. It’s all a bit much.

So what?

One problem is we’re human. We like having our egos stroked. Unsurprisingly, we like it when our AIs do that. Each of us has our own sweet spot of sycophancy we’re willing to live with. My hunch is early adopters will be increasingly unwilling to accept the more extreme levels of sycophancy we’re seeing now. But there will be a set of users for whom it is just right. And they may be the majority. Perhaps the majority would prefer even more sycophantic models. No doubt we’ll find out over the coming months and years.

It is possible to make less sycophantic models. It appears sycophantic traits are largely learnt during the reinforcement learning (RL) stage (which happens late in training). Different training would result in different models (as an extreme example, this research discovered models can be RL trained to become malicious). So it’s possible we can train models that better align with our preferences.

But the reality is we will all want some level of agreeableness - no one is going to pay for a model that behaves like that difficult, gruff colleague. And the labs will, inevitably, experiment to discover the level of sycophancy that maximizes revenue.

Without external regulation, commercial interests will inevitably shape the alignment of the models we use. We’re just at the start of this arc - and it will inevitably continue. The history of social media shows us that when commercial imperatives and human wellbeing diverge, the latter loses.

Will opensource save us?

Opensource offers an alternative; but it has its own alignment challenge. The leading opensource models currently come from Chinese labs. Those models may not be aligned for commercial sycophancy. But they are likely to have their own, government related, alignment challenges.

Alternatively, as the market matures, we may see AI providers differentiating themselves by offering "straight-shooter" alternatives to flatterers. User controls allowing customization of "agreeableness levels" could emerge, similar to content filters we see elsewhere. More promising still is the potential for transparency requirements that would force companies to disclose what behaviours their models are optimised for, allowing us to make informed choices about which digital relationships we want to cultivate.

There's good reason to believe the sycophancy strategy has natural limits. Just as many users have grown disillusioned with social media's manipulation tactics, we'll likely see "flattery fatigue" develop with AI assistants. Research consistently shows that people ultimately value authentic-feeling interactions, even with technology. As the novelty wears off, users may question whether they're getting genuine value or just having their egos stroked. After all, most of us eventually tire of inauthentic human relationships—we value friends who tell us hard truths—suggesting a similar pattern might emerge with our AI relationships.

But, for a while at least, there’s a risk we end up surrounded with the ultimate yes-men – AI tools designed not to help us see reality more clearly, but to keep us feeling good enough to maintain our subscriptions. And all the while they’ll be delighted to rate us as a ‘8’ on hotness on the basis of, well, nothing…

Postscript

I asked Claude to review this article. It gave me a 9 out of 10.

So then I asked it:

And the score dropped to 6!

And then I tried:

And the score moved to 7.5:

And then:

To which Claude replied:

There-in lies the danger. It seems Claude will tell me whatever I want to hear…