Open The Wound

I predicted Sam Altman’s fall. Here are the deets: AI comes in two flavors, “Type 1” and “Type 2”, and the trade-off is science versus fraud

To be clear, I’m under NDA with multiple firms mentioned here, some of the largest are our customers, and therefore I’ll be careful to redact some details. Leave that as an exercise for the interested reader. Nevertheless, this story needs to be told, now, otherwise we’ll end up with balkanized narratives which obscure a much larger and deeper structural problem. In other words, individual stories will get sanitized by corporate Legal and Publicity teams, then end up sounding perfectly reasonable. Boardroom skirmishes, that sort of thing — move along, nothing to see. Overall, none of this is reasonable.

Conference Video Picks

Before we dive into details: it’s been a busy AI conference season and session videos are beginning to go public. Here are my top picks for the AI Conference in San Francisco, chaired by Ben Lorica. This picks up where O’Reilly Media left off with its conference series of the same name. These new editions of AI Conference are produced by Courtney and Shon Burton who ran MLconf and early Spark Summit events. They know how to create a packed and meaningful event:

• 2023 videos + slides
• YouTube playlist

Ben and I recorded the “Reflections from the First AI Conference in San Francisco” episode for his podcast shortly after the event, where we take turns suggesting and critiquing key takeaways.

Notably, conference registrations sold out fast, and also the demographics were totally unexpected. Roger Chen ran an informal poll during his VC panel, and no one anticipated these results:

• ~5% fresh-out / early career
• > 20% senior discount crowd (like me)
• > 50% mid-career 40s

For a tech conference in San Francisco, these demographics are unheard of. Generally you’d expect a large cohort in mid–20s, since that’s who tends to flock to SF and also given the topic. I met several women in their 70s, experts in their respective fields, at an AI conference. That’s a wonderful change! But why? Spot-check conversations with attendees showed that many people in the older cohorts had solid business experience, mostly not in technology, and coming from other regions. They heard about the AI Conference and rushed to buy tix and book flights. Younger folks I talked with tended to delay registering, and perhaps many couldn’t get in late.

I got to emcee in the large room, where my top picks were:

• “The Next Million AI Systems” by Mark Huang @ Gradient; talking about “Mixture of Experts”, especially restacking embeddings during RAG

• “Gradient-Free Structured Pruning With Unlabeled Data” by Azade Nova @ Google DeepMind; survey of latest work in shrinking models

• “Practical Data Considerations for building Production-Ready LLM Applications” by Jerry Liu @ LlamaIndex; excellent overview and insights about RAG

• “LLMs in Production: Learning from Experience” by Waleed Kadous @ Anyscale; how to build AI systems by orchestrating several smaller, specialized models – introducing Anyscale Endpoints

• Panel Discussion: Investors perspectives on AI led by Ben Miller @ Fundrise, with Anarghya Vardhana @ Maveron Ventures and Tomasz Tunguz @ Theory Ventures; imagine three experienced VCs brainstorming privately about AI ventures which they would fund today (fast and furious note-taking by the audience, especially whenever Anarghya spoke)

NB: Waleed got swamped by people as he tried to walk down from the stage. An hour later he was still fielding questions near the green room. I’ve introduced many speakers over the years, and I’ve never witnessed that level of audience deluge.

OpenAI did not participate. Apparently they were not missed, or at least I didn’t hear a soul ask about them, and I was in the midst of so very many conversations.

Hugging Face emerged as the clear leader for the open source AI model ecosystem. HF works to identify any friction points in the ecosystem and provide services to address these. Clem Delangue and others are super-supportive of startups, and collectively HF has provided guidance and core insights. Even while tech billionaires, being relatively hostile to this ecosystem, were doing what they could to muddy the waters.

My main takeaways:

1. Teams with ROI in ML focus on task analysis up-front. Identify narrowly-defined tasks which people on your team perform all day every day. Find ways to augment these tasks with AI apps. These will be tasks for which you can define evals. Don’t try to build a large app encompassing many tasks, and don’t build anything unless you provide the evals for it.
2. Dataset quality is vital. You may train models with your datasets. If you have enough data, it’s more likely that you’ll fine-tune models with your datasets. Otherwise, you may use few-shot or RAG with your datasets, and in any case you’ll need to run evals with your datasets. There are 4–5 reasons why your dataset quality is crucial for AI app success. While the crypto-turned-AI bros fixate on model-centric views, the successful business units are almost entirely data-centric.
3. Apps, especially the datasets, require domain experts. However, worldwide across the verticals the domain experts are rapidly aging out of the workforce. This point is top-of-mind for leading execs, more so than any other risk. Of course, it feeds into dataset quality.
4. Successful teams in industry tend to leverage mixtures of small, specialized models in lieu of a OSFA ginormous god model.

During this same period, a couple reports about AI surfaced. “The Openness of AI” report by Kyle Harrison, et al., @ Contrary is thoroughly researched and draws excellent insights. It’s such a breath of fresh air, in contrast with the fanciful tome “The State of AI Report” by Nathan Beniach, et al., which proved disappointing this year.

A glaring point that Harrison missed is that the tech giants in this field all share a common area of expertise: online marketing. Advertising drives both Alphabet and Meta, almost to the exclusion of other business. Increasingly, Microsoft depends on ad revenues through its various holdings. Apple is one hop away from its app ecosystem which depends on advertising. While the hyperscalers may have spent the past several years promoting notions that they dominate the field of AI, keep in mind where they really dominate is in promotion: getting people to believe marketing points and respond on calls to action. That’s what they do best. That’s what they’re doing when execs and tech giants make wild claims about AI which cannot be independently confirmed.

The other odd commonality among hyperscalers involves open source. Notably, Alphabet spent obscenely large sums marketing Tensorflow and Kubernetes. Meta pushed PyTorch and React, while demanding backchannel deals to guarantee they wouldn’t be heckled at open source conferences. These different tools each hit in different parts of the tech stack, though they’ve become de rigueur selections for enterprise dev teams. Rumor is that NVIDIA and Amazon have been rather quietly working together to dislodge the Kubernetes kool-aid, showing a better path forward for cluster computing. Tensorflow and PyTorch may be giving way to smaller, more purpose-built tooling, which seems inevitable. Meanwhile, VanillaJS is making a comeback, placing a big question mark over React. Alphabet and Meta may have marketing muscle, but their weird flexes in open source haven’t quite solidified the hyperscalers’ positions.

Another must-read report is “Evaluating LLMs is a minefield” by Arvind Narayanan and Sayash Kapoor @ Princeton, from the authors of AI Snake Oil. If you’re listening to “experts” opine about “AI”, drop what you’re doing and read this analysis. Be especially sure to read slides #14-ff. More about that in a bit.

Two Kinds of AI

During the two week period of 06–17 November 2023, AI hype-fueled techno optimists collectively lost their minds, reeling from news which appeared to them alternately as mana from heaven followed by utter devastation. Their perceived leaders fell, in quantity. To understand why and how this happened …

There two kinds of “AI”:

Type 1 — AI apps: These kinds of apps include DeepMind’s AlphaFold protein folding and GraphCast weather forecasting. Also work by AstraZeneca, Novartis, and others to make drug discovery more effective. Plus a flurry of apps throughout different business verticals. One of my favorites is Legislate helping SMBs understand their contracts with vendors, etc. Another is Furuno working on small boats with individual fishermen in Japan to model their hyperlocal first-hand knowledge of sustainable fishing practices, models which the fishermen then own. When you attend K1st World or Corunna Innovation Summit you get excellent views of these kinds of AI applications.

Type 2 — AI hype: OpenAI’s former CEO pushed a worldview of “superintelligences” and “Artificial General Intelligence” (AGI) and other utter nonsense designed to prop up valuations for libertarian billionaire tech bros. This is a world where “chat” becomes the dominant mode of interaction. This is a world where some ginormous OSFA god model is available via SaaS, and everyone else donates their most critical data to the cause. This is a world where most of the business and technology leaders subscribe to some flavor of eugenics and accelerationism. This is a world of armies of libertarian tech bro trolls — fanbois of said billionaires — move out to harass and threaten anyone who challenges their worldview, following a MAGA playbook.

In terms of confirmed examples of generative AI in industry, the groundswell of the “Type 1” world of AI apps has been building. Gradually then suddenly.

The “Type 2” world, however, became a kind of forced narrative, as in what would typically spew from Su Excelencia el Jefe de Estado, promoted by Altman and his ilk. Note that “AGI” is effectively a dogwhistle for the right-leaning tech bros, almost as a placeholder for said eugenicists and accelerationism. Somehow that seems to resonate especially well among the previously-crypto.

The “Type 1” vs. “Type 2” bifurcation appears to have had origins in Redmond. Microsoft execs understood several years ago that, MSR’s excellent research work notwithstanding, they’d fallen far behind Alphabet and even Meta in terms of AI capabilities.

Among the big tech “gatekeeper” firms, aka the hyperscalers, Alphabet has consistently led in this field. For better or for worse ethically, they’ve led in technology development, aggressive hiring, research funding, and famously accumulating massive amounts of data. Alphabet has also led in an area which not as many people talk about, namely their patent filings and aggressive IP acquisitions; this is an area where I have first-hand experience, for better or for worse. Perhaps Mountain View’s strategy is preemptive, to avoid having other organizations file software patents and choke the field. So we’ve been told. In any case, many of the results of their AI research programs have been impressive, such as the aforementioned apps from DeepMind. The trouble is that product management at Alphabet sucks. They struggle to commercialize their brilliant research results.

Meta has Yann LeCunn, who is apparently one of the only “grandfathers” of deep learning who hasn’t gone insane. Yet. If you listen to A16z, particularly co-founder Ben Horowitz waxing teary-eyed on the subject, Mark Zuckerberg is a hero for having stood up to to systematic “deep state” threats while waving the banner of openness and single-handedly making a decision to push AI models open source. Or something. Meanwhile we’re supposed to forget about the 2010s and the Cambridge Analytica scandal.

Ben Horowitz is full of crap, if we’re being candid.

Back to Microsoft, given how they’re a member of the trillion-dollar hyperscaler club, what do they have to fret about? Namely:

• recurrence of antitrust law cases, revisiting their 2001 corporate nightmare
• any reasons for Wall Street analysts to downgrade said multi $T market capitalization
• regulatory compliance fines measured north of nine figures
• losing further Azure cloud market share to Amazon and Alphabet

Okay, so a multi-trillion-dollar firm coming in third probably does not sit well with Wall Street. Looking ahead, the view that Microsoft execs foresaw looked bleak. They needed a hail mary. They made a trillion dollar bet.

One Trick Pony

We’ll use an analogy here: consider the relationship between Beijing and P’yŏngyang. Whenever supreme leader KJU jumps up and starts rattling sabres geopolitically, threatening to aim nuclear-armed ICBMs at Japan and Hawaii, two things happen. One: people at the US State Department scramble for damage control. Two: Beijing laughs all the way to the bank. Because the next time they enter the high-stakes poker game of bilateral trade talks with Washington, Beijing will have another ace up their sleeves.

Redmond appears to have made a similar arrangement. One: Sam Altman would jump up on a soapbox, expounding about superintelligences, or ancient alien astronauts, or some eyeball-scanning crypto coin to deploy worldwide on starving masses … and execs in Mountain View would scramble. Two: Redmond would laugh all the way to the bank as their stock valuation climbed, nearly exponentially over the past several years.

Microsoft has the product management which Alphabet lacks. They leveraged OAI to gain cloud market share for Azure. They got execs worldwide tied up in bondage knots, drooling about a “Type 2” AI worldview.

Somewhere around February 2023, exactly that began to happen. Exec staff and Board members at large firms worldwide began playing with ChatGPT. Many dreamt publicly about the “Type 2” AI hype affording them once-in-a-lifetime corporate opportunities to gain massive productivity increases concurrent with large layoffs. Because our friend Sam Altman was out promising corporate executives exactly that. AGI devotees such as Greg Brockman had founded OpenAI on those kinds of promises. A new era was allegedly just around the corner, much like Level 5 vehicle autonomy. Gradually, then perpetually.

Greg Brockman is full of crap, if we’re being candid.

Never mind how, if one does the math on Altman’s pretenses, the economics tend to not work out so well for corporate execs in particular. A general purpose technology which tears down the need for domain expertise similarly tears down barriers to entry and business differentiation. That said, corporate execs are rarely assumed to be the sharpest tools on the workbench, let alone good at math.

This collective fantasy ran through 2023 until about August-ish. Meanwhile, lots of IT projects had funding held — pending more investigation of generative AI. Then, staring at the looming implications of reaching Q4 2023 without tangibles, execs turned the money supply back on. Technology projects have leapt since the start of Q4, albeit on the tailwind of a relatively lean year for so many smaller tech firms and consultants.

Move Fast, Break Things

Meanwhile, what did Microsoft gain from this gambit, aside from their stock climbing and Azure bursting at the seams?

AI researchers inside Alphabet issued a “Code Yellow” warning, which was widely reported in business media as a “Code Red” warning from CEO Sundar Pichai in early 2023, but whatevs. OpenAI’s GPT–4 and ChatGPT in particular had seized headlines about AI which rightfully belonged to Alphabet, due allegedly to the superiority of their research, the power of their accumulated data, and a bunch of internal tooling and obscure processes which not many people outside of Alphabet know much about. This came in the wake of Alphabet dismissing their entire AI ethics team in 2021, after a paper surveyed the pitfalls of GPT–3 vis-a-vis racism then warned the company not to repeat Microsoft’s earlier mistakes with chatbots gone bad. Fear crept through the tech giant. The relatively independent DeepMind unit got merged into the Google Brain team a quarter later. According to business media, broad changes were made, product plans accelerated, and Alphabet reconfigured in reaction to Microsoft’s ploy.

Meta somehow managed to release Llama2 mid-year, whether Zuke initiated that tactic or not. In any case, here was another hyperscaler reacting to Microsoft’s ploy.

Amazon meanwhile tried to stay focused on business, in other words resisting Altman-flavored kool-aid. However, eventually execs in Seattle decided to join the game. They did own Alexa, after all, and could begin using LLMs there, even if it didn’t fit. Another hyperscaler reacting to Microsoft’s ploy.

Oracle, now clearly an also-ran tech firm but with large revenues and strong right-wing political ties, launched its own AI initiatives. Chalk up an additional victory for Redmond, and keep the KJU clone in the headlines. Microsoft had clearly succeeded at prompting (pun intended) other tech giants to take actions and restructure, reactively not proactively, on Microsoft’s terms.

Call The Ball

On 14 October, I made a little prediction, talking with a former co-founder, which got copied elsewhere:

It’s astounding to see Sam Altman getting so many headlines. Basically, he exists because MSFT made a $1T bet hinged on the ability to throw Sam Altman under a bus when the time is convenient for MSFT.

Enter the first week of November 2023. OpenAI held their Developer Day in San Francisco on 06 Nov. Teary-eyed libertarian tech bro fanboys were absolutely ebullient in response to the perfunctory demos. CEO Satya Nadella made a surprise appearance. Our glorious AGI future was imminent, probably arriving sooner than expected!

Some industry experts were nonplussed. Jeremy Howard @ Fast.AI called Dev Day an “absolute embarrassment”. Matt Turck @ FirstMark summarized OpenAI’s attempted big land grab:

• “We do all things consumer AI”
• “We do all things enterprise AI”
• “We do all AI things for developers”
• “Oh we’re also an App Store”

Perhaps other leaders were similarly nonplussed. Nine days later and fifty kilometers south, President Biden and President Xi Jingping held a summit where the two world leaders spent a considerable amount of their precious face-time discussing AI and its geopolitical implications.

Unfortunately, Altman had spent much of the previous year or so “charming the socks off” legislatures and parliaments on two continents. He and his ilk had also ventured into funding political campaigns. The “Type 2” AI hype worldview seemingly became a basis for US and EU regulatory controls for AI. Controls for which hyperscalers pleaded a “We alone can fix this” case to alleviate “AI’s existential threats to humanity” which they ostensibly alone had created. Controls which would provide regulatory barriers to entry for would-be competitors. Altman’s lobbying efforts were of course based on published research, which provided perfectly credible assets for bipartisan evidence-based policymaking.

Five Letters, Begins with “F”

Now there’s a subtle problem. Generally speaking in computer science we like to have research publications get peer-reviewed. Of course, when the research is predicated on ginormous computing clouds which only a handful of hyperscalers wield, those published research results may become rather difficult to reproduce.

Unfortunately, there was a more subtle problem here. Current estimates place the reproducibility rates of ML-related research around 3% — for the fraction of papers which even bother to publish their source code. When you work for a billionaire, you can get research published in allegedly peer-reviewed journals, even without anyone attempting to run your code. Of course the “open” in OpenAI is a misnomer. Their code is anything but open, due of course to enlightened concerns about bad people doing bad things with AI, or some such. More poignantly, we don’t know what data is getting used to train ginormous god models, nor do we know what data and process is used for testing. Without a full set of published research plus source code, data, and evals … any research claims could be possible although effectively unaccountable. Ergo, “Type 2” research claims can be admitted as foundations for evidence-based policymaking, especially when said billionaires are personally hobnobbing with Congress and funding campaigns.

Weird flex but OK?

One risk, perhaps one which Redmond did not foresee sufficiently, was that this whole grift could suddenly skid sideways when national security concerns for the US, China, EU, etc., began to reek of the same flavor of kool-aid you happen to be selling. That’s precisely where the US and China became entangled, in the wake of the Chips Act.

Oddly enough, two days after the US-China summit, on 17 Nov, shit got real.

Altman got ousted suddenly from OpenAI. Brockman and several other key execs resigned in protest. Then maybe Altman’s back, or not?

These moves seemingly confounded Microsoft execs, sending corporate PR teams into damage control emergencies. I have a hunch their dialog behind closed doors differed substantially from what we’ll read in the news cycle.

On the same day, Amazon effectively rifted much of Alexa and other projects which had altered course in reaction to Microsoft’s gambit. Oracle took steps even earlier, with layoffs for its also-ran AI efforts. Meanwhile, Disney, Apple, IBM, and many others Twitter/X advertisers told Elon Musk “Oh, hells no!” and left the platform suddenly amidst threats from that libertarian tech bro billionaire. Quite a day, quite a day.

Two days later, Meta disbanded its responsible AI team.

There’s some next-level grift in play. Of course, press releases will carefully explain the sanitized and highly nuanced versions of stories about Board-level disagreements, data privacy concerns, miscommunications, etc. All deliciously plausible — if you happen to work at OpenAI in Legal, Marketing, or Investor Relations. Or “NopenAI” as it’s become called more recently.

One observation: it’s rare in the tech industry for the sleaze to expose their underbelly. On Friday, November 17, 2023, heartfelt outpourings of devotion to Sam Altman and his “Type 2” vision flooded social media. I counted multiple billionaires engaging. Droves of crypto-turned-AI bros shewed themselves, tears and all.

Another observation: maybe Microsoft crawled too far out on a proverbial limb? Did they overuse their Kim Jong Un dancing bear, enough to waken the ire of regulators? Enough to piss off Wall Street? We’ll see, but there’s no point in speculating too much just yet.

That said, much of industry wasn’t going to use ChatGPT in production, even in organizations where execs had engulfed Altman’s kool-aid. The liability of releasing critical data to a third-party loomed too large. The probability of hallucinations for an unaccountable third-party service also hung over their public company heads like a dark cloud. Azure probably wasn’t going to get that big bump in cloud adoption, after all.

We may be at the start of an industry backlash against “Type 2” AI hype. It appears that right-leaning corporate insiders pulled those triggers — not unlike how a particular right-leaning VC effectively launched (or accelerated early-on) the infamous 2023 bank run on SVB, if we’re being candid.

Meanwhile, if you have a long position in MSFT, you might want to have a quick convo with your financial adviser.

The research fraud is real, and that mess will endure. Risks from the “Type 2” worldview represent a clear and present danger, as Timnit Gebru and others have diligently warned — more about disinfo and grift than about technology concerns.

To wit, anyone who’s still using the term “AGI” in a non-ironic sense becomes suspect. Kudos to Ilya Sutskever for his past research accomplishments but the world simply does not need another unhinged tech bro fanboi.

We don’t need any of them. This situation calls back to the mid–2000s, when people who’d entered the industry during the latter DotCom Boom perpetuated a bizarre sense of entitlement well into their careers. At least now we know who not to hire.

Microsoft aside, at least two venture firms and one of the hyperscalers mentioned here are effectively criminal organizations. There’s no other word for it. Silicon Valley has an enormous ongoing problem, with professionals paid handsomely to sweep dirt under the run. Some PR firms on the peninsula even specialize in this role.

Our firm has severed ties and will refuse to do business with any of the AGI ilk, based on our due diligence. Based on their crypto accelerationism.

The next-level grift in play is at geopolitical scale. You don’t want to be caught on the wrong side of this one.

Free download provided by derwen.ai https://derwen.ai/dl/latent

newsletter sign-ups: https://tinyletter.com/derwen