MUMBAI: When your content becomes someone else’s main course, it’s only fair to send them the bill. Publishers and advertisers have long played the thankless role of feeding the internet’s knowledge machine. But with the rise of large language models (LLMs) and AI agents hoovering up open web content without so much as a nod—or a rupee—the tables are finally turning.
The IAB Tech Lab, the global technical standards-setting body for digital advertising, has taken a firm stand against the unchecked rise of generative AI. In a bold move, it announced a new framework dubbed the LLM Content Ingest API Initiative, aimed at helping publishers and brands monetise and manage how their content is consumed by AI systems.
“It is clear that AI agents powered by large language models are shifting how users engage with content”, said IAB Tech Lab CEO Anthony Katsur. “While this is a promising new way for people to access information, we have also seen data showing publisher traffic decreases at 15 per cent or higher, and revenue is down. Meanwhile, AI platforms are growing on the back of open web content, impacting publisher revenues and misrepresenting advertisers. This initiative is about giving publishers and brands a path forward that is fair, enforceable, and grounded in technical standards”.
The new framework targets AI’s relentless content scraping. It proposes a structured technical solution that ensures publishers get compensated for their content and brands control their digital reputation in AI-generated search summaries and chat responses.
At the heart of the initiative is the LLM Content Ingest API, a tool that enables content owners to manage how AI models ingest and present their material. The framework seeks to stem traffic and revenue losses and aims to empower publishers with tools to govern bot access, track AI scraping, and monetise their digital assets more effectively.
“The LLM Content Ingest API is one part of the framework to address the challenges publishers and brands face from the growing use of AI-based tools by consumers”, said IAB Tech Lab EVP, product & COO Shailley Singh. “The proposed technical framework is designed to foster better collaboration between LLMs/AI agents and content owners and to lay the foundations for fair value exchange for content, two of the biggest challenges that have emerged since the proliferation of generative AI and endanger the web economy as it works today”.
The IAB Tech Lab plans to kick off the effort with a workshop, bringing together publishers, brands and AI developers to co-create solutions. Among the talking points: managing bots that mine data without permission, building AI-friendly but fair delivery mechanisms, and ensuring brand information isn’t lost—or worse, misrepresented—in the LLM ether.
With publisher traffic already dipping by 15% and revenues following suit, the initiative couldn’t have come sooner. The Lab’s approach blends technical pragmatism with economic common sense, aiming to strike a balance between innovation and ownership in a world where chatbots can mimic your writing but never pay your bills.
