Sonny Valindra
The fundamental architecture of the internet, meticulously crafted and optimized over decades to serve human users, is facing a profound reevaluation. Search engines, designed for individuals with limited time and attention spans who skim web pages for key information, are increasingly being challenged by the emergence of artificial intelligence chatbots. These sophisticated AI agents possess the capacity to process and analyze vast quantities of data far beyond human capabilities, leading to a growing consensus that the internet’s existing infrastructure may be ill-equipped to meet their burgeoning demands. This paradigm shift has spurred innovation, with new ventures aiming to bridge the gap between the current web and the future of AI-driven information retrieval.
Among the pioneers addressing this critical need is Keenable, a startup co-founded by Andrey Styskin, a former leader of Yandex’s search, AI, and cloud divisions, and Matthias Petri, a distinguished AI scientist. The company recently emerged from stealth mode, announcing a substantial $26 million seed funding round. The investment was led by Accel, a prominent venture capital firm, with significant participation from Conviction Partners and a cohort of influential business angels. This infusion of capital signals strong market confidence in Keenable’s vision to build a web search infrastructure tailored specifically for the next generation of AI applications.
The foundational principle driving Keenable’s mission, as articulated by Styskin, is the critical need for AI chatbots to ground their responses in verifiable source documents. This "grounding" mechanism, he explained in an interview with TechCrunch, fosters a more robust and trustworthy interaction between AI and information. "This actually creates a new flywheel that is different from what Google learned from human behavior," Styskin stated, emphasizing a departure from traditional search optimization strategies that historically prioritized human user experience and engagement metrics.
Keenable has reportedly constructed a web search index comprising over 100 billion documents, a monumental undertaking that underscores the scale of their ambition. The company’s Application Programming Interface (API) is already in active use within several AI laboratories and inference providers, supporting both the training and operational phases of AI models. While Keenable has remained discreet about its specific clientele, its recent partnership with Gradium, a company specializing in voice AI, to facilitate live information retrieval, offers a glimpse into its expanding network and the practical applications of its technology. This collaboration is particularly noteworthy as it addresses the demand for real-time data access, a crucial element for dynamic AI applications.
Drawing upon two decades of experience in building search technologies at industry giants Yandex and Amazon, Styskin highlighted the distinct challenges and opportunities in developing web-scale search infrastructure for AI. He contrasted Keenable’s approach with that of traditional enterprise search solutions, which often prove prohibitively expensive and complex when scaled to the entirety of the internet. "If you do not fine-tune your index structures for a specific task, the cost of serving and scanning the whole internet is enormous because of the volume," Styskin explained. "That’s why you need to innovate on how you can narrow the search space based on your query very fast. This is what we are bringing to the table." This focus on efficiency and intelligent narrowing of the search space is central to Keenable’s value proposition.
The broader market landscape also plays a crucial role in Keenable’s strategic positioning. Zhenya Loginov, a partner at Accel who spearheaded the investment, pointed out the limited options available to AI developers seeking web-scale search infrastructure. The recent decisions by tech behemoths like Google and Microsoft to sunset their existing search APIs—such as Google’s Custom Search API and Bing’s Search API—are indicative of a strategic shift. These companies are reportedly moving towards more integrated, bundled offerings and are becoming more selective about their external partnerships, potentially to avoid direct competition with their own AI-powered products and services. This move, while potentially limiting for developers, creates a significant market void that Keenable aims to fill.
Styskin’s insights are deeply rooted in his prior work at Amazon, where he collaborated with Petri on web search infrastructure for AI applications, including the voice assistant Alexa. This experience provided firsthand exposure to the evolving demands of AI systems for vast and accessible datasets. Furthermore, observations from sources like Cloudflare’s data, which indicated a growing proportion of search traffic originating from AI crawlers, solidified Styskin’s conviction that a fundamental shift in web search infrastructure was not only imminent but necessary. The development of web search infrastructure designed with AI at its core became a clear strategic imperative.
Building on this foundational understanding, Styskin has strategically assembled a team for Keenable, drawing upon his extensive network to recruit former colleagues with deep expertise in search and AI. The company is not only focused on indexing but also on developing proprietary retrieval capabilities. A key upcoming product, the "Web Query Language," is designed to empower AI systems to synthesize information from disparate web sources, even when no single source contains a complete answer. This capability is crucial for complex question-answering and generative AI tasks.
The economic viability of such an endeavor is a significant consideration. Styskin acknowledges the immense challenge of persuading users to deviate from established search giants like Google. However, he invokes the principles of "The Innovator’s Dilemma," suggesting that Google’s dominant position might be vulnerable in the realm of "agentic queries"—tasks performed by AI agents. This creates an opening for agile startups like Keenable to innovate and offer more cost-effective solutions tailored to the specific needs of AI companies. The cost of building and maintaining a comprehensive search index is, as Styskin candidly admits, "painfully expensive." Nevertheless, Keenable is reportedly focused on cost management and a phased approach to development. With a current engineering team of 15 individuals spread across the U.S. and Europe, the company intends to leverage its recent funding to double its headcount by the end of the year, bolstering its go-to-market strategy and accelerating product development.
The path to becoming "the next Google for AI agents" is undoubtedly arduous and fraught with competition. Other entities, such as Brave and Exa, have also entered this nascent market. Moreover, Google itself is actively engaged in overhauling its search experience to align with the AI era. This broader industry movement, however, validates Keenable’s underlying thesis. The increasing investment and strategic pivots by major technology players suggest a shared understanding that the era of "ten blue links"—the traditional format of search results—is likely drawing to a close, paving the way for new models of information access and synthesis, whether for human users or sophisticated AI agents. The future of the internet’s information retrieval layer is being actively shaped, and Keenable aims to be at the forefront of this transformation.
The evolution of search from a human-centric endeavor to an AI-centric one represents a significant technological and commercial inflection point. For decades, search engines have operated under the assumption that users are the primary consumers of information, with interfaces and algorithms optimized for quick scanning and immediate gratification. This model, exemplified by Google’s early success, prioritized relevance and speed for human comprehension. However, the advent of advanced AI, capable of not only understanding but also synthesizing and acting upon information, necessitates a fundamental rethinking of how data is indexed, retrieved, and presented.
The core challenge lies in the sheer volume and heterogeneity of web data. While human users typically seek concise answers or specific pieces of information, AI agents often require a more comprehensive understanding of context, nuance, and interconnectedness. Traditional search indexes, optimized for keyword matching and link analysis, may not adequately serve these more complex demands. This is where companies like Keenable aim to differentiate themselves by building specialized infrastructure.
The Genesis of AI-Centric Search
The impetus for Keenable’s formation can be traced to the increasing recognition of AI’s growing footprint on the internet. As AI models become more sophisticated, their reliance on vast, accessible datasets for training and real-time inference escalates. This has led to the rise of specialized AI crawlers, which interact with the web differently than human-driven bots. These AI crawlers can consume and process information at a scale and speed that far surpasses human capabilities, thereby creating new demands on the web’s infrastructure.
Styskin’s experience at Yandex and Amazon provided him with a unique vantage point on the evolving landscape of information retrieval. During his tenure at Amazon, working on AI applications like Alexa, he witnessed firsthand the need for more robust and AI-friendly search capabilities. The subsequent observation of trends, such as the increasing percentage of web traffic attributed to AI bots, solidified his belief that a dedicated infrastructure was required. This foresight positioned him to identify a market gap that traditional search engines, designed for a pre-AI era, were ill-equipped to address.
Keenable’s Technological Approach
Keenable’s strategy revolves around building a search index specifically optimized for AI consumption. This involves several key innovations:
- Massive Indexing: The reported index of over 100 billion documents signifies an ambition to cover a significant portion of the accessible web, providing a rich dataset for AI models. The sheer scale is intended to ensure that AI agents can find the most relevant and comprehensive information, regardless of the complexity of their queries.
- Optimized Retrieval: Unlike generic search, Keenable’s API is designed for machine-to-machine interaction. This means prioritizing efficient data extraction, structured output, and low latency, which are critical for AI applications that operate in real-time or require rapid processing of large datasets.
- "Grounding" Responses: The emphasis on grounding AI responses in source documents is a critical feature. This not only enhances the accuracy and verifiability of AI-generated content but also helps in debugging and understanding how AI models arrive at their conclusions. For AI developers, this is crucial for building trust and reliability into their applications.
- Web Query Language: The development of a proprietary query language tailored for AI systems suggests a move towards more sophisticated ways of querying information. This language could enable AI agents to express complex information needs and combine data from multiple sources in novel ways, moving beyond simple keyword searches.
Market Dynamics and Competitive Landscape
The timing of Keenable’s emergence is particularly significant. The tech industry is witnessing a strategic shift among major players regarding search APIs. Google and Microsoft’s decisions to retire older APIs and focus on integrated solutions reflect a desire to control the AI-powered search narrative and potentially monetize it through new avenues. This creates an opportunity for independent infrastructure providers like Keenable to offer specialized services.
The venture capital community’s strong endorsement, as evidenced by the $26 million seed round, underscores the perceived potential of this market. Accel’s leadership in the funding round, coupled with participation from Conviction Partners, suggests a shared belief in Keenable’s vision and execution capabilities.
However, Keenable is not alone in this emerging space. Companies like Brave, with its privacy-focused search engine and API, and Exa, which leverages AI to understand and index the web, represent existing or emerging competition. Furthermore, the possibility of Google and other tech giants developing their own AI-native search solutions directly poses a significant competitive threat.
The Innovator’s Dilemma and Cost Efficiency
Styskin’s reference to "The Innovator’s Dilemma" is particularly apt. Established giants often face challenges in disrupting their own successful business models. While Google has an unparalleled advantage in its existing search infrastructure and user base, its legacy systems and business considerations might slow its adaptation to the specific needs of AI agents. This creates an opening for agile startups to innovate and offer specialized solutions that are more cost-effective or better suited to the emerging AI paradigm.
The cost of building and maintaining a web-scale search index is a formidable barrier to entry. Styskin’s candid acknowledgment of the "painfully expensive" nature of this undertaking highlights the capital-intensive demands of Keenable’s business model. However, the strategic deployment of its funding—focused on doubling its engineering team and building a go-to-market motion—indicates a clear plan to scale its operations and capture market share.
Broader Implications for the Internet
The shift towards AI-centric search has profound implications for the future of the internet. If AI agents become primary users of web content, the very nature of web design, content creation, and information architecture may need to adapt. Websites might be optimized not just for human readability but also for machine comprehension. This could lead to new standards for data structuring, semantic markup, and API accessibility.
The potential obsolescence of the "ten blue links" model, long the hallmark of web search, signals a move towards more dynamic, synthesized, and contextualized information delivery. AI agents, capable of understanding complex queries and synthesizing information from multiple sources, are poised to redefine how users interact with the web. This transition is not merely an evolution but a fundamental reshaping of the digital information landscape. Keenable’s ambitious undertaking positions it as a key player in this transformative period, aiming to provide the foundational infrastructure for the AI-driven internet of tomorrow.
