The field of generative artificial intelligence (AI) is undergoing a significant transformation due to advancements in hardware and software.
One notable example of this transformation is the development of the generative AI search engine, BeaGo. BeaGo focuses on optimizing inference speed and reducing operational costs. It leverages high-bandwidth memory (HBM) in its GPU chips to enhance data caching capabilities and provide rapid response times to user queries. This technological advancement positions BeaGo as a strong competitor in the AI market, where speed and affordability are crucial.
Kai-Fu Lee, a prominent figure in the AI sector, emphasizes the need for a shift in how AI applications are developed and integrated. He advocates for a vertically integrated approach that combines hardware and software development to lower costs and enhance performance. Lee's insights highlight the current unsustainable nature of the AI ecosystem, where infrastructure providers like Nvidia benefit more than application developers. To address this, Lee suggests companies should take control of their technology stacks by developing their own generative AI components, including specialized hardware. This approach reduces reliance on external suppliers and allows for tailored solutions to specific use cases.
The economic dynamics of the AI industry reveal a significant contrast between the revenues generated by chip manufacturers and those of application developers. Chip makers collectively earn approximately $75 billion annually from AI processing, while the infrastructure and application sectors generate significantly less—$10 billion and $5 billion, respectively. This poses a challenge for the long-term viability of the AI ecosystem, as it limits the financial incentives for application developers to innovate and expand their offerings.
The current cost of inference using established services, such as OpenAI's GPT-4, remains prohibitively high at $4.40 per million tokens, translating to 57 cents per query. In contrast, BeaGo aims to reduce this cost to nearly one cent per query, making it a more attractive option for businesses seeking to leverage AI technology. This disparity in costs underscores the urgent need for advancements that can drive down operational expenses, enabling a healthier ecosystem where application developers can thrive and users can benefit from enhanced services.
Looking ahead, Kai-Fu Lee envisions a future where generative AI fundamentally alters the way consumers and enterprises interact with technology. For consumers, he predicts a shift away from traditional smartphones as voice-activated devices become the norm. This transition will pave the way for new applications and business models that prioritize seamless communication and interaction with AI agents, rendering app stores obsolete. In contrast, the enterprise adoption of generative AI is expected to progress at a slower pace due to the complexities of integrating AI into existing business structures. Lee notes that many Chief Information Officers (CIOs) may lack a comprehensive understanding of generative AI's capabilities, hindering their ability to identify areas for investment and innovation.
The generative AI landscape is poised for rapid evolution, with the potential to create new applications and business models that have yet to be imagined. Lee's assertion that the next two years will see a rewriting of applications reflects a broader trend toward innovation driven by advancements in AI technology. However, the path to realizing this potential is fraught with challenges, particularly in the enterprise sector. The entrenched nature of existing business processes and the difficulty of integrating generative AI with legacy systems may slow the pace of adoption. Nevertheless, as companies begin to recognize the transformative power of AI, the landscape is likely to shift, paving the way for a new era of technological advancement that could rival the impact of previous computing revolutions.