“The RISC-V train has left the station and it is unavoidable”

In the dynamic landscape of AI hardware development, major players such as Google, AWS, Nvidia, AMD, IBM, and Microsoft have all embarked on the journey of unveiling or planning AI chips and underlying hardware innovations. Amidst this bustling activity, Tenstorrent stands out as a company with a unique focus. Beyond its diverse initiatives, the company is actively engaged in advancing RISC-V, which it touts as a superior technology capable of outperforming ARM, ultimately aiming to bring greater independence to processor architectures.

"The AI models of the future will require heterogeneous computing, combining our robust RISC-V CPUs with our AI graph cores."

Keith Witek, Chief Operating Officer at Tenstorrent, in interaction with Pratima Harigunani, delves into the profound impact of AI on hardware, exploring topics such as reusable chiplets, the significance of RISC-V, AI-powered software, 5G, and the role of non-proprietary processors. Drawing on his extensive experience, he provides valuable insights into the future of chip technology. Excerpts:

What have been the key milestones in Tenstorrent’s journey, and what sets the company apart in the processor industry?

Tenstorrent builds computers designed for AI applications. Our core business includes the production of AI Chips, which we sell on PCIe boards, integrated into servers, and made available through our cloud. We also operate an IP Licensing business, showcasing our RISC-V and AI cores. The remarkable growth of our IP business has been a pleasant surprise. Notably, our significant RISC-V core is engineered to be part of our future AI products and also function independently, capable of operating systems like Linux and Android. This versatility positions it as a host processor for a broad spectrum of high-volume applications.

Could you provide further insights into the emphasis on RISC-V?

We believe that the AI models of the future will require heterogeneous computing, combining our robust RISC-V CPUs with our AI graph cores. Unlike many current models constrained to run on just one type of hardware engine in isolation, such as GFX, CPU, etc., optimal results across a diverse range of models arise when the right mix and proportion of compute engines are assigned to a task, tailored to the specific requirements at hand. Both RISC-V and AI are experiencing rapid growth within our company, and we are very excited to offer this innovative technology and open business model to the global tech industry.

What motivated the choice of processors communicating directly over networks, and how successful has this approach been for Tenstorrent?

The simplest and most straightforward solutions are often the best ones. Opting for Ethernet as the fabric of our communication just made more sense. In contrast to purchasing expensive networking infrastructure and proprietary switches, our Ethernet approach strikes a balance between cost, flexibility, and performance. This strategy has proven highly effective for most requirements, resonating well with our customers. The use of Ethernet not only enhances cost efficiency but also showcases our commitment to open and intelligent design decisions. This, along with other thoughtful choices, contributes to providing robust solutions for high-end compute and AI systems.

What differentiates AI and HPC workloads, and what should enterprises prepare for in this evolving landscape?

The primary difference lies in the software driving these workloads, prompting us to develop two distinct software stacks. First in line is Buda, our ML compiler that automatically takes models, compiles them, converts them into a graph, maps them to the available hardware, and executes the model with exceptional performance. Buda serves as our secret sauce for seamlessly managing AI and ML workloads, achieving automation, high performance, and scalability, all with a push of a button. Complementing this is Metal, our kernel-level bare-metal software stack. Unlike Compute Unified Device Architecture or CUDA, it offers AI, ML, and HPC engineers complete kernel access, including every register on every chip. While we provide a base set of libraries and several pre-written models in Metal, the true strength lies in its open-source nature, enabling HPC engineers to build the solutions that they want with our architecture.

How significant is the India centre and team to your portfolio and strategy?

The significance is immense. Our substantial RISC-V team in Bangalore is not only well-established but also steadily expanding. Tenstorrent President and CTO Jim Keller has committed to the IT Minister to sustain and augment our investments in India, a pledge we fully intend to fulfil. India plays a pivotal role in the technology ecosystem, particularly in software and silicon development.

Beyond this, India exhibits robust economic growth fundamentals with a well-educated and motivated population, a sizable market, a youthful demographic, and more. In the current geopolitical landscape, with East-West tensions prevailing, India is also emerging as a preferred geography for tech company investment and expansion. It is a supporter of our global AI and RISC-V open-source strategy, empowering countries and companies to own and control their critical ecosystem technology without succumbing to the risks associated with monopolistic, proprietary, or geopolitical shifts in global policy.

Do Moore’s Law, Wirth’s Law, and Evan’s Law of modularity affect the industry today? Are there any observations that stand out for you?

Moore’s Law is not dead, but it is experiencing a gradual slowdown in its ability to deliver generation-on-generation benefits to the tech industry. Industry progress faces escalating challenges rooted in the basic laws of physics and chemistry. Furthermore, the costs associated with maintaining past benefits and sustaining the pace of improvement under Moore’s Law are increasing exponentially. These dynamics are compelling the industry, out of necessity, to transition towards more cost-effective, reusable, and flexible ecosystems, exemplified by the adoption of technologies such as RISC-V, open AI, open-source software, and reusable chiplets.

Wirth’s Law remains a persistent challenge as software complexity, line of code counts, and parameters in AI models continue to escalate seemingly without any apparent limit. Software continues to be the biggest challenge in ensuring the effective and reliable performance of high-performance computing and AI solutions. While AI holds promise in improving the design and development of software for these advanced hardware platforms, AI-powered software development has not yet fully materialized to offset the growing impact of software complexity and scale.

What about Neven’s law?

Neven’s Law suggests that quantum computing holds the potential to unlock future computing capabilities at a pace and scale unprecedented in the tech industry. However, quantum computing is still in its early stages and has yet to demonstrate widespread applicability to a broad set of common problems addressed by traditional computing. Despite its current limitations, there is optimism that ongoing research and innovation will lead to hyper-exponential expansion in computing capabilities in the coming years or decades.

There is a lot of action in the processor industry, from more cores, clocks, threads, and new architectures, to power efficiency, AI, accelerated cadence, APU, and NPU. What will ultimately define the winners in the sector?

My answer might not be the one you expect, but the ultimate winner will be the company that engineers its business to meet customer needs with solid performance, flexibility, cost-effectiveness, differentiation when needed, security of investment, and reliability. While technological prowess is undoubtedly crucial, customers are becoming more discerning. They no longer seek proprietary solutions that yield substantial gross margins for companies like NVIDIA at the expense of an ideally tailored solution. Winning companies will need technology that is powerful and efficient, but they must also prioritize meeting the evolving and sophisticated demands of their customers, avoiding solutions that are inflexible or susceptible to supply chain disruptions.

Please explain this.

Winning companies are those that can engineer both their business models and technology to empower customers with customized, affordable, low-risk, and performant hardware, intellectual property (IP), and chiplets. Even among smaller clients, there’s a noticeable trend to independently develop, own, and control their differentiated technologies, System on Chips (SoCs), and systems, often seeking to license our IP for this purpose. Looking ahead, the expectation is that an increasing number of companies will opt to leverage chiplets to build their custom technology solutions. Those positioned for success will capitalize on the opportunity, and this is why Tenstorrent is offering SoCs and IP, and very soon chiplets to customers.

Software continues to be the biggest challenge in ensuring the effective and reliable performance of high-performance computing and AI solutions.

What are you particularly, excited about?

Everything! This is an exciting time in the market. RISC-V is becoming unavoidable and is poised to make a significant impact. Drawing a parallel, just as Linux is to Windows, RISC-V is to ARM and x86. The RISC-V train has left the station. AI is no longer a passing hype cycle; it is here to stay and will permeate all aspects of technology and vertical, making an impact across the global customer base. It will melt like the Internet or electricity in our daily lives in a ubiquitous way where we almost will not realize its constant presence.

Our approach with a heterogeneous graph architecture stands out as the optimal solution for modern and emerging workloads. The overwhelming interest from companies licensing our IP to secure their future, manage costs, establish differentiation, and enhance security is astounding. The chiplet approach to solutions is poised to revolutionize time-to-market, costs, and innovation, and democratize tech development, extending its reach to a diverse array of market segments and customer demographics that were previously excluded from driving tech development and leadership for their core businesses.

How will 5G, Quantum computing, and silicon alternatives affect this space? Will Gilder’s law be strong now?

The expansion of communication bandwidth and capability, along with infrastructure improvements that reduce latency, will facilitate optimized compute distribution between end and cloud. This optimization spans use cases, efficiency, cost, the emergence of new applications, and more. Compute operations can be done whenever and wherever needed, under any environment or contingency.

What kind of processors do the connected intelligent machines need? What are your views on this market?

While I touched on this earlier, it’s evident that the market is trending towards the necessity for heterogeneous compute. This transformative shift is not unique to us; it is observable with AMD and Nvidia as well. The AI Chip industry is at an existential crossroads, where companies that built their hardware for current models may face challenges. As mentioned earlier, the models of tomorrow will demand heterogeneous compute, and Tenstorrent is at the forefront, actively leading this transformative charge.

What will be the future of the supply chain of processors and chip hardware? Will the bull-whip effect hold?

Unless a major disruption surfaces, chiplets are poised to shape the future of the supply chain. Chiplets drive faster time to market, greater flexibility, lower overall costs, the inclusion of new customers in the global tech development supply chain, opportunities for increased experimentation, innovation, and differentiation, as well as effectiveness in combining previously un-combinable technologies.

Keith Witek

Chief Operating Officer, Tenstorrent

pratimah@cybermedia.co.in