Search

Groq Day

The Six Five team discusses the annual Groq Day.

Watch the clip here:

If you are interested in watching the full episode you can check it out here.

Disclaimer: The Six Five Webcast is for information and entertainment purposes only. Over the course of this webcast, we may talk about companies that are publicly traded and we may even reference that fact and their equity share price, but please do not take anything that we say as a recommendation about what you should do with your investment dollars. We are not investment advisors and we do not ask that you treat us as such.

Transcript:

Patrick Moorhead: So let’s move into our next topic and that’s GroqDay.

So you’re like, “Wait a second. Are you Groqing? What is Groq?” So, let’s talk about Groq and what it is as a company. So, first off Groq is one of the leaders, if not the leader, in machine learning, right up there with NVIDIA. It’s quite smaller than NVIDIA, but there are some use cases and some models, particularly low latency inference like NLP, that they just dominate in the industry. Now, I don’t know how they compare to what NVIDIA announced last week, but that doesn’t matter because Groq is shipping this technology today.

Well, you and I showed up for their GroqDay event, which I thought was, was cool on, on a couple fronts.

So first off, it just wasn’t about, “Hey, why I’m great as a company and here’s what we’re doing.” They actually had their competitors there. Competitor called Mythic. Now, Mythic operates at a lower power than Groq does and hence lower performance. So are they really competitors? Yeah, they’re competitors, but close enough to where it doesn’t get uncomfortable.

They also had a bunch of cool customers on there as well, which is not different, or not unique, but it was great to see how folks like Google Brain, John Hopkins, differential sciences came in and talked about what they’re doing. More thought leadership than pushing Groq, which I appreciated, but I want to focus on one thing and that was really Jonathan Ross, the founder and CEO of the company, how he kicked it off. You and I, we probably do watch two keynotes a day, we could probably watch more if we wanted to. And what is it typically? It’s typically a monologue talking about the company.

Well, Jonathan actually asked three questions. That was his keynote. And it was, “Hey, is it a winner take all market?” I appreciated this. He was like, “I was wrong. This is not going to be a winner take all market. This is going to be many players who are going to be here.” The second question he asked was, “Hey, how do you hire and retain the best talent in this competitive space, folks like Google, Amazon and Meta?” His answer was interesting in that, “Hey, all we do is machine learning. So essentially, if you want to come in and be part of something that that’s all we’re doing … ”

I was thinking like, “Okay, we don’t do CPUs. We don’t do FPGAs, we don’t do G use for gaming. Come on in, this is all we do.” I thought that was a pretty interesting point. The third question was around is Groq a GPU killer? He was very clear. He is like, “No. We’re going to be in a future world that has CPUs, GPUs, FPGAs and ASICs.” By the way, I like that because that’s the story I tell, I’ll be honest with you.

But he did say, “Hey, there are some use cases that GPUs just do better. So hey, we’re not a GPU killer, we have to operate … Not only do we operate in systems with GPUs, but there are actually use cases that we do better at.” He brought out Argon supercomputer benchmarks, showing that it was 50% faster than the A100. So, that was 2,200, A100s versus 16 Groq chips. So, very interesting day, I’ve just started my coverage on the company. Daniel, you’ve been covering them a little bit longer.

Daniel Newman: Yeah, but he did a really good job and these GroqDays are always great to tune in. It’s been a bit of a series for the company. Jonathan’s a very interesting, thoughtful leader. He actually showed up at a Six Five summit last year, and guess what? They’re going to be back this year, so you will have another chance to hear from him at our event. I like that you pointed out the specialty in focus. I think that’s one of the big things here, is when you talk about … We’ve talked about GPUs on this particular pod a few times today. We talked about it, starting with Intel. Well, this is about high performance compute. This is about accelerated computing, low latency. The company really does operate on four tenets. It’s predictability, low latency, velocity, and scalability. So, they really aren’t trying to come into the market and be all things to all people, GPU.

They’re saying, “Hey, for these specific types of workload, where latency is critical, where you want to be able to scale … ” like you mentioned, what was it, 2,200 verse 16, you want to be able to scale at a reasonable amount of chips compute required to do so. They’re working on very specific applications and here’s the thing, is the benchmarks have been incredibly impressive in these specific areas. The company’s new, it’s entering a hyper-competitive market, with some very, very deeply entrenched players that are the known entities in accelerated compute. That’s be a challenge for Groq.

Having said that, the company has seen investment from, I believe, it’s TPG, from Tiger. Major dollars have been poured into this because we all know the magnitude that AI is going to play in our future and the ability for us to take exponential data sets more effectively, get that data to insight in a shorter period of time. That’s really what Groq is doing. I believe in his presentation last year at The Six Five Summit, he said his goal is to bring the cost of compute to zero. Ambitious, right? With no compute, if you’re in the business of compute.

The idea is between architectures, frameworks, software, and what developers are capable of, we can make the compute go further for every single ML application that you would build on Groq. So, the company’s moving in some interesting directions, like the leadership, they’ve recently brought some … you talked about the talent game. They brought some interesting talent onboard. I think they brought the CEO of Xilinks, the founder onto their board of director, but they’ve had a bunch of academics, big Silicon executives with huge experience, come in and want to be part of this story.

Of course, Pat, full disclosure, you and I are both part of this story as investors in the company as well. Always want to put that out there because yes, we are very bullish and optimistic. We put our money where our mouth is, as I like to say it. But you know, Pat, as I listen to the company, I continue to be encouraged. The size of market for AI is going to be massive. I don’t even think we understand how big this market is going to be and companies like Groq have an opportunity. Jonathan said it won’t be winner take all. Couldn’t agree with him more, but they could be a very, very important player in a small set that are going to be winners in this particular space. So Pat, I will leave it there.

Patrick Moorhead: Good stuff, man. Yeah, I’m learning new stuff every day and I’m at the very beginning of my research so far, but I’m glad they’re not trying to do everything for everybody, because that’s just impossible given the diversity in workloads out there.

Author Information

Daniel is the CEO of The Futurum Group. Living his life at the intersection of people and technology, Daniel works with the world’s largest technology brands exploring Digital Transformation and how it is influencing the enterprise.

From the leading edge of AI to global technology policy, Daniel makes the connections between business, people and tech that are required for companies to benefit most from their technology investments. Daniel is a top 5 globally ranked industry analyst and his ideas are regularly cited or shared in television appearances by CNBC, Bloomberg, Wall Street Journal and hundreds of other sites around the world.

A 7x Best-Selling Author including his most recent book “Human/Machine.” Daniel is also a Forbes and MarketWatch (Dow Jones) contributor.

An MBA and Former Graduate Adjunct Faculty, Daniel is an Austin Texas transplant after 40 years in Chicago. His speaking takes him around the world each year as he shares his vision of the role technology will play in our future.

SHARE:

Latest Insights:

In this episode of Enterprising Insights, The Futurum Group Enterprise Applications Research Director Keith Kirkpatrick discusses several new generative AI-focused product announcements and enhancements focused on contact centers and service applications.
Anthony Anter and Tim Ceradsky from BMC Software join Steven Dickens to share their insights on fortifying mainframe operational resilience through a strategic CI/CD pipeline approach, emphasizing the importance of early integration and comprehensive testing strategies.
Dario Gil and Ion Stoica, from IBM Research and Anyscale & Databricks respectively, join us to share insights on why an open future for AI is critical for innovation and inclusivity. They delve into the AI Alliance's role in this vision.
The Six Five team discusses Synopsys Investor Day 2024.