I think AMDs offer was fair (full remote access to several test machines), then again just giving tinycorp the boxes on their terms with no strings attached as a kind of research grant would have earned them some goodwill with that corner of the community.
Either way both parties will continue making controversial decisions.
> Now, why don't they send me the two boxes? I understand when I was asking for firmware to be open sourced that that actually might be difficult for them, but the boxes are on eBay with a simple $$ cost. It was never about the boxes themselves, it was a test to see if software had any budget or power. And they failed super hard
I know this is someone else's reasoning, so you can't answer this question, but, doesn't this just test if they want to spend the budget on this specific thing?
If I ask a company for a $100,000 grant, and they're not willing, it doesn't seem like correct logic to assume that means they don't have the budget for it. Maybe they just don't want to spend $100,000 on me.
Why does this mean they don't have a budget or power?
He assumes the software department wants to do this, which - yes - seems to be flawed logic on his side.
Let's imagine he's indeed correct. He receives the hardware, get's hacking and solves all of AMDs problem, the stock surges and tinygrad becomes a major deep learning framework.
That would be a collosal embarrassment for AMDs software department.
Chip vendors regularly send out free hardware to software developers. In this case I don't think the cost is the issue; AMD simply doesn't want what Geohot is offering.
Considering that AMD is only really supporting their datacenter GPUs with ROCm, this is the worst possible response. It means compute on AMD GPUs is only meant for the elite of the elite and forever out of reach for the average consumer and that Nvidia is not only outcompeting AMD on quality but also on cost.
> He refused. It had to come from AMD. That's absurd and extortionist.
I'm on the wrong side of the Twitter wall to read the source, but that doesn't sound absurd. Extortionist, maybe. Hotz's major complaint (last time I checked, anyway) is pretty close to one I have - AMD appears to have between little and no strategic interest in consumer grade graphics cards having strong GPGPU support leading to random crashes from the kernel drivers and a certain attitude of "meh, whatever" from AMD corporate when dealing with that.
I doubt any specific boxes or testing regime are his complaint, he'd be much more worried about whether AMD management have any interest in companies like his succeeding. Third parties providing some support doesn't sound like it'd cut it. The process of being burned by AMD leaves one a little leery of any alleged support without some serious guarantees that more major changes are afoot in their management view.
> ...he'd be much more worried about whether AMD management have any interest in companies like his succeeding.
This reads as incredibly entitled. AMD owes him nothing, especially if he's opposed to the leadership's vision[1] and being belligerent about it.
There is maybe 1 or 2 companies with enough cachet to demand management changes at a supplier like AMD - and they have market caps in the trillions.
1. Lisa Su hasn't been shy about AMD being all about partnering with large partners who can move volume. My interpretation of this is AMD prefers dealing with Sony, Microsoft, hyperscalers, and HPC builders, then possibly tier II OEMs. Small startups are probably much further down the line, close to consumers at the tail end of AMD's attention queue. I don't like it as a consumer, but it seems like a sound strategy since the partners will shoulder most of the software effort, which is a weakness AMD has against Nvidia. They can focus on cranking out ok-to-great hardware at more-than-ok prices and build up a warchest for future investments, and who knows when this hype bubble will burst and take VC dollars with it, or someone invents an architecture that's less demanding on compute (if you're more optimistic)
Sure. But we hear a lot about Hotz because all the unentitled people rolled their eyes and went over to buy Nvidia cards. He's one of the major voices who are unreasonable enough to pipe up on Twitter and air dirty laundry.
I doubt AMD are going to listen to him. They're in a great spot and are probably going to tap into the market in a big way. But Hotz isn't crazy to test them in an odd way - although he'd probably be better off dropping AMD cards like most other people in his price range would.
> But Hotz isn't crazy to test them in an odd way..
He should have just read the Lisa Su interview from Q1 2024 where ahe laid out AMDs strategy without equivocating
> ... although he'd probably be better off dropping AMD cards
I think this is what's best for everyone. Looking at his recent track record[1], he seems like a person who's gets really excited by kicking things off and experiencing the exponentially growth phase, and then when it flattens out into a sigmoid curve, he dusts his hands and declares his work done, and moves to the next thing.
. 1. Hired by Elon to "fix" Twitter, CommaAI, and soon, Tiny
One might argue he's had a pattern for even longer. While he did do some early hypervisor glitching, even his PS3 root key release was basically just applying fail0verflow's ECDSA exploit (fail0verflow didn't release the keys specifically because they didn't want to get sued ... so that was a pretty dick move [1]).
For his projects, I think it's important to look at what he's done that's cool (eg, reversing 7900XTX [2], creating a user-space driver that completely bypasses AMD drivers for compute [3]) and separating it from his (super cringe) social media postings/self-hype.
Still, at the end of the day, here's hoping that someone at AMD realizes that having terrible consumer and workstation support will basically continue to be a huge albatross/handicap - it cuts them off basically all academic/research development (almost every single ML library and technique you can name/used in production is CUDA first because of this) and the non-hyperscaler enterprise market as well. Any dev can get a PO for a $500 Nvidia GPU (or has one on their workstation laptop already). What's the pathway for ROCm? (honestly, if I were in charge, my #1 priority would be to make sure ROCm is installed and works w/ every single APU installed, even the 2CU ones).
I don't really see why those companies would prefer AMD over Nvidia, they are not hurting for money and therefore able to spend that money on Nvidia or build their own hardware, like Google did.
Meta and Microsoft are big enough they could just build their own TPUs with a stable software stack and cut off Nvidia and AMD at the same time.
From this perspective, AMD only ever makes sense as an "also ran company" for a few niche use cases.
> This reads as incredibly entitled. AMD owes him nothing, especially if he's opposed to the leadership's vision[1] and being belligerent about it.
A generation ago, everyone in sales and developer relations understood that "the customer is always right". Remember a sweaty dude on stage jumping about screaming "developers! developers! developers"? It was exhausting dealing with all the free software and hardware sent to developers, not to mention the endless free conferences for even the most backwater developer community. But that's an ethos for boomers, I guess.
On the one hand "incredibly entitled" and on the other you talk about AMD's leadership vision. Your long closing paragraph shows that entitlement of a developer has nothing to do with anything and isn't relevant in the conversation (I can show you guys at OEMs who are incredibly arrogant and entitled or outright a$$holes but so what?). It's just an opinion based on your personal bias.
In reality, AMD simply doesn't care about small AI startups or developers as you've noted. They don't care about me wanting to run all my AI locally so that I can manage my dairy farm with a modest fleet of robots. If they cared, and they sent him MI300s immediately (or sent them to the other 8 startups that asked for them), you wouldn't be chastising him about being "incredibly entitled".
> AMD appears to have between little and no strategic interest in consumer grade graphics cards having strong GPGPU support leading to random crashes from the kernel drivers and a certain attitude of "meh, whatever" from AMD corporate when dealing with that.
AMD has little interest in software support in general.
Their Adrenalin software is riddled with bugs that have been here for years.
Having watched some of his streams on the topic, I think you've captured it well. He's basically saying he's done wasting time on AMD unless/until they get serious. It's not so much that he wants free hardware from them, rather he wants to see them put some skin in the game as they basically blew him off the last time he tried to engage with them.
We do care about software and acknowledge the gaps and will work hard to make it better. Please let me know any specific issues that are an issue for you and Im happy to push for it to get resolved or come back with why it isn't.
"I estimate having software on par with NVDA would raise their market cap by 100B. Then you estimate what the chance it that
@__tinygrad__
can close that gap, say it's 0.1%, probably a very low estimate when you see what we have done so far, but still...
That's worth 100M. And they won't even send us 2 ~100k boxes. In what world does that make sense, except in a world where decisions are made based on pride instead of ROI. Culture issue."
This is his opinion, nothing more, nothing less. He currently has a partially implemented piece of software that hasn't seen a release since November and isn't performant at all.
To be fair, having seen his software evolve, and having seen ROCm evolve, I'm more optimistic for his software in a year than yours.
He picked his problem better. The whole reason that tinygrad is, well, tiny, is that it limits the amount of overhead to onboard people and perform maintenance and rewrites. My strong impression is that the ROCm codebase is simply much too large for AMD's dev resources. You're trying to race NVidia on their turf with less resources. It's brave, but foolish.
I can see how Tinygrad could succeed. The story makes sense. AMD's doesn't, neither logically nor empirically. NVidia would have to seriously fumble.
That said I'm deeply worried about anyone whose based their company on amd gpus. The only reason why they do well in hpc is because there's an army of dreadfully underpaid and over performing grand students to pick up the slack from AMD. Trying to do that in a corporate environment is company suicide.
> That said I'm deeply worried about anyone whose based their company on amd gpus
Sony Interactive and Microsoft XBox seem to be doing great without an army of underpaid students. AMD does great at the top and bottom: the corporates in the middle that are unwilling or unable to pay people to author/tweak their software for AMD GPUs will do better going with Nvidia, which has great OOTB software, and a premium to go with it.
I suppose if AMD had infinite resources, it'd fix this post-haste.
Sony and Microsoft don't need to interact with AMD's software, they just use the hardware. Just see that Sony made its own upscaling software via ML and even created hardware with several pieces of RDNA architectures, so much so that they called it RDNA 2X (RDNA 2 (compute) + WMMA from RDNA 3 and RT from RDNA 4).
This is false. 3D VCache is enabled by TSMC's 3DFabric packaging. It also didn't really play a role in AMD passing Intel. Chiplets are also enabled by TSMC technology, CoWoS.
When AMD passed Intel, they hadn't even decided to use TSMC at all yet. Of course now Intel is behind in leveraging TSMC technology. They started late.
AMD is so behind NVidia that it's not even funny. If AMD board had any sense, they'd be carpet-bombing every researcher, AI startup, and random Joes with the latest engineering samples of unreleased top-of-the line products. And giving them a direct line to the engineering team.
This would end up costing maybe tens of millions at most, but the potential return is indeed measured in billions.
And yep, lots of people like geohot are (to put it mildly) eccentric. So deal with it. They are not merely your customers, they are your freaking sales people.
As it is, I work in a startup that does a bit of AI vision-related stuff. I'm not going to even touch AMD because I don't want to deal with divas on the AMD board in future. NVidia is more expensive right now, but they're far more predictable.
> AMD is so behind NVidia that it's not even funny.
Do you really want all AI hardware and software dominated by a monopoly? We're not looking to "beat" Nvidia, we are looking to offer a compelling alternative. MI300x is compelling. MI355x is even more compelling.
If there is another company out there making a compelling product, send them my way!
I'm willing to try AMD, and I even built an AMD-based machine to experiment with AI workflows. So far it has been failing miserably. I don't care that MI300X is compelling when I can't make samples work both on my desktop and on a cloud-based MI300X. I don't care about their academic collaborations, I'm not in the business of producing papers.
I'll just pay for H100 in the cloud to be sure that I will be able to run the resulting models on my 3090 locally and/or deploy to 4090 clusters.
If AMD shows some sense, commits to long-term support for their hardware with reasonable feature-parity across multiple generations, I'll reconsider them.
And AMD has a history of doing that! Their CPU division is _excellent_, they are renowned for having long-term support for motherboard socket types. I remember being able to buy a motherboard and then not worrying about upgrading the CPU for the next 3-4 years.
> I'm willing to try AMD, and I even built an AMD-based machine to experiment with AI workflows. So far it has been failing miserably. I don't care that MI300X is compelling when I can't make samples work both on my desktop and on a cloud-based MI300X.
Anush was actively looking for feedback on this on github today...
I have quad w7900s under my desk that work well for workloads on my desktop that translate well to MI300x. There are some perf gaps with FAv2, and FP8 but otherwise I get a seamless experience. lmk if you have a pointer to any github issues for me to track down to make your experience better.
AMD's hardware might be compelling if it had good software support, but it doesn't. CUDA regularly breaks when I try to use Tensorflow on NVIDIA hardware already. Running a poorly-implemented clone of CUDA where even getting Pytorch running is a small miracle is going to be a hard sell.
All AMD had to do was support open standards. They could have added OpenCL/SYCL/Vulkan Compute backends to Tensorflow and Pytorch and covered 80% of ML use cases. Instead of differentiating themselves with actual working software, they decided to become an inferior copy of NVIDIA.
I recently switched from Tensorflow to Tinygrad for personal projects and haven't looked back. The performance is similar to Tensorflow with JIT [0]. The difference is that instead of spending 5 hours fixing things when NVIDIA's proprietary kernel modules update or I need a new box, it actually Just Works when I do "pip install tinygrad".
> AMD's hardware might be compelling if it had good software support, but it doesn't. CUDA regularly breaks when I try to use Tensorflow on NVIDIA hardware already.
Time will tell, no? Transmeta shipped a lot of Crusoes. It was run by brilliant people. It was a “compelling alternative.” Maybe Cerebras is the Transmeta of this race, I don’t know. But. It’s not about making an alternative. It most definitely is about “beating” NVIDIA. Otherwise, you are just shoveling dollars - shareholders’, undercompensated employees at AMD and TSMC, etc. - to Meta, like everyone else.
People keep forgeting CUDA is not only about AI, graphics matter as well, as does being a polyglot ecosystem, the IDE integration, the graphical debugging tools, the libraries, having a memory model based on C++ memory model, and the last point is quite relevant, as NVidia employs a few key persons from C++ ecosystem that work on the ISO C++ standard (WG21).
You dont think AMD being competitive with Nvidia (3,37 trillion USD MC) would be "nearly 100B good"? Believe it or not the only reason thats not the case is good bug-free software. Thats what tinygrad is doing
AMD already has major ongoing projects with OpenXLA/IREE. Lots of established engineers/researchers, and it’s in collaboration with Google/AWS. Hotz is delusional if he thinks that he can do better by ripping off Karpathy’s toy autograd implementation.
> AMD already has major ongoing projects with OpenXLA/IREE.
And how's that been going? The AMD stock price compared to NVidia seems to speak volumes about the efficacy of these projects.
IREE has been around for 5 years, without producing anything overtly practical. They seem to be focused more on academic jobs and citations. It's also focused on the general case of a compiler for "all" AI-type tasks, supporting everything from WASM to CUDA.
OpenXLA seems to be a bit more practical, but I spent the last 2 hours trying to make it work on my AMD card (Radeon Pro W7900) and failing.
I personally don't like Tinygrad's approach of doing their own thing rather than integrating into PyTorch/JAX/..., but it at least is _practical_ with a reasonable end-goal. Is it going to be successful? Who knows. But it's more practical than anything AMD has done within the recent 5 years.
I am an ML scientist, my company and several others are using IREE to deploy our models to edge devices. It is the most promising technology in this area.
Those academic publications are a sign that the people involved actually know what they’re doing, and are making sure their work holds up to scrutiny.
How does Tinygrad fall short? Performance is fine [0]. It's much smaller than Pytorch and all, but that's kind of in the name.
I've been hearing about MLIR and OpenXLA for years through Tensorflow, but I've never seen an actual application using them. What out there makes use of them? I'd originally hoped it'd allow Tensorflow to support alternate backends, but that doesn't seem to be the case.
Yeah, AMD is already pouring a lot of support into OpenXLA/IREE, which has a lot of well-respected compiler engineers and researchers working on it, and companies like AWS are also investing into it.
I don’t really think TinyCorp has anything to offer AMD.
Complex how? He requested payment in the form of MI300X servers, which is unconventional, sure, but the value of the payment is not out of line with the support he proposed to provide IMO.
