We just launched Chip Benchmark, an open-source tool for hardware-centric benchmarking of open-weight LLMs across accelerators like NVIDIA A100/H100/L40S and AMD MI300X. It measures throughput, latency, and time-to-first-token with transparent scripts and an interactive web dashboard—making apples-to-apples comparisons easier.
We're actively welcoming contributions, new hardware support, and benchmark requests.
We've been doing lots of GPU kernel profiling and optimization on cloud infrastructure, but without local GPU hardware, that meant constant SSH juggling: upload code, compile remotely, profile kernels, download results, repeat. Or, work entirely on cloud which is expensive, slow, and annoying. We were spending more time managing infrastructure than writing the kernels we wanted to optimize.
So we built Chisel: one command to run profiling commands on any kernel. Zero local GPU hardware required.
Next up we're planning to build a web dashboard for visualizing results, simultaneous profiling across multiple GPU types, and automatic resource cleanup. But please let us know what you would like to see in this project.
We built Crackd to solve a fundamental problem: helping students understand where they stand technically, and helping them learn how to become the absolute best.
Technical students have no reliable way to know how good they are. Grades don't work because they vary by professor and university. Getting internships can boil down to arbitrary indicators, and online advice is filled with contradictions.
More importantly, they know what the best players do differently: their training methods, their practice routines, the specific skills they're mastering.
CS students are flying blind by comparison. They face a collapsing job market with nothing but twitter threads giving them contradictory advice.
We built Crackd to fix this. Students judge pairs of technical students, and choose the person they believe is more “cracked.” After enough comparisons, the Elo algorithm generates a leaderboard. Humans are terrible at absolute judgments but good at comparisons. You can't reliably say if someone is a 7/10 developer, but you can usually tell who has a better profile based on technical achievement.
What matters about Crackd is letting other students see precisely what the top students do differently. Their projects, internships, skills, school clubs, etc. This is the information that turns rankings from a leaderboard into a roadmap for ambitious students.
For the time being, Crackd is restricted to college students as we refine our platform and build our initial community.
We're attempting to cover as many dimensions of technical skill as possible, covering projects, experiences, and including a miscellaneous section so students can add whatever info they feel showcases their best work. But we'd love your feedback on what other dimensions we should include to create the most comprehensive assessment of technical capability. We would love to hear from you at contact@crackd.io
I'm an undergrad at a T10 college. Walking through our library, I often notice about 30% of students have ChatGPT or Claude open on their screens.
In my circle, I can't name a single person who doesn't heavily use these tools for assignments.
What's fascinating, though, is that the most cracked CS students I know deliberately avoid using these tools for programming work. They understand the value in the struggle of solving technical problems themselves. Another interesting effect: many of these same students admit they now have more time for programming and learning they “care about” because they've automated their humanities, social sciences, and other major requirements using LLMs. They don't care enough about those non-major courses to worry about the learning they're sacrificing.
Another obvious downside of the idiosyncratically American system that forces university students to take irrelevant classes to make up for the total lack of rigorous academic high school education.
> the most cracked CS students I know deliberately avoid using these tools for programming work. They understand the value in the struggle
I think they are in the right path here
> they've automated their humanities, social sciences, and other major requirements using LLMs.
This worries me. If they struggle with these topics but don't see the value in that struggle, that is their prerogative to decide for themselves what is important to them. But I think more technically apt people who have low verbal reasoning skills, little knowledge of history, sociology, psychology, etc, is a net positive for society. So many of the problems with the current tech industry is the tendency to think everything is just a technical problem and being oblivious to the human aspects.
Hey HN! My friend and I are building kojo, the duolingo for coding. It consists of 100% personalized llm-generated curriculum and learning path. The app will be out in the app store in the next couple of weeks. Sign up for waitlist now to get notified when it's out!
We're actively welcoming contributions, new hardware support, and benchmark requests.
Repo here: https://github.com/Herdora/chip-benchmark Dashboard: https://herdora.com/benchmark
Feedback and contributions welcome! We made it super easy to add other architectures by including the script we used for benchmarking.