> solutions were directly provided in the issue report or the comments This is f... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

OldGreenYodaGPT on Feb 21, 2025 | parent | context | favorite | on: Some critical issues with the SWE-bench dataset

> solutions were directly provided in the issue report or the comments

This is fine, many of my real tickets already explain the solution. A good ticket often offers a solution or where to start looking.

softwaredoug on Feb 21, 2025 [–]

Yep that's fine for an issue, but a problem if you're trying to eval whether AIs can solve coding problems.

Consider applying for YC's Fall 2026 batch! Applications are open till July 27.
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact