If an AI agent finds zero bugs in a software utility, how can that be viewed in the sense the AI agent is not very good at finding bugs?
What if there are actually zero bugs?
> Five issues felt like nothing as we had expected an extensive list.
The expectation here may not match reality, but not necessarily because Mythos isn't as capable as claimed. curl may just happen to be a well-hardened tool that doesn't have too many security vulnerabilities in its present state.
The author considered the same w.r.t. remaining bugs:
> More to find
> These were absolutely not the last bugs to find or report. Just while I was writing the drafts for this blog post we have received more reports from security researchers about suspected problems. The AI tools will improve further and the researchers can find new and different ways to prompt the existing AIs to make them find more.
> We have not reached the end of this yet.
> I hope we can keep getting more curl scans done with Mythos and other AIs, over and over until they truly stop finding new problems.
And that makes sense, it'd be quite the argument of coincidence to say there was just 1 proper find remaining & it was only Mythos that managed to find it just at the point in time it released while the other projects have been hoovering up every other find quickly until that point. Possible, but not the safest assumption to start questioning with.
You don't need to go back to coding by hand if you know how to do it already. There is a middle ground.
If you understand good software architecture, architect it. Create a markdown document just as you would if you had a team of engineers working with you and would hand off to them. Be specific.
Let the AI do the implementation of your architecture.
I think Firefox applies more aggressive subpixel rendering and path smoothing before stroking. It resamples the glyph outline path at a higher precision level before handing it to the stroke algorithm.
I'm guessing it's not trivial to fix without breaking other things? The weakness seems to be that anyone can turn UUIDs into details like email. But I assume this functionality is necessary for other flows so they can't just turn off all UUID->email/profile look ups. And similarly hiding author UUIDs on posts also isn't trivial.
Conceptually, I agree it should be easy, but I suspect they're stuck with legacy code and behaviors that rely on the current system. Not breaking anything else while fixing this is likely the time consuming part.
But a user's email isn't always forbidden. The API endpoint which turns UUIDs into a user email presumably also has use cases where you do want to expose the user email. For example, when seeing a list of people you've already invited via email to collaborate with, or listing users within your organization, etc. So a user's email isn't always forbidden PII, it depends on the context.
The trouble is the UUID->email endpoint has no idea what the context is and that endpoint alone can't decide if it should expose email or not. And then public Notion docs publicly expose author UUIDs.
Their mistake was architecting things this way. From day 1 they should have cleanly separated public identifiers from privileged ones. Or have more bespoke endpoints for looking up a UUID's email for each of the narrow contexts in which this is allowed. They didn't do this, and they certainly should have, but fixing this mess is likely a non-trivial amount of work. Though I bet it could be done immediately if they really cared and didn't mind other things breaking.
I'm absolutely not defending their choice to expose emails in this way. They should have addressed this years ago when it was first reported, and I want them shamed for failing to care. But just trying to say it's likely not a one line fix.
You literally don’t know that. Add this to the mammoth file titled “HN comments in which the author makes some completely unsubstantiated technical claim”
It literally is easy to fix. For example they could shut down the servers. Which is what they should do immediately if there is no faster fix for a privacy leak like that.
I just randomly looked at Railway and for $20 a month you get a whole lot. I've hosted many a web project (successful personal projects and enterprise projects alike) and I don't see a large barrier to entry on "hosting a website" here.
Blocking AI scrapers and crawlers is not a huge ordeal. Planning for a unicorn before just putting a product up isn't the way to go.
> Blocking AI scrapers and crawlers is not a huge ordeal.
If you have content they want, then it is a huge ordeal. You can pay some one like CloudFlare to take care of it for you, but if you can't or won't make a deal with those types of companies, it's going to take up a significant chunk of your time.
Yes - the postinstall hook attack vector goes away. You can do SHA pinning since Git's content addressing means that SHA is the hash of the content. But then your "lockfile" equivalent is just... a list of commit SHAs scattered across import statements in your source? Managing that across a real dependency tree becomes a nightmare.
This is basically what Deno's import maps tried to solve, and what they ended up with looked a lot like a package registry again.
At least npm packages have checksums and a registry that can yank things.
You can just git submodule in the dependencies. Super easy. Also makes it straightforward to develop patches to send upstream from within your project. Or to replace a dependency with a private fork.
In my experience, this works great for libraries internal to an organization (UI components, custom file formats, API type definitions, etc.). I don't see why it wouldn't also work for managing public dependencies.
Plus it's ecosystem-agnostic. Git submodules work just as well for JS as they do for Go, sample data/binary assets, or whatever other dependencies you need to manage.
> But then your "lockfile" equivalent is just... a list of commit SHAs scattered across import statements in your source? Managing that across a real dependency tree becomes a nightmare.
The irony is that this is actually the current best practice to defend against supply chain attacks in the github actions layer. Pin all actions versions to a hash. There's an entire secondary set of dev tools for converting GHA version numbers to hashes
What if there are actually zero bugs?
> Five issues felt like nothing as we had expected an extensive list.
The expectation here may not match reality, but not necessarily because Mythos isn't as capable as claimed. curl may just happen to be a well-hardened tool that doesn't have too many security vulnerabilities in its present state.
reply