>I think the best (only?) way to solve the plugin security problem would be to properly sandbox them with an explicit API and permission system.
I want to say "and especially prevent them from touching my private data (i.e. the whole point of Obsidian plugins being to read/write the documents)".
But if it can't talk to the internet, I kind of don't see the issue.
EDIT: Apparently due to how JS and Electron works, Obsidian plugins are just JS blobs that run in the global scope, and can read/write the whole filesystem (limited by user permissions) and make HTTP requests? Can someone confirm/deny this pls?
> But if it can't talk to the internet, I kind of don't see the issue.
No internet access doesn't save you.
With file system access it can delete a file.
Without sudo access it can silently add something to your user's crontab so a few days from now it runs a custom shell script that does anything with internet access. If you're not checking into this sort of thing regularly, you wouldn't know.
It can add something to your user's shell's rc so when you open a new terminal session, a bad side effect happens.
Malware scanning won't protect from these sort of things and every time a new version is available, it's another opportunity for something bad to happen.
To be fair this isn't a problem unique to Obsidian. Code editor plugins and most programming language package managers have the same problem.
Theoretically in an Electron app, you could run plugins in a separate v8 context without the node native FS libraries available. Short of OS-level sandboxing that's probably the best they could do.
Installing a plug-in and reviewing its code at that point is one thing. But if the plug-in can be updated withut you knowing, then there’s little guarantee of security.
You can automatically check for updates but it's off by default, and still requires a manual click. Also the new plugin review system automatically scans every release.
It doesn't do anything about first-party malware, but it can help a lot in gauging how dependencies are kept up-to-date and whether they contain any known CVEs, e.g. the same way that e.g. Trivy does and Artifacthub highlights.
I am curious how well this works out in practice for the ecosystem, though. In my experience blanket scans have a good chance to produce false-positives (= CVE exists but doesn't apply to the context it's used in), so the scans need some know-how to interpret correctly, which can lead to a lot of maintainer churn.
Read through the blog post. A permissions system is planned in addition to the automated scans and more controls for teams.
All are necessary because permissions alone can't solve certain malicious behaviors. Look at some scorecards on the Community site you'll quickly see why some of the warnings are not things a permissions system or sandboxing could catch.
The blog post contains details about the rollout, but it will be a phased approach because it requires changes to the plugin API.
I'm not sure that "Plugins will declare what they access" should be interpreted as a planned sandbox system. My (cynic) interpretation that it's an opt-in honor system, that would give a good overview about well-maintained plugins, but doesn't do anything to restrict undesired API access by malware.
We haven't shared anything about sandboxing yet. Yes, to start disclosures will be opt-in because we have to help thousands of developers with existing plugins migrate.
However, a permissions system alone is not enough. For example if a user allows a plugin with network connections, it would be easy for a plugin to abuse that permission. That's why scanning the code is still necessary to give users trust in the plugin.
Take a look at scorecards on the Community site, you'll see why some issues are not something a permissions system or sandboxing could catch.
Speaking as someone who has been building a business around an Obsidian plugin - I think you're on the right track.
What actually matters is that the plugin developer is pro-social, discloses the behavior, the user accepts that disclosure, and that the user isn't duped by their inability to review all of the code for every update.
Sorry, I think think my comment came off too dismissive.
I do think that self-reports on permission usage are a step in the right direction, and can also help in decentralized uncovering of unintended API access.
However I think with the recent pace of supply chain attacks, I think we'll be in for a rough couple months until a sandboxing system is added.
Hey kepano - can you please grandfather in existing plugin IDs?
Forcing a migration seems really user-unfriendly unless there's a symlink or something.
We have a "caution" score because our plugin (system3-relay) has a 3 in it (part of our business name), and we have thousands of daily active users that would need to essentially download a new plugin if we change it.
Yes. That's fixed! There will be some false positives and false negatives as we iron out kinks in the new system, but we're working feverishly in the #plugin-dev channel on Obsidian Discord to help devs. Please be patient, we're only a handful of people working on it :)
Obviously this wouldn’t be compatible with existing plugins, so I’d separate legacy plugins and new plugins, and add a lot of friction to install the legacy plugins, which will be deprecated at some point.
Security and authorization is just hard and at one point if you are designing a platform you have to ask yourself if it's worth the risk for the sake of flexibility. To plan for a perfectly safe system is a hopeless proposition.
IMO this is an outdated view. Existing developer platforms have had to rely on static heuristics and capability-based permission systems, but now AI can run at scale and surface a lot of user-unfriendly intent that wasn't possible before.
The permission system are definitely useful for hard limits - but AI review can surface way more detail (what kinds of things are actually sent over the network, etc).
In fact, a combination of the two is likely to be even more effective. As another commenter mentioned, heuristic-based analysis can generate false positives, but that's less of a problem if it's possible to analyze these in an additional AI step.
Also worth pointing out that the N isn't too terribly large: the article says that the ecosystem has about 4000 plugins and themes? With that volume, you could almost reasonably just use static analysis to flag suspicious plugins (saving tokens), have an AI do a pre-analysis and pass to a human for final decision-making.
We do not generally hold victims of crimes accountable for failing to defend themselves adequately.
If someone threatens you with a knife and gets you to hand over your wallet, your bank doesn’t get to say ‘you should have hired better security’ when the mugger uses your credit card.
The problem here is the mugger, and that’s who the state goes after. Even if the victim walked into a bad area. Even if the victim could have defended themselves.
Same with ransomware attackers. They are the problem. We might encourage potential victims to behave in ways that make it less likely for them to be targeted. But if they are targeted, we should still focus our societal disdain on the criminal not the victim.
While I’m sympathetic to this argument (it would be great if the internet were a safe place), in practice this thinking leads to governments trying to impose legislation that hurts legitimate uses but does little to protect from the long tail of harm. There’s little that can be done about North Korean state sanctioned cybercrime without a great firewall.
If the perpetrators of this hack were caught and in a developed country, they would certainly be prosecuted for their crimes and not get off light (especially if any data is actually leaked).
I think states should be able to do better than a ‘great firewall’ to defend their domestic net infrastructure from malicious foreign actors.
But I do think it should be much more states’ responsibility to make their domestic network safe for citizens and businesses and institutions to operate.
This reminds me of some ways Microsoft used to try catching/dissuading leakers. If someone could find a source for these..
The Xbox 360's dashboard used to have 'aesthetic' rings that actually encoded your serial number, so they could catch leakers
I think I remember hearing somewhere (maybe Dave's Garage) about beta builds of Windows using intricate patterns as wallpapers to trick people into thinking it was also a leak prevention measure.
That’s orders of magnitude faster, especially if we assume they’re not running their workloads on potatoes like mine. Is Ruby’s syntax really that much more complicated than C++, or is this a tooling problem?
I don't think the post necessarily means it took multiple hours to format the codebase, I think they're probably just saying they worked on it off-hours and landed it while no one was working so that it didn't run into merge conflicts.
iirc they also vendor(ed) many of their dependencies, several layers deep, which still counts for "stores" though it's rather different than "wrote" / "maintains".
That sounds even more insane to me, but I guess most of that code does not really touch financial transactions, otherwise it would be a nightmare being responsible to verify that.
Ruby code touches financial transactions. Card payments were migrated to Java when I left in 2022. Non-card payments (e.g., ACH, checks, various wallets) were still processed by Ruby.
PCI-related/vaulting code lived in its own locked-down repo. I think that was a mix of Go and Ruby.
Once you have the foundations in place for account balances and the ledger, processing a payment isn’t that daunting. Those foundations, however, took a lot to build and evolve.
> Once you have the foundations in place for account balances and the ledger, processing a payment isn’t that daunting. Those foundations, however, took a lot to build and evolve.
Pretty much. I've worked at places with PHP payment processing that worked just fine, and at a place with C++ payment processing (and no testers) and it worked just fine. I wasn't around when the systems were first built though so not sure if there were tears along the way.
I recently wrote a very esoteric Python script. 100 lines of code. No classes, no functions, but yes argparse.
I've tried out the latest open source models on the task. They go bananas. It's like Enterprise fizzbuzz (https://github.com/enterprisequalitycoding/fizzbuzzenterpris...). They love classes and imports and reinventing the wheel. A great way for me to tell trash AI slop code is it'll define a useful constant then 15 lines later do it again with a different name.
They love making code that looks impressive. "Wow look at all the classes and functions. It's so scalable. It's so dynamic. It validates every minutae against multiple schema and solves a problem I never thought about." But it was trash code. One really was 400 lines and it didn't even look like it would work. Can't even imagine what it means for 4.5M moderately good human lines to become what? 27M fluffy filler repeat lines that don't even make sense?
Yeah maybe I need to do the old "you are a veteran engineer" nonsense. I've had some success telling it to implement everything it suggests and be production ready. I hate when it takes a shortcut and says I'll have to change it. That's kinda the whole point of me not writing the code...
Unless I’m mistaken, it’s a monorepo. So it’s not 25M LoC in a single app, it’s (all?) of their server-side code and shared libraries. There’s also a variety of other languages in use.
16 years and thousands of engineers write a lot of code.
International Baccalaureate math has some stats questions that require a calculator that can do stats questions. Not really possible by hand in exam conditions!
My Casio FX-260 Solar IIs [1][2] (I recently bought 3 more of them) cost me $5 CAD a piece on clearance at Walmart. No battery, a modern solar panel that works great even in dimly lit rooms, and a modern SOC with all the standard scientific calculations, scientific notation, engineering notation, significant figures, and all the basic stats calculations too (sum, mean, pop stddev, sample stddev, permutations, combinations, factorials).
It’s my favourite calculator and the one I always reach for, despite having a bunch of more complicated 2-line calculators etc. It’s just so easy to use and very fast to do anything I’d want with a calculator. If I need graphing I’ll reach for Desmos. If I need algebra I’ll use Sage. I haven’t used Sage since my undergrad, however.
The basic $12 Casio scientific has stats like mean, standard deviation, regression... Stats is a huge field, we're talking highschool level. I think it probably covers it
Oh that’s neat! Probably should’ve checked your link. Not sure what the advantage of the Ti-84 would be for highschool math, but the UX on NumWorks calculators is completely a game changer, especially with stats and graphing questions.
Maybe everything is possible on the Casio, but it’s so much clearer on the NumWorks (especially for eg. Physics questions, where you might want to retrieve values you calculated earlier with full precision, etc). Genuinely felt like a cheat code when I was in highschool. I showed mine to my teacher and they swapped the whole’s schools standard calculators from the Ti-84 CE to the NumWorks, which is cheaper too.
I wonder if it makes sense for browser vendors to agree upon and ship various ‘standard models’ that are released into the public domain or something, and the API lets you pick between them.
The models themselves would be standardized and the weights and everything should be identical between browsers. They’d be standard and ‘web-safe’ like CSS colors or fonts. Probably would help to give them really boring/unbranded names too. These would work identically across browsers and web developers can rely on them existing on modern setups.
If you want more models, you could install them as a user or your browser could ship them or the web developers could bundle them through a CDN (and another standard for shared big files across domains would probably be needed)
It doesn't make sense at all. So as a user how do you choose which model to use? There could be 3824 models to choose from. The browser might as well set one as default, and we all know how that goes (see: search engine).
Not to mention many other UX questions the come with this, most importantly, how unusable these local models are on regular 3-year old laptops that are constrained in RAM, GPU/CPU capability and likely disk space despite what enthusiasts say here. (They have a Macbook Pro with 32+GB of RAM, reports it works great with xyz model -- fine -- but somehow thinks it works for everyone and local models are the future.)
The Chrome model requires either "16 GB of RAM or more and 4 CPU cores or more" or "Strictly more than 4 GB of VRAM", and "22 GB of free space" (it uses around 4.4GB but it doesn't want to use the remaining free space).
The model is pretty slow on my M4 Pro mac.
The API allows the browser to use a cloud service instead, but then privacy is lower. So, more privacy for the rich.
> It doesn't make sense at all. So as a user how do you choose which model to use? There could be 3824 models to choose from. The browser might as well set one as default, and we all know how that goes (see: search engine).
...what's the exact problem here? Believe it or not, most non-tech-savvy users use the search engine just fine.
With regards to search engines, Google paid billions of dollars [0] to become the default on major browsers. I guess GP's implying that something similar might happen with LLMs.
The rate of model development is an issue here. Once there are many cross-origin models, it becomes a fingerprinting vector. Also even the small models are many GBs.
I think initiatives for forge federation are trying to do too much. When running a forge for a project, I'd don't want to be dealing with spam or large amounts of data from other instances. And people should be able to report bugs and upload attachments, without having to give permission to share those with other instances.
A good system to download and migrate issues and pull requests is important, but that doesn't require federation.
I would love to see a smaller scoped federation of:
- Forks across instances, including for the purpose of PRs (Git)
- Activity feeds and notifications (Activity or ATproto)
- Authentication and some user settings (OAuth)
Because we are headed into a world where attacks on project hosting are more common, and loss of issues/PRs can halt a project while setting up an alternative and attempting to restore archived information.
The attacks span from forged DMCA takedowns, to national blocking orders, to suspicion that a contributor is from a sanctioned country (whether they still live there or not), to rogue project admins, and some other more creative attacks.
Project infrastructure should be distributed, with copies of data in as many computers as possible, across as many jurisdictions as possible.
It's easier and enables more features to have 1 common platform.
For example, the social features of GitHub, which I like (like stars, browsing repositories by tags etc..)
But also For PRs, the way to make a pull request to a repo hosted at A, from your own node hosted at B.
And like other commenters said, you can do this workflow with git over email like a lot of projects to, but the main goal of the federation here to me is the user experience, the UI being able to link all of theses separate repositories, issues, PRs, etc, like everything was hosted at the same place.
I think the best (only?) way to solve the plugin security problem would be to properly sandbox them with an explicit API and permission system.
reply