The problem could also be solved by putting vlc itself in a sandbox. There is no reason a media player should have direct access to all of your files anyway. I wonder if the flatpak for vlc is set up with strong sandbox rules.
>Probably do a multi-process media player, like Chrome is doing, with parsers and demuxers in a different process, and different ones for decoders and renderers. Knowing that you probably need to IPC several Gb/s between them.
Chrome and other browsers and apps, and drivers like virtual webcams, and libraries like Syphon, can all pass "zero-copy" image buffers around between different processes by sharing buffers in GPU memory (or main memory too of course) and sending IPC messages pointing to the shared buffers.
That's how the browser's web renderer processes efficiently share the rendered images with the web browser user interface process, for example. And how virtual webcam drivers can work so efficiently, too.
>IOSurface
Share hardware-accelerated buffer data (framebuffers and textures) across multiple processes. Manage image memory more efficiently.
>Overview: The IOSurface framework provides a framebuffer object suitable for sharing across process boundaries. It is commonly used to allow applications to move complex image decompression and draw logic into a separate process to enhance security.
And Android's "SurfaceTexture" and GL_TEXTURE_EXTERNAL_OES:
>The image stream may come from either camera preview or video decode. A Surface created from a SurfaceTexture can be used as an output destination for the android.hardware.camera2, MediaCodec, MediaPlayer, and Allocation APIs. When updateTexImage() is called, the contents of the texture object specified when the SurfaceTexture was created are updated to contain the most recent image from the image stream. This may cause some frames of the stream to be skipped.
>The main benefit of external textures is their ability to render directly from BufferQueue data. SurfaceTexture instances set the consumer usage flags to GRALLOC_USAGE_HW_TEXTURE when it creates BufferQueue instances for external textures to ensure that the data in the buffer is recognizable by GLES.
And Syphon, which has a rich ecosystem of apps and tools and libraries:
>Syphon is an open source Mac OS X technology that allows applications to share frames - full frame rate video or stills - with one another in realtime. Now you can leverage the expressive power of a plethora of tools to mix, mash, edit, sample, texture-map, synthesize, and present your imagery using the best tool for each part of the job. Syphon gives you flexibility to break out of single-app solutions and mix creative applications to suit your needs.
The situation has changed a bit, though. There are practical and technological means to address most of these concerns, though it would require some work. Admittedly, I have a very Linux-focused perspective. So here we go, as the devil's advocate:
> - you need to be able to open files without user interactions (no file picker), in order to open playlist, MXF or MKV files;
> - you need raw access to /dev/* to play DVD, CD and other optical disk (and the equivalent on Windows);
> - you need ioctl on such devices, to pass the MMC for DVD/Bluray;
> - you need the same if ever you have a database of files (media center oriented);
Well, portals can also punch holes selectively to select a folder, I think. Plus, the database can sit inside the sandbox.
> - you need raw access to /dev/v4l* for your webcams and be able to control them;
Pipewire should sidestep that issue, though it
> - you need access to the GPU stack, which is running in kernel-mode, btw, to output video and get hw acceleration;
> - you need access to the audio stack, also in low-level mode;
> - you need access to the DSP acceleration (not always the GPU);
> - on linux, you have access to x11 for the 3 above features, which is almost root;
Wayland is here to stay. You just need access to DRI, which is a common use-case with flatpak/DRI. Admittedly, I know nothing about DSP interfaces. Not sure why you'd need low-level access to the audio stack, and not just pipewire or pulseaudio.
> - you need access to the system settings to disable screensavers, and adjust brightness;
> - you need to expose an IPC (think MPRIS on Linux);
All of this is handled via d-bus or portals (inhibitors, MPRIS), so fine for flatpaks.
> - you need to unzip, untar, decrypt, decipher and so on;
Usually provided as part of the Flatpak runtime.
> - you need access to mounts to be able to see the insertion of DVD/Bluray/USB/SD cards and such;
Well, desktop environments usually offer to open new devices with vlc. That could be a lost feature, but I didn't even know it was there.
> - many OpenGL client libraries need access to the /etc too;
Not sure about this one. Do they? What for? Couldn't a sandboxed /etc do?
> - you need access to the network, as input and output (think remote control);
Yes, you're right, that's one extra permission. I wish it was possible to ask for it at runtime with flatpak, on first use. I don't think even Android supports asking for this at run-time.
> - you need access to /etc/ (registry) for proxy informations, fonts configuration and accessibility;
> - you need access to the fonts and the fonts configuration (see fontconfig).
This doesn't fulfil the "without user interaction" bit, though, does it? The problem with those "let the user pick a file and the program just gets an opaque file handle"-style solutions is that they break down for any kind of file format that implicitly or explicitly consists of more than one single, atomic file that can stand for itself and doesn't require accessing data stored in other files.
E.g. playlists (letting the user pick the playlist through the OS-blessed file dialogue isn't enough, as you also need to access all media files referenced by that playlist), any kind of multi-part files (multi-part MKVs, multi-part RAR archives, whatever...), subtitles (the current expectation is that if you open a video, the video player is able to automatically find matching subtitle files stored in the same directory), locally stored HTML documents (your browser might need to load additional CSS/JS/images/... and you might also want to follow any links without having to go through a file picker each time...)
