<hiroshi[m]>
what do you mean with a ton of lines of code v0idify . which alternative to kvm do you suggest ?
moonloo has joined #spectrum
<moonloo>
can someone explain why virtio wayland is required? why wouldn't just sharing the wayland-0 socket from the host to the gues not work?
<v0idify>
moonloo, i believe that it would probably have security issues (for example, drawing over other windows) but if the compositor is to be trusted i believe there shouldn't be a problem
<v0idify>
hiroshi[m], KVM has a pretty big codebase I believe, Qubes uses Xen for that reason. There's also other even smaller hypervisors/microkernels/things like that. I'm honestly not sure what thing each thing does yet, but these components are security-critical
<moonloo>
im looking to jury rig something using amazon's firecracker. host w/ minimal software w/ sway. running wayland applications in firecracker vms. im also evaluating google's gvisor for this purpose but it seems to have some difficulties with unix sockets.
<moonloo>
gvisor also has an ugly OCI spec dependency.
<moonloo>
Qubes and Xen seem like either overkill or not the right decisions, if amazon AWS is using KVM (firecracker) you can be sure that KVM is either secure or any 0days are going to be spent compromising them and not you. Also, Xen just isn't getting that much attention anymore I believe.
stigo has quit [Remote host closed the connection]
TheJollyRoger has quit [Ping timeout: 268 seconds]
cole-h has quit [Ping timeout: 240 seconds]
stigo has joined #spectrum
moonloo has quit [Quit: Connection closed]
<hiroshi[m]>
v0idify: kvm has pretty big codebase and linux kernel has big codebase too. docker daemon is a nasty daemon running as root in your system, where security isn't the main objective of the project
<hiroshi[m]>
saying that kvm has a pretty big codebase doesn't help at all . modern browsers are all insecure but we need to use it if we want to search in the modern web
<hiroshi[m]>
moonloo: firecracker is design for servers, in the core it uses rustvmm (which is the core of crosvm) . qyliss explained this in https://alyssa.is/using-virtio-wl/
dfgg has quit [Quit: WeeChat 1.4]
<hyperfekt>
these days all the big cloud providers are running kvm, which means while larger it also has a lot more eyes on it than xen. the original evaluation by joanna rutkowska mostly laments qemu, which spectrumOS doesn't use
dfgg has joined #spectrum
tilpner_ has joined #spectrum
tilpner has quit [Ping timeout: 256 seconds]
tilpner_ is now known as tilpner
moonloo has joined #spectrum
tilpner has quit [Remote host closed the connection]
tilpner has joined #spectrum
<v0idify>
hiroshi[m], yes but in a system like spectrum or qubes, we _assume_ that these other codebases (the whole linux kernel, browsers) are already compromised but not with Xen (or in this case KVM)
<v0idify>
because Xen or KVM is what is providing that security
<v0idify>
moonloo, that's fair
<v0idify>
it's true that qubesOS was made on a different moment in hypervisor history(?)
tilpner_ has joined #spectrum
tilpner has quit [Ping timeout: 240 seconds]
tilpner_ is now known as tilpner
tilpner_ has joined #spectrum
tilpner has quit [Ping timeout: 260 seconds]
tilpner has joined #spectrum
tilpner_ has quit [Ping timeout: 260 seconds]
tilpner_ has joined #spectrum
tilpner has quit [Ping timeout: 264 seconds]
tilpner_ is now known as tilpner
tilpner has quit [Ping timeout: 240 seconds]
tilpner has joined #spectrum
tilpner_ has joined #spectrum
tilpner has quit [Ping timeout: 260 seconds]
tilpner_ is now known as tilpner
<moonloo>
It would appear that passing through the wayland socket to a firecracker vm would be a hassle (vsock proxy). what about waypipe (wayland network proxy)? has anyone examined that option to facilitate wayland communication between VM and host?
<hyperfekt>
wdym? that's what virtio_wl is for...
<hyperfekt>
the reason it doesn't work is that wayland works on shared memory and passes for descriptors to that shared memory over the socket. they're meaningless without sharing the memory
<hyperfekt>
*file descriptors
<moonloo>
Firecracker doesn't have virtio_wl. If shared memory is required then why does wayland work for containers that unshare IPC? waypipe (https://gitlab.freedesktop.org/mstoeckl/waypipe/) is wayland over network.
<puck>
it shares file descriptors for shared memory and GPU buffers over the wayland domain socket. waypipe serializes these over the same tunnel, so it's not as efficient as something designed to take advantage of being able to share memory, which containers can
<puck>
iirc spectrum is currently working with crosvm, which does have virtio_wl support
<hyperfekt>
who said containers don't share ipc? waypipe, just like virtio_wl, is a translation layer for wayland, that makes sure either side gets file descriptors that contain what they should
<moonloo>
when a container unshares the IPC namespace I assumed it stops shared IPC with the host or other containers but I guess that's wrong?
<puck>
IPC namespace is something else
<moonloo>
for example when you unshare IPC alsa stops working, but x11 and wayland sockets work if they are bound into the container.
<puck>
it is SysV IPC objects, Alsa uses SysV semaphores apparently
<moonloo>
so to get wayland working in firecracker either the crosvm solution has to be ported or you must use waypipe. waypipe seems easier.
<puck>
or use crosvm? :p
<moonloo>
according tot he spectrum blogposts the author opted for the wayland virtio instead of waypipe, why?
<moonloo>
firecracker is very consolidated and I really like their http API. also them being amazon - with a big fat target on their head I would rather use their solution.
<puck>
right, but crosvm is backed by google, who are running this on many many devices, and firecracker is derived from crosvm :p
<moonloo>
is setting up crosvm as simple as firecracker? With just two binaries (jailer and firecracker which is statically linked)
<puck>
yeah, crosvm is written in rust too
<moonloo>
spectrum appears to be closed source? would like to use it for reference
<hiroshi[m]>
v0idify: kvm is one alternative, xen is an other alternative but you will need to build the linux kernel with it. kvm will be running on base isolated from netowrk . you are looking for a big deal/drama with kvm where the other options aren't even close
<v0idify>
hiroshi[m], umm, what? if you use xen it doesn't matter what OS you use on top if xen itself and the software that communicates with domX from dom0 is safe
<v0idify>
assuming, of course, that you also assume that whatever is plugged into dom0 can't pwn the OS
<v0idify>
i'm not looking for drama i'm just wondering about why has KVM been chosen as similar projects have avoided it
<hiroshi[m]>
well that is the thing. you don't know what will be plug into the os
<hypokeimenon[m]>
Which similar projects?
<v0idify>
Qubes for example
<v0idify>
hiroshi[m], well, Qubes doesn't plug usb and network devices into dom0, and plugs them into a domX instead
<hypokeimenon[m]>
Qubes has been mentioned already but I was hoping you were referring to more (plural).
<v0idify>
fair enough, I can't think of any other
<v0idify>
I assume they have existed before though
<v0idify>
but the "most cloud providers use it" argument is fair enough for me
<hypokeimenon[m]>
Yeah, to be honest I am sure more exist.
<v0idify>
there's one which name I don't remember right now, by quarkslab
<v0idify>
but it's abandoned
<moonloo>
what hypervisor does amazon aws use for its standard instances? (firecracker seems to be for serverless only)
<moonloo>
is it also kvm based?
<hyperfekt>
aws uses nitro which uses kvm
<hyperfekt>
but notably not qemu, like spectrum.
<hyperfekt>
most things older than a few years will implicitly assume qemu for kvm and speak of both using the latter name
<moonloo>
seems like it would be a safe bet to use whatever the big guys are using, the odds that someone is going o be saving an 0day to use just to violate your privacy when there are millions of $s to be had is low.
<v0idify>
yup
<hyperfekt>
that argument exists on a scale. same thing can be said for any software
<hypokeimenon[m]>
Are there any obscure but interesting alternatives to KVM?
<hyperfekt>
plenty, but none you will want to use i presume
<v0idify>
the other issue then is, with the way spectrum is designed, would the usb stack/network stack/.. be isolated? as far as I understand it's not possible with KVM but I don't see how it wouldn't be possible when using a hypervisor
<hyperfekt>
if not xen, hyper-v or kvm it's hardly going to be very enjoyable for desktop workloads
<hyperfekt>
there's no good reason it can't be, just takes work.
<v0idify>
(also.. joanna is an anti-masker or something like that. I regret opening twitter)
<hyperfekt>
crosvm doesn't have the capability for other VMs to provide services but alyssa's fork does. she's specifically working on that kind of thing
<v0idify>
oh awesome!
<moonloo>
on the other side of virtualization there also exists google' gvisor which is interesting. it uses ptrace but can be made to use kvm too. i wasn't able to get wayland to work with it though, perhaps I was doing something wrong.
<v0idify>
how many hypervisors has google made smh :P
<moonloo>
gvisor is something they do use at scale i believe
<moonloo>
it emulates many syscalls and services them, there is no direct pass-through. tbh it's probably good enough even for us.
<hyperfekt>
gvisor is basically a sandbox. a good one, but still. all these solutions exist along various spectra (hehe) and everyone has to decide for themselves what tradeoffs they want to make
<v0idify>
no, there has to be one magic solution that solves every problem!
<hyperfekt>
basically you start out with virtual machines who can't communicate, and then you implement something that lets them do so to a degree you deem necessary, and hope your implementation is good
TheJollyRoger has joined #spectrum
<hypokeimenon[m]>
Have any hardening projects used Nix as a base before
<hyperfekt>
i don't know what you mean, nix is a packaging/build method
<hyperfekt>
nixos is a linux distro
<hyperfekt>
people have definitely built both virtual machines and the things that run them with nix before
<hypokeimenon[m]>
<hyperfekt "nixos is a linux distro"> this
<hyperfekt>
doesn't seem hugely relevant though since it's not part of the tcb
<hypokeimenon[m]>
I only ask because everyone seems to go with Debian
<moonloo>
nix is just package management, i usually wrap it in a container and use it to get some apps. i prefer void linux, very clean easy to reason about base system.
<hyperfekt>
kinda like qubes uses fedora as dom0 by default but you could make it use any relatively standard linux
<hypokeimenon[m]>
<moonloo "nix is just package management, "> Cool. First time I've heard Void talked about outside an obscure mention for Whonix rebasing.
<v0idify>
Void ++ and it's not related to my name :)
<v0idify>
great distro, like arch but simpler and oh so much more stable
<v0idify>
and the community isn't a shit show
<moonloo>
void is probably the closest you can get to linux from scratch without having to tear your hair out.
<hyperfekt>
rn i'm not aware of any integration between spectrum and nixos
<hyperfekt>
but, the usb stack getting exposed or not is mostly a question of where you attach it
<v0idify>
can I run crosvm inside a qube? :thonk:
<v0idify>
right, so I could build a host kernel with the usb stack removed (or just disable it on the kernel cmdline) and then attach it?
<hyperfekt>
just like you can attach the usb controller to the host or a usb qube in qubes you could attach it to the host or a vm in spectrum, if the proper inter-vm protocols are set up
<v0idify>
yes that's what I mean
<hyperfekt>
v0idify: if you have the nested virtualiziation capabilities for it i don't see why not
<v0idify>
okay I'm starting to get a better idea of everything, might try to run some spectrum components later on
<v0idify>
not sure if I can have nested virt inside a Qube, most examples are the other way around (vm(qubes) vs. qubes(qube(vm)))
<moonloo>
is it possible to just rmmod usb and modprobe it when you need it? virtualization of the usb stack seems like a hassle and the threat model is dubious.
<hyperfekt>
why would you remove the module in that case?
<moonloo>
well if you are leaving your computer unattended for a few minutes and have reason to believe someone will try to quickly stick a usb stick rather than keylog your keyboard might as well rmmod it :)
<hyperfekt>
i can imagine that spectrum will be provided as a canonical image at some point (what most people think of when they say OS) to make sure the system is secure and easy to use
<moonloo>
does crosvm have some capability of passing through opengl/vulkan or is it software rendering only? to sandbox mpv for example, or a browser with some gpu acceleration.