<IdleBot_bf4161f7>
Oh cool, I know how to negotiate for my desired feature of explicit rejection of obligatory NixOS ties. There is a huge and networked bug surface in NixOS called systemd, so it would be nice to have an option to run Spectrum on any GNU/Linux/Nix installation on supported CPU architecture.
<lejonet>
IdleBot_bf4161f7: +1 on that
<lejonet>
Even if I understand that in the beginning, to not need to implement everything and the kitchen sink, a reliance on nixos could be needed
<adisbladis>
To add to this: It would be a shame if the spectrum effort would not be usable on top of nixos
<lejonet>
I think "all" you need to take care off, to ensure non-reliance and thus nixos being mandatory, is to ensure anything that has to do with how something setting up services don't assume a specific init/service system
<lejonet>
not being*
<lejonet>
with that said tho, I think it will be fairly easy to achive, as long as its taken account for from the beginning
<lejonet>
achieve*
<IdleBot_bf4161f7>
In a way, too much ties with NixOS would mean that use for isolating only a subset of things on a NixOS-stable is harder…
<IdleBot_bf4161f7>
A clean CLI level should be sufficient
<IdleBot_bf4161f7>
(unique UID functionality requires root, be it sudo or setuid or some other mechanism on the true SpectrumOS — but it can be provided as root-required program, for everyone to invoke how they prefer)
IdleBot_bf4161f7 has quit [Remote host closed the connection]
IdleBot_b5116448 has joined #spectrum
<lejonet>
Yeah, hence why I think thinking about the abstraction from init/service system from the beginning is probably the way to go, for more general usability
<lejonet>
something close to what nixos does when wrapping things might be a good way to do the service-level isolations without the service needing to know about them?
<IdleBot_b5116448>
It might be that once we have isolation figured out, and when we have fully proper networking support (the host should not even have an IP addres on its own, for example), we can consider what to do with services. One option is to let most services run in their own NixOS with a minimal profile
<lejonet>
Yeah, it definitively is a later problem, but doesn't hurt brainstorming some ideas about how to tackle it, even if it isn't at the top of the prio list
<IdleBot_b5116448>
Arguably, a Spectrum VM constellation could be nice to be able to run normally, and then one could opt to manage all this using some init system support
<IdleBot_b5116448>
I dunno, maybe given that I have most of the detailed design, I should just try to write it down and then defend this vision…
<lejonet>
Probably not a bad idea regardless :) That way its easier to see how everything fits together and so, makes it easier to reason around it
<IdleBot_b5116448>
Well, this might end up as a project design takeover…
<lejonet>
Well, nothing says that your design MUST be used, but if its a sane design, and the community agrees around it, why not use it then?
<lejonet>
Better to have it out to be reasoned and debatted over, than just in your head and you hoping it might become reality :)
<IdleBot_b5116448>
It is not just in my head, it is basically what I currently use with some cleanup/hardening
<ehmry>
I see a cultural problem here, I came to Nixos from Gentoo because everything GUI was rendering better when it came from Nix
<ehmry>
and thats because is Nixpkgs there is a tendency to include all the dependencies so that all the advertised functionality is present
<ehmry>
whereas Gentoo had a suckless approach of building minimal packages by default
<lejonet>
IdleBot_b5116448: well, put it in a organised matter with "architectural" documents, so it can be reasoned around :)
<ehmry>
so to move away from a Nixos-centric Nixpkgs we have to advocate for optionally reducing some of the features of packages
<ehmry>
because I think systemd is a runtime dependency of a lot of stuff
<lejonet>
Unless I'm completely mistaken, nixpkgs is tied with nix, and iirc the intention is to use nix regardless, the service modules of nixpkg tho, they assume systemd because that is what nixos uses, but those 2 are decoupled
<lejonet>
You can use nixpkgs without the service modules, i.e. using nix on non-nixos distros, OS X and what not, as a "pure" package manager
<IdleBot_b5116448>
Yes, systemd libraries are still linked
<IdleBot_b5116448>
On my current system I use Nix and Nixpkgs (and the kernel is from Nixpkgs etc), but do not run systemd; that works
<IdleBot_b5116448>
But the libraries are still there, indeed
<IdleBot_b5116448>
I think there is now some negotiation about option choice in Nixpkgs; but mostly centered around size
<ehmry>
yea, I'm actually waiting for Nixpkgs to bootstrap right now because I'm trying to remove perl as a runtime dependency of openssl
<lejonet>
Yeah, if we completely want to remove the systemd legacy, it will be more work, but nixpkgs and nix is usuable without systemd as the init system :)
<IdleBot_b5116448>
Can confirm; a couple years without systemd running on a Nix-managed system now
<ehmry>
yes, and even the Nixos modules could be reused if the systemd support module is stubbed out
<lejonet>
My guess is that the question of how much is fiddled with nixpkg (i.e. if spectrumos will have its own fork, "just" a overlay above nixos nixpkg or something completely different) depends on how much time in total that the community can devote to spectrum :)
<IdleBot_b5116448>
Most of the modules already support exporting a runner script (which works just fine)
<IdleBot_b5116448>
I would prefer if they were functions, not modules, but oh well, I use them as if they were
<IdleBot_b5116448>
And even if the runner script is unusable (which happens), one can still extract the generated config files
<IdleBot_b5116448>
Anyway, I think «isolated Firefox for every Nix user on Linux» does not depend that much on the underlying init system
<lejonet>
Indeed
<IdleBot_b5116448>
And the user can configure the root-needing UID-management part as they prefer
<qyliss>
IdleBot_b5116448: there will be no systemd, and no real re-use of the nixos tree. Just pkgs.
<qyliss>
(and maybe nixos/lib)
<qyliss>
But I expect that things would still be mostly usable on NixOS
<qyliss>
lejonet: for now at least, I expect to have our own nixpkgs tree, because I imagine we'll need to do a lot of work in it (to be upstreamed!) and it's easier to iterate that way.
<qyliss>
the good news is that the host system won't actually need many modules
<qyliss>
(which is why I think trying to reuse NixOS just isn't worth it at all)
<IdleBot_b5116448>
Honestly, I would expect that you want to overhaul Nixpkgs after you already use the prototype as your day to day installation, not before
<lejonet>
qyliss: that is the "just" a overlay option I listed, and yeah, I figured as much
<qyliss>
well no, that's not really an overlay
<IdleBot_b5116448>
What do we need overhauled?
<qyliss>
IdleBot_b5116448: not all that much
<qyliss>
pkgs is fine for our use cases
<qyliss>
and NixOS just doesn't buy us much
<lejonet>
qyliss: hence the qoutes around just, with a overlay I'm meaning that we're building ontop of nixpkgs, not the nixpkg feature overlay
<qyliss>
Oh, right
<qyliss>
then yes
<lejonet>
Overloaded words bleh :P
<qyliss>
In future I could see moving to a nixpkgs overlay, too.
<qyliss>
Would be easier for people to use with existing Nix installations.
<IdleBot_b5116448>
Yes, that is what I expect. I am not sure do we even need a fork of Nixpkgs except for some minor speedup of integration
<qyliss>
But less nice for rapid development.
<qyliss>
IdleBot_b5116448: we'll see
<lejonet>
Indeed
<qyliss>
the nixpkgs repo I have on https://spectrum-os.org/git/, btw, is mostly there for testing CGit, so don't read anything into that.
<lejonet>
The initial development could always be done in its own fork, for it to be lifted upstreams/into a nix overlay with time perhaps?
<qyliss>
Today's plan: write "Software Bill of Materials", bill NLnet
<qyliss>
lejonet: yeah, that's what I was trying to say
<lejonet>
qyliss: then I understod you correctly :)
<qyliss>
Oh and public-inbox is getting a new release soon so I'm going to send some more of my patches there
<IdleBot_b5116448>
Monday plan: SW BoM gets published, then immediately overhauled completely after a discussion
<qyliss>
Yes
<qyliss>
:D
<qyliss>
There are some fairly safe bets like "Linux"
<IdleBot_b5116448>
First version is very likely to use CrosVM, NSJail, Xpra
<lejonet>
<sarcasm> what, I thought you were basing everything on hurd?!</sarcasm>
<qyliss>
IdleBot_b5116448: gonna aim for virtio-wayland probably
<qyliss>
I'll try that first and fall back to Xpra if I don't get very far with it.
<qyliss>
But yes
<qyliss>
And Nix and Nixpkgs, obviously
<IdleBot_b5116448>
Linux is only a safe bet if you can avoid mentioning which subsystems you are going to use and which you are actually going to blacklist
<qyliss>
I will be saying "Linux".
<IdleBot_b5116448>
Argh, I would need to actually set up Wayland? That is a pity…
<qyliss>
virtio-wayland is definitely where I'd like to be
<qyliss>
If I can avoid dealing with X I will.
<lejonet>
Perfectly reasonable imo
<IdleBot_b5116448>
I am not currently sure a Wayland composer convenient for me exists…
<IdleBot_b5116448>
The least you will be dealing with X is having to have XWayland for some stuff anyway
<lejonet>
X is many things, easy to secure or easy to contain isn't two of em that I've heard at least
<qyliss>
XWayland just sorta works, IME
<qyliss>
I've never had to do any set up for it
<qyliss>
Which is far from the case with X11
<qyliss>
We may have to patch / write a Wayland compositor, at some point
<qyliss>
(Less work than it sounds, I believe)
<qyliss>
We'll need to turn off a lot of "features" that break isolation.
<lejonet>
If the API is defined enough, shouldn't be too much of a headache yeah, but then, my coding skills are far from what they were nowdays lol
<IdleBot_b5116448>
I just have a ton of extension code for my WM to create my preferred workflow…
<IdleBot_b5116448>
And Wayland unifies enough things in a single process that composer is definitely more complicated than X11 WM, unfortunately
<qyliss>
that's true
<qyliss>
But way less code overall
<qyliss>
And it's a reasonable combination to make.
<IdleBot_b5116448>
Well, it does combine some pieces of code that should always behave the same and some pieces that are the actual decisions about UX into the same thing
<IdleBot_b5116448>
It seems to provide a library that does a large part of the former part
<qyliss>
Yes
<qyliss>
Nothing wrong with libraries
<qyliss>
And it felt amazing, coming from X, when I was able to just install a package and run a program, and the whole graphical environment immediately came to life and started working
<IdleBot_b5116448>
For all drawbacks of a process boundary, it does make it clearer if trying to write a clean Rust (or whatever) binding is going to be too annoying to complete
<IdleBot_b5116448>
And also, does it mean that each composer needs to provide its own interface to XRandR-like magic?
<qyliss>
Sort of, although that mostly just seems to work by magic
<qyliss>
fwiw, I think, e.g. a Rust binding to wlroots is totally feasible -- the person who tried it just kept pushing too long with the wrong abstraction.
<qyliss>
If that's what you're referring to.
<IdleBot_b5116448>
In the world I actually happen to inhabit, it is _extremely_ useful to be able to play with modes manually and quickly
<qyliss>
I'm sure that's possible. I've just never had to.
<adisbladis>
qyliss: That would suck in some ways though... You would not have eglstreams so no nvidia driver
<adisbladis>
And no hwcomposer (so no aarch64 android hardware)
<adisbladis>
You'd only be able to run on hardware which has mesa support
<adisbladis>
My opinion: Just use kwin
<adisbladis>
It's an excellent and scriptable compositor
<IdleBot_b5116448>
Just yesterday I had a projector with crazy wrong aspect ratio in 1024x768, it worked fine in 800x600 for 45 minutes then decided that the picture shaking is a cool idea, then it agreed to work some more in 640x480…
<qyliss>
adisbladis: maybe
<qyliss>
it depends exactly how configurable and scriptable it is
<qyliss>
But also, while I'm going for broad hardware support, I do have a special hatred for nvidia.
<adisbladis>
qyliss: I'm way more interested in hwcomposer
<qyliss>
I've never heard of that
<adisbladis>
qyliss: That's the android hardware composer
<adisbladis>
qyliss: Which you'd need on any hardware not having mainline gpu drivers (mali etc)
<adisbladis>
But the compositor situation is similar to nvidia in that the compositor needs hwcomposer support
<qyliss>
Kwin apparently supports hwcomposer but not eglstreams?
<adisbladis>
qyliss: kwin does both
<qyliss>
Ah, this was as of 2016
<adisbladis>
It was merged a few months ago
<adisbladis>
I believe kwin is the only compositor in wide use that has hwcomposer support
<adisbladis>
There were some experimental patches to add it to wlroots
<qyliss>
Would appreciate if people could have a look over and see if I've missed anything important
<qyliss>
(I'm sure there are things, but it's a lot to keep in my head all at once)
<lejonet>
What granularity should the software BoM be at?
<lejonet>
Like, compiler granularity or overview granularity?
<qyliss>
It should be any external software that's critical to the success of the project. It's mostly, as I understand, for the people doing the security review to get an overview.
<qyliss>
So, coreutils and gcc don't need to be in there.
<lejonet>
I don't see any jails/jails-like software listed, have there been one decided or is it enough to write "jails-software"?
<qyliss>
I haven't even really looked into what options there are, so let's write "undecided jail software e.g. NSjail", since that's the one IdleBot_b5116448 mentioned.
<lejonet>
Yeah, sounds good
<lejonet>
More than that I can't see anything missing from the top of my head atm
<IdleBot_b5116448>
Is it me, or is list not UTF-8-clean?
<qyliss>
err
<qyliss>
it should be
<qyliss>
IdleBot_b5116448: you mean the bom.txt file itself?
<IdleBot_b5116448>
So… sound passthrough goes via Wayland?
<IdleBot_b5116448>
Well, in all of your replies to me it looks like — gets replaced with ???
<qyliss>
Oh, that's very strange
<qyliss>
I'm not even typing an em dash
<qyliss>
I just do ASCII --
<IdleBot_b5116448>
I am pretty sure we will want some inter-VM network management that will be more involved than what CrosVM has…
<IdleBot_b5116448>
Re: em dash — but I do type them, and these get mangled over the roundtrip
<lejonet>
openvswitch could work, but that assumes regular networking stack so we might want something more tailored towards inter-vm comms
<qyliss>
Oh yeah, I see that now.
<qyliss>
the list itself shouldn't change the body at all
<qyliss>
Let me check the version I got from you directly...
<qyliss>
I will note that your mail client is behaving a bit strangely, e.g. stripping out people's names from To: and CC:
<IdleBot_b5116448>
That part I know
<qyliss>
Huh, yes, the copy I got directly is fine
<qyliss>
So it must be mailman
<qyliss>
I'll look into that in a bit.
<qyliss>
IdleBot_b5116448: you're right about netwokring
<qyliss>
not sure what to say about that right now...
<qyliss>
I'd like to implement something similar to Qubes' NetVM model.
<IdleBot_b5116448>
The end goal is probably having 4 VMs just for network…
<qyliss>
4?
<IdleBot_b5116448>
Plus or minus. You want DHCP stuff to be run by a separate machine, maybe drivers should be separate, inter-VM communication bridges should be separate from all that, and between driver and DHCP&stuff it is a good idea to have addressless filter
<qyliss>
I think we probably don't need any fancy networking software
<qyliss>
As long as we can forward packets between VMs, I don't immediately see any reason we'd need anything beyond what Linux provides
<qyliss>
But do correct me if I'm wrong.
<lejonet>
Initially, that should be enough
<qyliss>
If a need for something more becomes apparent we can think again
<lejonet>
Yeah
<IdleBot_b5116448>
Hm. Maybe it is just that I always end up needing to convert between ports and unix domain sockets somewhere
<qyliss>
We'll see.
<IdleBot_b5116448>
What is the plan for the software that wants DBus?
<qyliss>
Don't have one yet
<qyliss>
Ah shit I forgot to make the v2 patch a reply
<qyliss>
IdleBot_b5116448: I just copied and pasted some of your message into a new email, and tested the round-trip through mailman. It was fine.
<qyliss>
So I think the issue must be with your client.
<qyliss>
Oh, I think I see the problem
<qyliss>
I'm pretty sure you can't just dump UTF-8 bytes as the body of message
<qyliss>
Don't they have to be 7-bit ASCII (encoded)?
<IdleBot_b5116448>
I guess there is a lot of things that are obviously necessary but can be handwaved away as «these are just things that the user will likely choose to run in one of the VMs»
<qyliss>
My message that came through fine was quoted-printable
<qyliss>
IdleBot_b5116448: for now, at least, yes.
<IdleBot_b5116448>
I think allowing non-ASCII-encoded mail is called «8-bit-clean server», and I frankly hoped that in 2019 all servers and clients are like that, and frankly it is rare to see exceptions
<qyliss>
IdleBot_b5116448: can you send a message to postmaster@spectrum-os.org?
<qyliss>
Then I can see if it’s mailman or postfix that’s unhappy
<qyliss>
Gonna go out for a bit. When I come back if there’s no more feedback I’ll send everything off to NLnet.
<hyperfekt>
What do we require jailing for?
<lejonet>
Everything :)
<hyperfekt>
Maybe I'm confused about the architecture. But the essential parts are crosvm for hosting VMs, applications/services in VMs, communicating with other applications/services in other VMs, and with the host compositor. What part do you want to jail?
<lejonet>
iirc jails are to add another layer outside of crosvm
<IdleBot_b5116448>
The idea is that we are not completely sure if there is a way someone can trick the VM directory sharing into disclosing contexts of some files not intended to be accessible; so the VM itself should not see too much
<lejonet>
Because VM breakouts do exist, tho iirc they are few and far between
<hyperfekt>
afaik crosvm already comes with seccomp and does isolation for all its processes.
<hyperfekt>
actually looking at it again, crosvm depends on minijail
<lejonet>
I think the general idea is to not put too much trust in a layer, but build a nice little onion :)
<hyperfekt>
You want to wrap seccomp around crosvm processes twice in the hope only either crosvm authors or alyssa makes a mistake in the policy?
<hyperfekt>
idk if double filtering is even a thing
<lejonet>
I think the jail around the crosvm process will more be to limits its access to files, so more for integrity/spread
<lejonet>
so that the crosvm process can't access more than it actually have to, in terms of files
<hyperfekt>
hm i thought crosvm covered that too already
<lejonet>
I have very little insight of what crosvm actually do, just basing it on what I've read in this channel and on the homepage
<adisbladis>
Doesn't hurt to add more isolation
<adisbladis>
A namespace is cheap
<lejonet>
Indeed
<adisbladis>
Fun attack target ^_^
<adisbladis>
You pwn the vm, you escape it and you end up in a container
<IdleBot_b5116448>
Remember that all that fun happens only after breaking out of a browser
<hyperfekt>
qyliss: i haven't actually gotten around to doing this yet, you might want to do it instead of using the patch my PR for crosvm has