qyliss changed the topic of #spectrum to: A compartmentalized operating system | https://spectrum-os.org/ | Logs: https://logs.spectrum-os.org/spectrum/
<pie_> layers, but probably coincidental https://github.com/deliciouslytyped/nix-idrer/tree/master/layers https://github.com/deliciouslytyped/nix-rer/tree/master/layers (note, these repos dont actually work yet)
cole-h has quit [Quit: Goodbye]
tilpner has joined #spectrum
cole-h has joined #spectrum
<qyliss> Just updating crosvm, and it looks like it currently fails to build upstream…
<qyliss> Does anybody have a working crosvm checkout locally who could confirm that it's not just me?
<qyliss> hyperfekt? edef?
<hyperfekt> qyliss: not at home rn, sorry 😬
<puck> hrm, i could check mayb
<puck> oh gh
<puck> this thing is meant to run inside of repo
<qyliss> You can just clone the two dependencies into the right place
<qyliss> (I'm writing documentation for how to hack on crosvm right now since I'm setting it up on my new computer :3)
<puck> ok, here we go
<puck> /bin/sh: /bin/echo: No such file or directory
<puck> thanks libminijail
<qyliss> If you add minijail from Nixpkgs it'll use that instead
<puck> aha
<qyliss> (or just patch that out, of course)
<qyliss> I think I might have figured out what happened, though.
<qyliss> I think I just forgot to update cras
<puck> works if i add dtc into my env
<qyliss> Oh yeah you need that too.
<puck> yeah, i got a successful build
<puck> [nix-shell:~/projects/crosvm/platform/crosvm]$ ./target/debug/crosvm version
<puck> crosvm 0.1.0
<qyliss> Cool, thanks.
<qyliss> Thought it was probably my problem :P
<puck> i like the list of crosvm args tbh
<qyliss> The one in --help?
<puck> run --help, yeah
<qyliss> The upstream one?
<qyliss> I think it's horrible
<puck> yeah
<puck> like. --android-fstab, --x-display, etc
<qyliss> It tries to be a table, but half the lines are too long.
<puck> .. OH
<puck> --vfio =PATH Path to sysfs of pass through or mdev device
<puck> mdevs HERE WE GOOOO
<qyliss> Want to explain what that means?
<puck> so do you know about pci passthrough?
<qyliss> I know they exist...
<puck> so basically, they work by giving userland access to the PCI device, which it can then control over ioctl + mmap for the .. well, memory map bits
<puck> of course, with this API, you don't even really "need" to have a physical PCI device backing the device path, right?
<puck> since, well, it's all just ioctls and mmaps, which any arbitrary kernel driver can do
<puck> so: mdevs, or "mediated devices", are a sort of virtual PCI device, that is not backed by a direct PCI device, but by a kernel driver
<puck> for example, both Intel and Nvidia's virtual GPU architecture is backed by this; you basically tell the driver "please create me a virtual GPU" and you pass that path in
<qyliss> Oh, cool.
<puck> this means that i can use, say, a slightly modified crosvm to run payloads that need GPU access
<puck> this is called "GVT-G" for Intel, fwiw
<qyliss> I just wrote some documentation on how to hack on crosvm: https://spectrum-os.org/doc/developer-manual.html#_hacking_on_crosvm :)
<Irenes[m]> exciting!
<qyliss> It's not all that great a setup, but at least it's now documented
<Irenes[m]> yeah
<Irenes[m]> the main thing is that others can reproduce it
<qyliss> Yeah
<qyliss> If anybody feels like trying to follow the instructions and checking they can cargo build, that would be much appreciated :)
<Irenes[m]> no promises, but I'll give it a try
<MichaelRaskin> How much traffic needed?
<MichaelRaskin> (I guess space = traffic here)
<qyliss> About 500M?
<qyliss> Plus a bit in the nix store
<MichaelRaskin> Ah nice
<Irenes[m]> two notes about the directions so far
<Irenes[m]> 1) in the blockquote that shows the directory structure, I think chromiumos/platform/crosvm/ is supposed to be a git clone of your crosvm repo. you should make this explicit.
<cole-h> One quick note: `git clone ...third_party/minijail` for adhd is wrong :P
<Irenes[m]> 2) also in that blockquote, I think there's a typo where ... yes what cole-h said
<qyliss> Going to be afk for a bit now to eat dinner, but thanks for all the feedback
<Irenes[m]> hmm
<Irenes[m]> this assumes some background knowledge about Nix, which is fine
<qyliss> Source is at https://spectrum-os.org/git/doc if anybody feels like sending patches, else I'll fix stuff up when I get back :)
<cole-h> I think after that block-quote it would be nice to have explicit directions e.g.: `mkdir -p chromiumos/platform; cd chromiumos/platform; git clone crosvm...` etc
<Irenes[m]> I tried to `nix-shell https://spectrum-os.org/git/nixpkgs -p cargo -p pkg-config -p libcap -p minijail -p dtc` but it doesn't like the URL, so I will check out your nixpkgs locally
<cole-h> Irenes[m]: try `-I nixpkgs=https://spectrum-os.org/git/nixpkgs`
<Irenes[m]> oh! yeah
<MichaelRaskin> Hmm should minijail be aosp or platform?
<cole-h> Can't say if it works yet, but it at least doesn't immediately fail :P
<cole-h> Irenes[m]: JK that doesn't work since it isn't a tarball lol
<Irenes[m]> haha ah well
<cole-h> Is this cgit?
<Irenes[m]> I don't know offhand
<cole-h> Nice, we crashed the site
<cole-h> 504 lol
<Irenes[m]> oops lol
<Irenes[m]> same here
<Irenes[m]> well, when I have a local clone I'll make sure not to lose it
<cole-h> :D
<Irenes[m]> I'm going to wait at least an hour before I try again, so feel free to try meantime knowing it won't be at the same time as me
<cole-h> Hehehe
<cole-h> qyliss: I don't think your server likes handing out whatever the size of nixpkgs is -- 504s with curl error 22.
<hyperfekt> puck: thanks for explaining that lol. i knew about gvt-g but i had no idea how it worked
<puck> yeah, i really looked into this because of nvidia vgpu reasons
<puck> slighty OT, but i had a horrible idea of doing mdev over TB3/ethernet, so having one kernel "emulate" having a PCI device attached, that is actually virtualized on another machine entirely
<MichaelRaskin> A bit like WebUSB
<cole-h> qyliss: So I fixed some stuffs and added a quick-and-dirty "Getting Started" section. It's a starting point, because I'm sure it could be improved. Here's the diff: https://gist.github.com/cole-h/aaf784741321197b3f5f16c759255597
<cole-h> (no email patch because I still need to set up my email client)
<edef> <MichaelRaskin> hyperfekt: if you want randomness to matter, there is of course also the direct definitional conflict with ASLR etc.
<edef> MichaelRaskin: i think that is *possible* to solve
<edef> MichaelRaskin: like, you can reasonably relocate post-hoc if you understand all the data structures
<edef> MichaelRaskin: ofc that does eat the benefit of CoW to some degree
<MichaelRaskin> _I_ do not actually care about CoW, that's true
<edef> MichaelRaskin: but i suspect you could do on-demand relocation for pages, and avoid relocating pages that are never accessed
<MichaelRaskin> «Understand all the data structures» in the context of Firefox makes me shudder
<edef> ie i think there are reasonable strategies for ASLRing forked VMs
<edef> in the context of Firefox, maybe not
<edef> but if you want to really get into the woods, this is doable with symbolic execution techniques
<MichaelRaskin> I don't really believe it on _that_ size of the codebase
<MichaelRaskin> I want to spawn fresh Firefox VMs quickly, and lose as few mitigations as possible in the process.
<edef> < MichaelRaskin> I don't really believe it on _that_ size of the codebase <- elaborate? i can see the case for manual annotation techniques, but my claim is that most of the code can be mechanically annotated trivially, and the rest *could* be covered with symbolic execution-derived techniques
<MichaelRaskin> edef: there are many nice and generic techniques that do work, but have a significant blowup
<edef> define blowup
<edef> you create a "template" that you fork from, and for this template you need to know where every pointer is
<edef> but you can fork from it an endless amount of times
<edef> so i consider it a thing you *build*, and it would be a drop in the bucket relative to firefox's own build
<MichaelRaskin> edef: to create the template with symbolic execution you need quite a bit of memory per instruction.
<edef> right, accepted
<edef> the more boring technique is to just create n single-use process images
<edef> you can just batch-create those and later fire them up, and they would compress fairly well against each other
<edef> ie you're still paying the full cost of relocating but it's not in the latency-critical path anymore
<MichaelRaskin> Yeah, that I have of course considered
<MichaelRaskin> I guess by now I have also reduced my number of pages opened in Firefox (and, argh, please no, but have to, Chromium) enough that all this is less critical
<MichaelRaskin> For me personally
<qyliss> cole-h: yeah I need to upgrade that server
<qyliss> Thank you for the diff!
<qyliss> fwiw, patches are usually sent with git send-email, rather than a mail client
<qyliss> So all you need to do is give it your SMTP credentials, rather than worrying about setting up mutt or whatever :)
<Irenes[m]> I should really try to get used to that workflow
<Irenes[m]> GitHub is so streamlined... for what it is
<Irenes[m]> but, like, corporate support for fascism
<leah2> dunno, i found sending quick pr always clumsy
<Irenes[m]> there's definitely an element of what you're used to
<leah2> fork the project, update remote, commit to a branch, push, click on link
<qyliss> fwiw, until I can get a bigger server, do git clone --reference and point it to an existing Nixpkgs clone
<qyliss> Then you'll only get the objects specific to Spectrum
<Irenes[m]> ah! thanks
<qyliss> And have lower disk usage to boot :)
<MichaelRaskin> Github still doesn't have proper threaded discussion on issues, how far deeper you can go?
<Irenes[m]> I'm by no means suggesting that GitHub is the be-all end-all of code review.
<Irenes[m]> yay I have a spectrum-nixpkgs checkout now
<colemickens> I'm new to the email workflow and have only used it twice, but with the hosted-git-hub workflow... I like that I can keep my branches in sync and know that the PR is in the "latest" state.
<colemickens> I feel like if I were doing a nixpkgs PR that way it would quickly get annoying "when I did rebase last, is all-packages.nix conflicting", etc. But maybe in that workflow there's an expectation that the maintainer might do more manual work when merging in a patch. idk?
<qyliss> There is, yes.
<qyliss> One thing I really dislike about GitHub is when I have to tell somebody "add one more space here" or whatever
<qyliss> If I got a patch with that problem I'd just fix it up when applying.
<Irenes[m]> ah yeah makes sense
<colemickens> I have felt that tension. "I would never write this and will probably fix it up, but I can't ask this person to do this on this PR they sent 6 months ago.". Not that that happens
<colemickens> for nits, spacing, capitalization, small stuffs
<qyliss> tbh asking that sort of stuff of people feels disrespectful to me
<qyliss> but I do it anyway, because the alternative is pushing to their branch, which feels like an overstep even if permitted
<Irenes[m]> I had a feature that I was used to as a Googler, where if for example I didn't like how they'd worded a comment (very frequent) I could add my own suggested change which fixed the wording
<colemickens> for sure. I read a post about FOSS projects that let papercuts steer people away and I empathize greatly with it.
<Irenes[m]> useful for things like that where it's easier to just show what I mean than say it
<qyliss> Or to just do it myself locally, but then GitHub doesn't turn the PR purple, and I don't want people to think their PR got "closed".
<colemickens> I read it like 4 hours before the ZFS zstd pr was temporarily closed, heh
<Irenes[m]> but yeah the dynamics of accepting a contribution for an open source thing are very different
<colemickens> yeah, I tend to not point-keep but it stinks to see your commit in the history without the purple merge label
<Irenes[m]> qyliss: okay! "cargo build" works, following these directions
<colemickens> I wonder what the percentage drop-off rate is for people trying the git send-email workflow.
<Irenes[m]> I would imagine it is very high. How many people these days even have an SMTP credential they can give it? I don't...
<Irenes[m]> I mean I guess I do but only if I generate one from Mailgun, which I use to send promotional emails
<qyliss> Do you use gmail?
<Irenes[m]> and then it won't have my personal address
<colemickens> I can't even remember what I did. I might've made an "app specific password" on my google acct?
<Irenes[m]> funny you should ask. I'm in the process of trying to self-host email because I am doing more consulting for things where my clients share my desire to not send stuff over Google's servers.
<Irenes[m]> I use gmail right now though.
<qyliss> git send-email will include a From: in the body if it doesn't match the address you're sending from, fwiw.
<qyliss> And then use that when applying.
<Irenes[m]> yeah I have a security feature set on my Google account that intentionally makes the app-specific passwords impossible
<Irenes[m]> that means no third-party SMTP, as a side effect
<colemickens> oh! you're a VIP eh!
<Irenes[m]> I'm afraid so
<leah2> i wonder if xoauth2 would work
<Irenes[m]> I mean that's available to everyone these days but most people should not set it
<qyliss> edef: did you have something special for doing git stuff with gmail? I think I vaguely remember something like that.
<qyliss> Anyway, I feel like there might be a big preconceived idea of how git works with email, which isn't true.
<qyliss> Like, feeling you have to set up a mail client to use it.
<qyliss> I wonder if I could do anything to dispell that.
<Irenes[m]> hmm yeah
<leah2> you just need a MTA running :p
<Irenes[m]> I bet there are tutorials on the workflow out there?
<qyliss> Probably not? Because any documentation will immediately cover that.
<Irenes[m]> you could make sure to link to them, both on your project page and when people ask?
<qyliss> I do link to https://git-send-email.io/
<Irenes[m]> okay
<qyliss> Somewhere..
<qyliss> I probably couldn't be clearer there tbh
<Irenes[m]> hmm yes I see
<Irenes[m]> well
<Irenes[m]> I think the bar for being willing to contribute to an experimental OS is quite high anyawy
<Irenes[m]> *anyway
<qyliss> Yeah...
<Irenes[m]> if this costs you any contributors it will be way down the line when things are more stable
<qyliss> I'm not expecting much in the way of external contribution, at least for a while.
* Irenes[m] nod
* colemickens imagines a bot that replicates GH PR events as emails...
<qyliss> colemickens: that's GitGitGadget
<qyliss> And Git uses it.
<MichaelRaskin> As evidenced by you thinking The List is not worth putting on the Contributing page
<Irenes[m]> anyway I'm in that weird niche where I probably can't use git send-mail but I can figure out better ways
<Irenes[m]> huh. nice.
<qyliss> MichaelRaskin: what?
<leah2> Irenes[m]: see my link, could be easy
<colemickens> qyliss: very cool, will check it out
<Irenes[m]> thank you!
<Irenes[m]> I shall
<MichaelRaskin> qyliss: well, there is a list of things people can already try doing to contribute, without being blocked by you current work…
<qyliss> Oh, that list.
<qyliss> I thought you meant the mailing list
<qyliss> The List is coming
<Irenes[m]> although if I'm going to set up an email thing this weekend it's going to be an IMAP server, heh
<leah2> dovecot?
<qyliss> Despite not expecting much, I will go quite far to try to accomodate contributors, because I really would love for this to not just be me.
<qyliss> Using GitHub is too far, though :P
<Irenes[m]> I still need to do the research. Is dovecot what the kids are using these days?
<Irenes[m]> pardon my uh
<Irenes[m]> that was a weird way to say it
<leah2> it's what i'd recommend
<Irenes[m]> hmm okay
<qyliss> I would use Dovecot if I were going to host my own mail.
<qyliss> I think everybody I know who does uses it.
<leah2> we used to run cyrus ages ago
<qyliss> At some point I will host my own mail but it's pretty low on the todo list :)
<leah2> dovecot just works :p
<MichaelRaskin> Irenes: fortunately, qyliss is not keen on requiring all the crazy checks for email, so sending patches is much simpler than sending to gmail
<Irenes[m]> yeah there just isn't a ton of incremental security value to moving from gmail to some other solution that runs on somebody else's computer
<Irenes[m]> haha
<Irenes[m]> good to know
<qyliss> Although I guess I am already running most of the components :D
<leah2> there is no security in email anyway :p
<adisbladis> Also: There is no illusion of security ;)
<Irenes[m]> I mean, I agree and I discourage people from using it and I have ideas about what should replace the ecosystem around it
<MichaelRaskin> I believed dovecot is the IMAP/POP3 part and Postfix is the SMTP part, am I completely behind on that?
<Irenes[m]> haha yes what adisbladis
<colemickens> confession: I love treating my computers ephemerally too much to do anything stateful like email.
<Irenes[m]> said
<qyliss> fwiw my lists do get spam, and none of that has made it through so far
<cole-h> qyliss: --reference is neat, wow!
<qyliss> So I'm checking enough :)
<Irenes[m]> colemickens: yeah I have reservations about it too, but ah well!
<qyliss> I think of my email like a git repo
<leah2> ah this reminds me to update spamassassin
<colemickens> actually, today's project is to redo zfs partitioning and setup auto snapshoting and sending to rsync.net, so maybe this will all change soon
<qyliss> It's a big append-only thing that is distributed among all my computers
<qyliss> As state goes, there's not much I'm more comfortable with.
<Irenes[m]> makes sense
<qyliss> cole-h: so, I've applied your URL fixes (thanks)
<Irenes[m]> I agree with what you said the other day about needing declarative ways to manage state
<MichaelRaskin> By now I believe many people are using Signal in a way that is less secure than their email use.
<MichaelRaskin> email is fine
<Irenes[m]> MichaelRaskin: ugh don't get me started
<qyliss> cole-h: I'm not sure about the getting start section, though.
<qyliss> I think I want to avoid people just copying and pasting it and not reading the prose, which I think contains important context.
<colemickens> Wait what? I've not heard, what are people doing with Signal that is insecure?
<Irenes[m]> contacting people they've never met in person and ignoring the safety numbers
<Irenes[m]> publishing their phone numbers to the entire world for this purpose
<cole-h> qyliss: Sure, no problem. It was just a quick list of steps I thought of.
<adisbladis> Phone numbers is a terrible identity
<leah2> ack
<qyliss> cole-h: I appreciate the effort you went to, though. Thank you so much for that.
<adisbladis> And for the last 5 years I've never held a phone number for more than ~3 months at a time..
<cole-h> Wasn't that much effort, if I'm honest :P
<cole-h> Hardest part was getting the right amount of `../` in those `cd`s :D
<qyliss> Well, few other people have put that much effort into Spectrum so far
<qyliss> So I appreciate it a lot :)
<cole-h> Hahaha
<adisbladis> In fact it's even longer, more like 6 years and ~15 phone numbers
<qyliss> I don't use Signal any more, but I'll admit to rarely verifying OMEMO fingerprints
<MichaelRaskin> qyliss: the layer under OMEMO is not as horrible as with Signal, though!
<qyliss> cole-h: here's your commits :D https://spectrum-os.org/git/doc/log/
<cole-h> Hehe :D
<cole-h> qyliss: Not committing to anything yet (finals aren't even over), but over the summer I might attempt to do a bit of hacking on spectrum 👀
<qyliss> :333
<cole-h> So, hopefully more commits of mine will be showing up in your logs :)
<cole-h> (and ack about git send-mail -- I just remember sending stuff with my @outlook.com email failing dkim checks or whatever for the sr.ht lists)
<cole-h> err, send-email
<qyliss> I run the mail server, so I can always go in and dig out your message if it gets stuck :P
<qyliss> I suppose it would be nice if every Git page had a link to contributing information
<qyliss> I wonder if I could add that
<qyliss> BTW, I'm loving how active this channel is recently. Thank you so much, everyone. <3
<Irenes[m]> for sure!
<Irenes[m]> thanks for all your work
<cole-h> I can at least provide snark and nitpicks if nothing else :D
<Irenes[m]> I can provide vaguely paternalistic wisdom on emotional and technical projects that resembles a politician posturing to take the credit for something they have nothing to do with. But I'll try not to. It's a bad habit and I'm trying to stop.
<Irenes[m]> just, you know, acknowledging my flaws and stuff
<qyliss> Oh! And I should make cgit generate tarballs so you can do nix-shell -I https://spectrum-os.org/git/whatever/????.tar.gz
<qyliss> I've seen other cgits do this, so there must be some setting or something I missed
<cole-h> `snapshots=tar.gz`
<qyliss> Oh, cool!
<qyliss> And then I shall send an email to get the VPS upgraded
<qyliss> I'd like to have it not be a VPS, and be a server in my apartment or something at some point
<cole-h> Ooh
<qyliss> But I can't put off having enough memory on doing that
<cole-h> Computer memory or brain memory? :P
<qyliss> hehe
<cole-h> Are all these warnings in cros upstream's fault?
<cole-h> Looks like :(
<cole-h> btw qyliss the "crosvm rustdoc" link on https://spectrum-os.org/doc/#devel is dead
<cole-h> Oh hey I found the website's source -- I can send a patch for that
<qyliss> Thank you!
<cole-h> Cool, sent :D
<cole-h> Aw man, it didn't do the thing
<cole-h> [PATCH www] didn't work :(
<qyliss> Came through to the list, though :)
<cole-h> Yay!
<cole-h> Why didn't the prefix thingy work :(
* cole-h investigates
<qyliss> --subject-prefix="[PATCH www]" should work?
<cole-h> It might, but I meant the 'git config sendemail.subjectPrefix "PATCH www"' from CONTRIBUTING
<qyliss> Oh
<cole-h> s/might/probably will/
<qyliss> Hmm
<qyliss> I wonder why that didn't work
<qyliss> More things to look into :)
<cole-h> Ah
<cole-h> `git config format.subjectprefix` it appears to be
<qyliss> Oh, of course
<qyliss> Because it's done by format-patch, not sendemail
<cole-h> lol
<cole-h> Next patch inc!
<qyliss> :DDDD
<cole-h> And sent!