mastodon.gamedev.place is one of the many independent Mastodon servers you can use to participate in the fediverse.
Mastodon server focused on game development and related topics.

Server stats:

5.7K
active users

A naïve security question to #NixOS: if it's so easy to keep previous versions of everything (default behavior), does that mean that an attacker might just try out absolute paths to older/vulnerable versions of popular tools to get even more access to my local host?

E.g., via a browser vulnerability that makes local tool execution possible and with a vulnerable local tool (updated to fixed version but older version still available) he might get root permission...

As long as I don't purge older versions periodically, they're here to be used for exploits, right?

@publicvoit mostly not: the vast majority of vulnerabilities cannot be exploited by the software just sitting on a disk somewhere: it needs to be running, and somehow the attacker needs to feed their malicious input to it. Usually, the attacker gains the privileges of the account that is running the vulnerable software. So the attacker starting the 'old vulnerable' software they found in the nix store doesn't typically give them any privileges they didn't already have.

The main exception is 'setuid root' executables, like 'sudo': those can be invoked by regular users but will be running as root. A vulnerable setuid root program would be a problem. However, Nix does something smart here: files in the nix store cannot be setuid root. Instead, switching your system to a new generation will create a bunch of wrappers, like "/run/wrappers/bin/sudo", that make the *current* version of those programs executable as setuid. The previous versions may still be in the Nix store, but are 'harmless' since there is no setuid wrapper pointing to them.

Once an attacker is on your system there's lots of attack surface to elevate these privileges - famously I think having the docker daemon running and being able to access the docker socket basically makes you root (docs.docker.com/engine/securit) - but I don't think Nix is meaningfully more vulnerable in this respect compared to other distributions.

Docker Documentation · "Security""Review of the Docker Daemon attack surface"

@publicvoit @raboof Mitigations to avoid that initial foothold include:
- Having most systemd services configured via NixOS options run with DynamicUser set, enabling lots of system isolation: unix.stackexchange.com/questio
- The nature & implementation of the nix store precludes whole classes of common exploits like library/dll injection and malicious patching.
- You can comparatively easily add further isoation to potential entry points by declaring them in NixOS containers.

Unix & Linux Stack Exchangesystemd DynamicUser vs UserTo run processes without root privileges you can use DynamicUser= or a static user with User= in systemd. A good explanation for DynamicUser can be found in this blog post: http://0pointer.net/blog/

@publicvoit @raboof Personally I like to run my services in NixOS containers with private networking - often with their own network interface passed through.

@AngryAnt @raboof I can relate to that but this is only achievable with a high level of Nix knowledge.

So at that stage of Nix(OS), it is no option to me. I failed with much simpler tasks already. 😔

Emil "AngryAnt" Johansen

@publicvoit @raboof Personally I had that same impression prior to taking on containers, but ended up concluding that this had been me over-estimating complexity.

It really is one of those areas where NixOS makes doing something complicated easy vs. other distros.

@AngryAnt @raboof Frankly, I haven't even looked at this #NixOS feature.

My experience so far which doesn't necessarily contradict your comment: if something is working, its "source" looks clean and you can reproduce it a number of times. 👍

However to get to that point in the first place, I had to bump into too many issues, rely on multiple experts on the Internet to help me with my specific issue until it somehow worked and I did not even understand all the mistakes I did which is a pity with respect to learning.

My most significant example: simply invoking a local #Python script.

You could read about the very last mile of that journey here: discourse.nixos.org/t/nix-shel

Note that it took me half a year, at least six iterations of approaches including switching to poetry, various variations of nix-shells/shellscript combinations and so forth.

And this all for a task that is easy as "python3 ./myscript.py" on all(!) other operating systems.

Hardly a motivation for me for NixOS.

NixOS Discourse · Nix-shell isn't executing its Python/poetry command and doesn't exitHi, Edit: current NixOS stable here. My config: GitHub - novoid/nixos-config: NixOS + flakes + home-manager with xfce, zsh, tmux, ... (host jackson) I’m trying to write a shell script wrapper that deals with Python dependencies (using poetry) and calls a series of Python commands. However, even the first Python call isn’t executed and it doesn’t exit. You can see everything from my minimal example on: https://paste.debian.net/1343505/ I tried to strip down the issue to its core. So I even r...

@publicvoit this is a very interesting case. Are you able to share what the script / dependencies are, such that you *had* to switch to Poetry *and* use an FHS environment?

Since you use Poetry, presumably some 3rd-party dependencies are involved after all?

@justinas @publicvoit Well unless you go down the #poetry2nix rabbit hole, using poetry+fhs is a natural convergence point I would say. You want to use the language's packaging tools, not #nix's for others to be able to work on it, too. poetry is widely used and works well. uv might replace it at some point. But as soon as you use anything serious (i.e. precompiled, think numpy, scipy, sklearn, pick your poison), you're f*cked on #NixOS. So you make an fhsUserEnv.

@nobodyinperson @publicvoit

I am somewhat familiar with Python, and do use Poetry, usually with poetry2nix tacked on top.

I guess I'm really trying to figure out if the OP is solving an XY problem here. Or maybe rather I'm trying to figure out "what made you complicate things so much", as I often need to do with people reporting issues with Nix.

As you mentioned, you have to manage 3rd-party dependencies *somehow*, even on conventional Linux distros: no OS is going to magically have all the dependencies you need pre-installed.

Then, I'm very curious what prompted the switch to Poetry from whatever previous dependency management scheme OP used to use. Nix does not force Poetry upon you on any way. In fact, the old school pip+venv approach definitely works (baring FHS shenanigans).

In any case, the simplest possible answer, i.e. "just add Poetry and Python to your path" just works with OP's minimal example. gist.github.com/justinas/d63f9

The FHS shell wrapper weirdness is probably what's causing the issues here, so that's why I'm curious why FHS is required here. I haven't tried scientific stuff, but I've been happily compiling Pillow, lxml, etc. all my career on various distros - may it still be possible to opt-out of prebuilt wheels? CUDA stuff will likely still need some `LD_LIBRARY_PATH` shenanigans though.

Gistmain.pyGitHub Gist: instantly share code, notes, and snippets.

@justinas @publicvoit This example has only Python dependencies. Try the same with numpy 🙂

@nobodyinperson @justinas Yes, I can confirm. numpy is really nasty with NixOS.

@publicvoit @nobodyinperson I would be interested to hear if either of you mean anything more complicated than "you can not use the prebuilt wheels for numpy". Honestly, I'd rather have to compile stuff than put my app into an FHS env if given the choice.

While compiling numpy takes a while, thankfully it being packaged in Nixpkgs helps us easily acquire all the required dependencies, even when we don't use a Nix derivation to build it 🙂 gist.github.com/justinas/d091a

Gistpoetry.lockGitHub Gist: instantly share code, notes, and snippets.

@justinas @publicvoit So how do you not use prebuilt binaries then? poetry2nix without preferWheels=true? That'll explode right into your face due to weird infinite recursions or obscure missing overrides as soon as you do anything substantial with Python. venv and tell pip to not use wheels? For each project you will need to hand-pick the system dependencies. If you have more than 3 Python projects and need to quickly iterate with other non-nixers, an fhs env is the only viable way I think.

@nobodyinperson @justinas Funny, executing a 30-line-Python script never ever exploded into my face on all OSes outside of NixOS. 🤷

@publicvoit @justinas Not the Python script - poetry2nix 😉. Way before executing anything 🫠 (I know that doesn't make it better... 😂)

@nobodyinperson @justinas Just to give you some background:

I'm not a professional programmer. My tools are used mostly by me on one single host. They're not complicated. The effort of creating the poetry overhead is almost as complex as the scripts themselves.

I know that Nix people do have great reasons for all that boilerplate. I also believe them being true.

From my naïve perspective: why is it (python3 ./tool.py) working EVERYTHING else but not in NixOS?

I don't want (and need) to self-compile, package, embed, abstract, containerize, ...

To me, it's all way(!) too complicated layers on top of layers that really do not seem to make any sense at all.

Don't get me wrong: Nix people are awesome but since I don't plan to learn all that shit at my age (any more), it is really a hostile environment to me. My naïve reasons to go that way were proven wrong on so many levels.

It'd be different if I still was in my 20s, having too much time and energy to invest in Nix stuff.

@publicvoit @justinas Yup, sounds like you're better off with one of the standard distros, then. 👍

@publicvoit Again, I'm really interested in how you managed Python dependencies on a traditional distro. Nix does not force you into Poetry in any way.

@justinas requirements.txt + pip + venv was enough.

@publicvoit right. And what prompted the use of FHS and Poetry each in the case of NixOS?

@justinas issues with unmet C-dependencies because of numpy and Co + Nix wizards helping me with that.

@publicvoit Right. I don't think Poetry changes anything in regards to that, the magic is all in the FHS environment. You should be able to use the FHS environment with pip if you find it nicer, unless I'm missing something.

Just to reiterate, straightforward alternatives to an FHS sandbox here are:
a) Using the Nixpkgs python infrastructure, where numpy is already packaged natively, i.e. `python3.withPackages (ps: [ ps.numpy ])`. I understand this may be out of question if Nixpkgs is missing any package you want, or if you don't want to hard-depend on Nixpkgs.
b) Using Python normally (without relying on Nix's `pythonPackages`), but avoiding installing numpy from wheels that contain pre-compiled C libraries - they are what causes trouble after all.

gist.github.com/justinas/d091a demonstrates b as:
- A poetry2nix shell (latest revision)
- Just poetry, no poetry2nix (previous revision)

Again, the choice of Poetry specifically is irrelevant here - b) can be done with pip by running `pip install --no-binary :all:`. In any case, to compile you do need to have the toolchain / libraries NumPy's C code depends on, but you can piggy-back on Nixpkgs' numpy derivation here (see the `inputsFrom` thing).

GitHubBuild software better, togetherGitHub is where people build software. More than 150 million people use GitHub to discover, fork, and contribute to over 420 million projects.

@nobodyinperson I'm afraid we're talking past each other. Yeah, what I tried to point out is that the "non-Nix" approach of just using venv+pip or Poetry in isolation, without any kind of Nix packaging definitely works in minimal cases (as in OP's demonstration with just the Jinja package, as well as in legacy projects I maintain).

That is why I asked about the need for FHS in that minimal example - I'm interested in the particular case, not a broad "eh you'll need FHS someday for something".

I personally really dislike overly-generalized short quips like "Python doesn't work on NixOS at all, just use FHS", because often these push people having very simple use-cases into these complex "solutions", where their old workflow (e.g. venv+pip) would do just fine in their case. And then that overcomplication adds to the impression of "nothing works on Nix, even when you have 1000 LoC of boilerplate [which you didn't need]"

I don't doubt that there are interesting scenarios where poetry2nix explodes, if you have those on hand, I would be very interested to see them. And I would be doubly interested in cases where the "old-school" non-Nixified approaches (e.g venv+pip) to Python do not work on NixOS, and the reason isn't just "you're trying to load a shared library compiled for FHS". That one's not super interesting to me: everyone and their grandma knows random dynamically-linked code does not just run on NixOS - universally so, not a Python thing.

@justinas Last example of poetry2nix being annoying was when using Adafruit circuitpython sensor libraries. These have circular dependencies to some common package which apparently poetry et al. just ignore but poetry2nix dies horribly on, because of nix. Interpreting the kilometer-long traceback took me quite a while to figure out that was the problem, another long while how to break the cycle for each package.

And yes, all other problems are because of FHS incompat and envvars missing...

@publicvoit @raboof Dev environments will always be a challenge on NixOS and, as I'm sure you have learned, python is an especially hard one.

That said, this particular weakness/challenge of the setup does not generalise. While it is super tempting to let frustration spill over, I am sure that this has also been your experience after spending any amout of time with the system.

NixOS containers are an interplay between systemd configuration and NixOS constructs - clear strong suits.

@publicvoit @raboof Summarised, NixOS remains an "easy things can be hard, hard things can be easy" of the Linux world.

You mention running a python script, clearly falling in the first camp, in the context of talking about system containers, clear entrants of the second camp :)

@publicvoit @AngryAnt @raboof Uuh, I have been called an 'expert', what an honor 😎

@publicvoit @AngryAnt @raboof Well if you need any kind of dependency, a simple `python ./myscript.py` won't work on any distribution. Poetry or uv can ease that step of specifying and fetching dependencies, and work identically across distros. The only difference on #NixOS is that you will need to run everything from within an fhsUserEnv, e.g. execute my `fhs` command here before:

gitlab.com/nobodyinperson/nixc

This is how I do it.

GitLabmodules/fhs.nix · main · Yann Büchau / 📦 Yanns NixOS Config · GitLabMy NixOS Config

@publicvoit @AngryAnt @raboof IF however you need to pin external, non-Python depedencies as well, then what other good cross-distro options except ugly gigantic containers do you have anyway besides #nix? Then embracing the nix weirdness is worth it.

@nobodyinperson @AngryAnt @raboof On any other OS I did never have to use something more complicated than a venv. 😉

@publicvoit @AngryAnt @raboof Same for NixOS. If a venv does the job for you, use my linked fhs command as the one thing to execute before doing anything Python, then do whatever venv stuff you need to do. If you don't need external system dependencies, do you really need nix?

@nobodyinperson @AngryAnt @raboof My problem is that I thought that NixOS is a solution to some issues I had with other Linux distros. That failed and I got additional issues as well.

When I have to setup my system, NixOS (and Nix) will be a part of my past.

@publicvoit @nobodyinperson @AngryAnt @raboof
Wow that's vague.

Remember, bugs (i.e..sensible bug reports) are hugs!

@publicvoit @nobodyinperson @AngryAnt @raboof

You count complaining on social media as a bug report?