37 Matching Annotations
  1. May 2021
    1. The command nix-shell will build the dependencies of the specified derivation, but not the derivation itself. It will then start an interactive shell in which all environment variables defined by the derivation path have been set to their corresponding values, and the script $stdenv/setup has been sourced. This is useful for reproducing the environment of a derivation for development.

      QUESTION: What exactly does nix-shell execute from the Nix expression (i.e., shell.nix, default.nix, etc.)?

      ANSWER: Based on my current understanding, the answer is everything. It calls $stdenv/setup (see annotation below) to set up the most basic environment variables (TODO: expand on this), and "injects" the most common tools (e.g., gcc, sed) into it.

      It also defines the phases (TODO: verify this) and builder functions, such as genericBuilder. For example, the default builder is just two lines:

      source $stdenv/setup
      genericBuild
      

      TODO: pkgs/stdenv/generic/builder.sh is a mystery though.

      QUESTION: Once dropping into nix-shell, how do I know what phases to execute by looking at a default.nix? (E.g., [..]freeswitch/default.nix)

      ANSWER: As far as I can tell, one can override the phases in their Nix build expression (to build the derivation, see at the bottom), but they won't get executed as only the $stdenv/setup (see above) will get sourced, and no builders are called that, in return, invoke the phases (again, see above).

      So if one is using nix-shell

      • to create/hack on a package, the person has to manually invoke the builder or phases (TODO: still fuzzy on this subject)

      • to set up an environment, then one doesn't even have to worry about builders/phases because we just use nix-shell to clear the environment and to inject tools that we need for a given task

      QUESTION: When dropping into nix-shell, is this Nix expression (i.e., freeswitch/default.nix) executed? Or just parts of it?

      ANSWER: As stated above, all of the input Nix expression is evaluated, but no builders and build phases are called; although, nothing prevents one to override the phases, in case they are creating/hacking on a package.

      QUESTION:

      The command nix-shell will build the dependencies of the specified derivation, but not the derivation itself.

      What is the "derivation" here exactly? I know that it is a build expression, but does that mean the default.nix (or other Nix expression) nix-shell is invoked with?

      <sup>This statement also seems like a contradiction with how `nix-shell` works (i.e., if one issues `nix-shell -p curl`, then `curl` will be available in that sub-shell), but `-p` acts like a shortcut to as if `curl` had been listed in `buildInputs` so this is not the case.</sup>

      ANSWER: I have the feeling my confusion comes from the fact that the term "derivation" is used ambiguously in the manuals, sometimes to mean multiple things (see list below).

      TODO: Substantiate this claim, and make sure that it not coming from my misunderstanding certain topics.

      • Nix build expression (such as default.nix) whose output is going to become the store derivation itself (see last item at the bottom about the Nix manual's glossary definition)

      • store derivation.

      Had multiple cracks at unambiguously define what a derivation is, and here's a list of these:

      QUESTION: What is the difference between nix-shell -p and nix-shell invoked with a Nix expression of mkShell (or other that achieves the similar effect)?

      QUESTION: nix-shell does not create a sub-shell, so what does it do? (clarification: so nix-shell indeed does it; I confused it with nix shell)

  2. Feb 2021
    1. A Nix expression describes everything that goes into a package build action (a “derivation”)

      Come up with an ultimate definition for what a "derivation" is.

      So round up all the places where it is mentioned across Nix* manuals, and check out these:


      From Nix Pills section 6.1. The derivation function (see annotation):

      A derivation from a Nix language view point is simply a set, with some attributes. Therefore you can pass the derivation around with variables like anything else.

      So there is clearly an ambiguity between what derivations are perceived to be and what is stated in the Eelco Dolstra's PhD thesis. Or maybe I'm having issues with reading comprehension again...

    2. For each output declared in outputs, the corresponding environment variable is set to point to the intended path in the Nix store for that output. Each output path is a concatenation of the cryptographic hash of all build inputs, the name attribute and the output name. (The output name is omitted if it’s out.)

      QUESTION: So when I see $out in a builder script, it refers to the default output path because the output attribute in the Nix expression has never been explicitly set, right?

    3. A derivation causes that derivation to be built prior to the present derivation; its default output path is put in the environment variable.

      That is, if an input attribute is a reference to a derivation in the Nix store, then

      1. that derivation is built first (after a binary substitute is not found, I presume), and
      2. the path to the built package (for a better word) is handed to the shell build script.
    4. derivationA description of a build action. The result of a derivation is a store object. Derivations are typically specified in Nix expressions using the derivation primitive. These are translated into low-level store derivations (implicitly by nix-env and nix-build, or explicitly by nix-instantiate).

      Organically related to the annotation regarding my nix-shell confusion.

      The dissection of this definition to show why I find it lacking:

      A description of a build action.

      The first (couple) time(s) I read the manuals, this description popped up in many places, and I identified it with Nix expression every time, thinking that a derivation is a synonym for Nix expression.

      Maybe it is, because it clearly tries to disambiguate between store derivations and derivation in the last sentence.

      The result of a derivation is a store object.

      Is this store object the same as a store derivation?

      Derivations are typically specified in Nix expressions using the `derivation primitive. These are translated into low-level store derivations (implicitly by nix-env and nix-build, or explicitly by nix-instantiate).

      QUESTION: So, the part of the Nix build expression (such as default.nix) where the derivation primitive is called (explicitly or implicitly, as in mkDerivation) is the derivation, that will be ultimately be translated into store derivations?

      ANSWER: Start at section 15.4 Derivation.


      QUESTION: Also, why is typically used here? Can one define derivations outside of Nix expressions?

      ANSWER(?): One could I guess, because store derivations are ATerms (see annotation at the top), and the Nix expression language is just a tool to translate parameterized build actions into concrete terms to build a software package. The store derivations could be achieved using different means; e.g., the way Guix uses Guile scheme to get the same result))


      I believe, that originally, derivation was simply a synonym to store derivation. Maybe it still is, and I'm just having difficulties with reading comprehension but I think the following would be less misleading (to me and apart from re-writing the very first sentence):

      Derivations are typically the result of Nix expressions calling the derivation primitive explicitly, or implicitly usingmkDerivation`. These are translated into low-level store derivations (implicitly by nix-env and nix-build, or explicitly by nix-instantiate).

    5. $stdenv/setup

      QUESTION: Does this refer to pkgs/stdenv/generic/setup.sh? According to 6.5 Phases in the Nixpkgs manual?

      ANSWER: I'm pretty sure it does. It sets up the environment (not sure how yet; I see the env vars, but not the basic commands - sed, awk, etc. - that are listed below) and defines a bunch of functions (such as genericBuilder) but it doesn't call these functions!

    6. The function mkDerivation in the Nixpkgs standard environment is a wrapper around derivation that adds a default value for system and always uses Bash as the builder, to which the supplied builder is passed as a command-line argument. See the Nixpkgs manual for details.

      "Documented" in the Nixpkgs manual under 6.1 Using stdenv.

      Used the double-quotes above because I don't consider it well documted. Will give it a try too; worst case scenario is that I'll fail as well.

    7. C.12. Release 1.6 (2013-09-10)In addition to the usual bug fixes, this release has several new features:The command nix-build --run-env has been renamed to nix-shell.
    8. See annotations with the build-phases tag.


      Why are the build phases not enumerated in the Nix manual? If the instructions on how to create a derivation (and thus, a package) then why not go all in instead of spreading out information in different manuals, making the subject harder to grasp?...

      (By the way, it is documented in the Nixpkgs manual under 6.5 Phases; not sure why it is not called build phases when every page refers to them like that.)

    9. Chapter 14. A Simple Nix Expression

      This such a stupid move to go through a derivation example before introducing the language.

    10. Add the package to the file pkgs/top-level/all-packages.nix. The Nix expression written in the first step is a function; it requires other packages in order to build it. In this step you put it all together, i.e., you call the function with the right arguments to build the actual package.

      In addition to this rant, step 3. should be more generic, instead of tying it to Nixpkgs; at least, either show how to build your own Nix expression repo, or don't add this step, but it is not at all necessary to write a derivation. There is a Nixpkgs manual for a reason.

    11. $ nix-env -i firefox --substituters ssh://alice@avalon This works similar to the binary cache substituter that Nix usually uses, only using SSH instead of HTTP

      So a substitute is a built binary for a given derivation, and a substituter is a server (or binary cache) that serves pre-built binaries, right?

      Update: in the next line it says that "it will fall back to using the binary cache substituter", so I guess that answers it.

    12. substitute

      this is another key topic. Also:

      • substitute vs. substituter => this (I think)

      See annotations with the substitute tag

    13. When you ask Nix to install a package, it will first try to get it in pre-compiled form from a binary cache. By default, Nix will use the binary cache https://cache.nixos.org; it contains binaries for most packages in Nixpkgs. Only if no binary is available in the binary cache, Nix will build the package from source. So if nix-env -i subversion results in Nix building stuff from source, then either the package is not built for your platform by the Nixpkgs build servers, or your version of Nixpkgs is too old or too new.

      binary caches tie in with substitutes somehow; get to the bottom of it. See annotations with the substitute tag.

      Maybe this?

    14. closure

      Another gem: who knows what a "closure" is.

      [This highlight] (a couple lines below) implicitly explains it though:

      The command nix-copy-closure copies a Nix store path along with all its dependencies to or from another machine via the SSH protocol. It doesn’t copy store paths that are already present on the target machine.

      or this, also just a couple lines below:

      the closure of a store path (that is, the path and all its dependencies)

    15. the closure of a store path (that is, the path and all its dependencies)
    16. The command nix-copy-closure copies a Nix store path along with all its dependencies to or from another machine via the SSH protocol. It doesn’t copy store paths that are already present on the target machine. For example, the following command copies Firefox with all its dependencies:
    17. subscribes you to a channel that always contains that latest version of the Nix Packages collection.

      That is a misleading statement. The latest version is where the master branch points, isn't it?

      So a channel points to a Nixpkgs commit (on a branch named after the channel) where all packages inside are deemed stable, and all packages are built to have available binary substitutes by a (hydra) build farm.

    18. A Nix channel is just a URL that points to a place that contains a set of Nix expressions and a manifest.
    19. garbage collector roots

      Definitely avoid this, when a term is used but only introduced formally way later. (There is also a reference to "garbage collector roots" almost at the beginning as well.)

    20. $ nix-env --switch-profile /nix/var/nix/profiles/my-profile $ nix-env --switch-profile /nix/var/nix/profiles/default These commands switch to the my-profile and default profile, respectively. If the profile doesn’t exist, it will be created automatically.

      learn more about profiles; creating new profiles was new info

    21. Chapter 10. ProfilesProfiles and user environments are Nix’s mechanism for implementing the ability to allow different users to have different configurations, and to do atomic upgrades and rollbacks.
    22. user environment
    23. In Nix, different users can have different “views” on the set of installed applications. That is, there might be lots of applications present on the system (possibly in many different versions), but users can have a specific selection of those active — where “active” just means that it appears in a directory in the user’s PATH. Such a view on the set of installed applications is called a user environment, which is just a directory tree consisting of symlinks to the files of the active applications.
    24. nix-env -qas

      ... and it takes AGES to complete

    25. 4.3.1. Change the Nix store path prefix

      There is a lot of place in this manual (and probably in the others as well) where the prefix is referred to (usually with italics, such as "prefix/store"), so in the book

      • this should be linked to this section (or the one in the book), and

      • establish a clear and well-communicated notation to convey this

    26. At the same time, it is not possible for one user to inject a Trojan horse into a package that might be used by another user.
    27. Chapter 6. SecurityNix has two basic security models. First, it can be used in “single-user mode”, which is similar to what most other package management tools do: there is a single user (typically root) who performs all package management operations. All other users can then use the installed packages, but they cannot perform package management operations themselves.Alternatively, you can configure Nix in “multi-user mode”. In this model, all users can perform package management operations — for instance, every user can install software without requiring root privileges. Nix ensures that this is secure. For instance, it’s not possible for one user to overwrite a package used by another user with a Trojan horse.

      Would have been nice to link these to the install chapter where single- and multi-user modes were mentioned.

      How would this look in a topic-based documentation? I would think that his chapter would be listed in the pre-requisites, and it could be used to buld different reading paths (or assemblies in DocBook, I believe) such as practical, depth-first (if there are people like me who want to understand everything first), etc.

    28. reentrancy
    29. You can uninstall Nix simply by running: $ rm -rf /nix
    30. $ mkdir /nix $ chown alice /nix

      Traditionally, when a command should be invoked with sudo, it is either included in the example, or the shell indicator is # instead of $.

    31. To explicitly select a single-user installation on your system:

      It should be noted in this section also that since nix 2.1.0, single user install is the default.

    32. nix-shell '<nixpkgs>' -A pan

      What is happening here exactly?

      nix-shell's syntax synopsis always bugged because it looks like this

      SYNOPSIS
      nix-shell [--arg name value] [--argstr name value] [{--attr | -A} attrPath] [--command cmd] [--run cmd] [--exclude regexp] [--pure] [--keep name] {{--packages | -p} packages...  | [path]}
      

      and the canonical example is nix-shell '<nixpkgs>' -A pan; what tripped me up is that path is usually the first in examples, and I thought that the position of arguments are strict. As it turns out, nix-shell -A pan '<nixpkgs> is just as valid.

      Side note<br> Apparently there is no standard for man pages. See 1, 2.

      '<nixpkgs>' path is the one specified in the NIX_PATH environment variable, and -A pan looks up the pan attribute in pkgs/top-level/all-packages.nix in the Nixpkgs repo.

    33. since packages aren’t overwritten, the old versions are still there after an upgrade. This means that you can roll back to the old version:

      Wouldn't hurt to tell folks that this is a convenience layer, and one could also just use the old package from the /nix/store, even though that path would be long and obscure; one could use symlinks of course.

      Or, onc could just use nix-shell -p that specifies a specific version (that's already in the store), but, of course, it's not that simple...

      https://github.com/NixOS/nixpkgs/issues/9682

  3. Jan 2021