Introduction

I have been fiddling with Docker recently in an effort to building a containerised data science development environment both for work and personal use. A lot of times I found myself needing to tweak environment variables in Linux, especially with the system’s PATH variable. In particular, I need to step down from root to a non-privileged user before starting a long-running service during container start-up; since the service’s binary resides in a conda environment, I need to make sure conda is properly set up when switching to a new user. I had only some vague understanding back then that most of these environment variables are set up by the various start-up scripts sourced during user login, but I didn’t know about the details; so I took the opportunity to read through the internet about the initialisation process of Linux shells.

The Start-up Scripts

Linux sources a number of start-up scripts when a shell is opened. Some examples include:

• System-wide:
  • /etc/profile;
  • /etc/profile.d/*.sh: readable sh scripts in /etc/profile.d/ will be sourced as part of /etc/profile;
  • /etc/bash.bashrc on some distributions (e.g. Ubuntu); /etc/bashrc on some other systems;
  • /etc/environment, arguably not a start-up script per se, but its contents are used to set environment variables;
• Per-user:
  • ~/.profile;
  • ~/.bash_profile;
  • ~/.bash_login;
  • ~/.bashrc;

Although, as always, not all of them are present in all Linux distributions.

When a user starts a shell, some or all of the above scripts may be sourced; the specific scripts sourced depends on two factors:

1. Whether the shell is a login shell or a non-login shell, and;
2. Whether the shell is an interactive shell or a non-interactive shell.

So let’s look at login shells and interactive shells respectively.

Login Shell vs. Non-login Shell

A login shell, quite self-explanatorily, is a shell that is started for a user when he or she tries to login to the system. A login shell will typically authenticate the user by asking for his or her username and password.

Once a user is logged into the system, additional shells started by the user (by issuing sh, bash, or zsh via a terminal, for example) are typically non-login shells. In many cases, a user can force opening a login shell after logging in by appending a dash (-) as an option to the shell command (e.g. bash -l) or by explicitly enabling the -l/--login option when issuing the shell command.

For a GUI desktop environment, the desktop environment that logs you into the system is typically the login shell; subsequent terminal emulators usually operate as non-login shells.

Interactive Shell vs. Non-interactive Shell

See that blinking prompt at the console? A shell that awaits the user’s command in a loop is typically an interactive shell; this is the most common type of shell a user would encounter.

A non-interactive shell is usually used to execute a shell script. Commands such as bash some_script.sh or bash -c 'echo "This is sourced by a non-interactive shell"' will operate in non-interactive shells.

Some Examples Ways to Start Different Types of Shells:

• Interactive login shell:
  • bash -l: start an interactive login shell using bash as the current user;
  • bash -l -i: same as above, only explicitly indicate an interactive shell with -i;
  • su -l user: starts an interactive login shell as user;
  • su - user: same as above.
• Interactive non-login shell:
  • bash: start an interactive non-login shell using bash as the current user;
  • bash -i: same as above;
  • su user: starts an interactive non-login shell as user.
• Non-interactive login shell:
  • bash -l -c command: starts a non-interactive login shell as the current user and execute command;
  • sudo -i -u user command: starts a non-interactive login shell as user and executes command;
  • su -l -c command: same as above.
• Non-interactive non-login shell:
  • bash -c command: execute command in a non-interactive non-login shell as the current user;
  • sudo -u user command: starts a non-interactive non-login shell as user and executes command;
  • su -c command user: same as above.

For the su and sudo commands, the default shell for user as specified in /etc/passwd will be invoked. To use a different shell with the above commands, one should also supply the path of the desired shell with the -s or --shell option, e.g.: su -l -c 'echo $(whoami)' -s /bin/sh someuser shows the user name of someuser in a login sh shell.

Which Scripts for Which Shell?

Now that we have got the different types of shells are out of way, we may proceed to find out the rules for determining which type of shell sources which scripts. In general:

• When a user invokes a login shell, the following scripts are sourced:
  • /etc/profile, which in itself invokes the scripts in /etc/profile.d. Note that only scripts readable by the logging-in user will be sourced — if a start-up script is placed in /etc/profile.d by root it will not be readable by ordinary users by default; one should chmod +r the script after placement.
  • The first available script among the following: ~/.bash_profile (bash only, mostly present on Mac OS X), ~/.bash_login (bash only), ~/.profile (all shells), in that order.
• When a user invokes an interactive shell, the following scripts are sourced:
  • /etc/bash.bashrc, which sets up system-wide stuff that are only relevant in an interactive context, e.g. console prompt, terminal type (xterm, colour), bash completion, etc.
  • ~/.bashrc, a per-user version of /etc/bash.bashrc, also sets up aliases by sourcing ~/.bash_aliases.

So… Where Should My Environment Variables Go?

In general, I find it a good practice to place environment variable set-ups in the start-up scripts for the login shells, namely /etc/profile, /etc/profile.d/*.sh, and ~/.profile. This is perhaps especially true for accumulative environment variables such as PATH; updating such variables in start-up scripts sourced for interactive shells may lead to duplicated segments when interactive shells are invoked in a nested manner.

Instead of editing the PATH variable all in /etc/profile, it can be beneficial to modularise the updates to the environment variable and place them in separate scripts in /etc/profile.d/, one for each application. For example, the Manjaro Linux distribution uses this strategy to set up environment variables for Java and Perl. It also makes it easier to control which variable setting applies to which user by setting up the read permissions on these scripts accordingly.