Activating a conda env inside a Docker container when using docker-compose to start Jupyter notebook

I have the following `Dockerfile`. ``` FROM continuumio/miniconda3:4.5.11 # create a new user (defaults to 'al-khawarizmi') USER root ARG username=al-khawarizmi RUN useradd --create-home --home-dir /home/${username} ${username} ENV HOME /home/${username} # switch to newly created user to avoid running container as root USER ${username} WORKDIR $HOME # build and activate the specified conda environment from a file (defaults to 'environment.yml') ARG environment=environment.yml COPY ${environment} . RUN conda env create --file ${environment} && \ echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc && \ echo "conda activate $(head -1 ${environment} | cut -d' ' -f2)" >> ~/.bashrc ``` The Dockerfile allows the user to specify a conda environment file as a build arg. Here would be a typical `environment.yml file`. ``` name: nessie-py channels: - conda-forge - defaults dependencies: - python=3.6 - "notebook=5.7.*" - "matplotlib=3.0.*" - "numpy=1.15.*" - "pandas=0.23.*" ``` The user can run the image in the standard way and the conda environment will be automatically activated. Running ``` $ docker run -it image_name:image_tag ``` yields a bash prompt within the Docker container with the conda environment activated. ``` (environment_name)$ ``` Now I would like to use `docker-compose` to start a Jupyter notebook server within the container (built with a conda environment file specifying Jupyter as a dependency). When I use the following `docker-compose.yml` ``` version: "3.7" services: notebook-server: build: context: ./ ports: - "8888:8888" volumes: - ./:/home/al-khawarizmi command: jupyter notebook --no-browser ip=0.0.0.0 ``` I get the following error. ``` $ docker-compose up Creating network "nessie-py_default" with the default driver Creating nessie-py_notebook-server_1 ... done Attaching to nessie-py_notebook-server_1 notebook-server_1 | [FATAL tini (7)] exec jupyter failed: No such file or directory nessie-py_notebook-server_1 exited with code 127 ``` I suspected that this error meant that the conda environment is not activated. I then tried adding `tty: true` and `stdin_open: true` to the `docker-compose.yml` thinking that this should invoke and interactive bash prompt prior to running the `command`. This resulted in the same error as above. I also tried defining a `start-notebook.sh` script that explicitly activates the conda environment prior to running the notebook. ``` #!/bin/bash set -e # activate the environment and start the notebook conda activate nessie-py jupyter notebook --no-browser ip=0.0.0.0 ``` results in a different error ``` $ docker-compose up Creating network "nessie-py_default" with the default driver Creating nessie-py_notebook-server_1 ... done Attaching to nessie-py_notebook-server_1 notebook-server_1 | notebook-server_1 | CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'. notebook-server_1 | If your shell is Bash or a Bourne variant, enable conda for the current user with notebook-server_1 | notebook-server_1 | $ echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc notebook-server_1 | notebook-server_1 | or, for all users, enable conda with notebook-server_1 | notebook-server_1 | $ sudo ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh notebook-server_1 | notebook-server_1 | The options above will permanently enable the 'conda' command, but they do NOT notebook-server_1 | put conda's base (root) environment on PATH. To do so, run notebook-server_1 | notebook-server_1 | $ conda activate notebook-server_1 | notebook-server_1 | in your terminal, or to put the base environment on PATH permanently, run notebook-server_1 | notebook-server_1 | $ echo "conda activate" >> ~/.bashrc notebook-server_1 | notebook-server_1 | Previous to conda 4.4, the recommended way to activate conda was to modify PATH in notebook-server_1 | your ~/.bashrc file. You should manually remove the line that looks like notebook-server_1 | notebook-server_1 | export PATH="/opt/conda/bin:$PATH" notebook-server_1 | notebook-server_1 | ^^^ The above line should NO LONGER be in your ~/.bashrc file! ^^^ notebook-server_1 | notebook-server_1 | nessie-py_notebook-server_1 exited with code 1 ``` This error suggests that `bash` is not sourcing `~/.bashrc` prior to running the script. I tried explicitly sourcing `/opt/conda/etc/profile.d/conda.sh` prior to activating the conda environment. ``` #!/bin/bash set -e # activate the environment and start the notebook . /opt/conda/etc/profile.d/conda.sh conda activate nessie-py jupyter notebook --no-browser ip=0.0.0.0 ``` which results in a different error! ``` $ docker-compose up Creating network "nessie-py_default" with the default driver Creating nessie-py_notebook-server_1 ... done Attaching to nessie-py_notebook-server_1 notebook-server_1 | Could not find conda environment: nessie-py notebook-server_1 | You can list all discoverable environments with `conda info --envs`. notebook-server_1 | nessie-py_notebook-server_1 exited with code 1 ``` I can check to see which conda envs are discoverable in the container by running ``` $ docker run -it nessie-py conda info --envs ``` which says that the environment does indeed exist. ``` $ docker run -it nessie-py_notebook-server conda info --envs # conda environments: # nessie-py /home/al-khawarizmi/.conda/envs/nessie-py base * /opt/conda ``` I am out of ideas at this point. This should be possible. Here is an example of a project with a `docker-compose.yml` file, a Dockerfile that specifies a conda environment and starts a Jupyter notebook server. The additional complexities that I need include adding a non-root user to the Dockerfile and creating a new conda environment instead of updating the default `base` conda environment.