Shiny Python base image

shiny for python

In this part, we will described the specific Dockerfile for Python.

Dockerfile.python

Here the content of the Dockefile.python

 1# Build image from base image `python`
 2FROM python:${PYTHON_VERSION}
 3
 4# Define geographic location during installation
 5ENV TZ=Etc/UTC
 6ENV DEBIAN_FRONTEND=noninteractive
 7ENV SHINY_LOG_STDERR=1
 8
 9EXPOSE 3838
10CMD ["shiny-server"]
11
12# Install dependencies
13RUN apt-get update \
14&& apt-get install -y \
15    software-properties-common \
16    wget \
17    nano \
18    wget \
19    curl \
20    lsb-release \
21    libcurl4-openssl-dev \
22    gcc \
23    gfortran \
24    g++ \
25    libreadline-dev \
26    zlib1g-dev \
27    libbz2-dev \
28    liblzma-dev \
29    libpcre2-dev \
30    libcairo-dev \
31    libpng-dev \
32    libicu-dev \
33    libxml2-dev \
34    libssl-dev \
35    make \
36    libtiff-dev \
37    xorg-dev \
38    libx11-dev \
39&& rm -rf /var/lib/apt/lists/*
40
41# Upgrade pip and install wheel, shiny :
42RUN pip install --no-cache-dir --upgrade pip \
43&& python -m pip install wheel shiny
44
45# Install shiny server
46ARG SHINY_SERVER_VERSION=latest
47RUN mkdir /opt/scripts
48COPY ./scripts/install_shiny_server.sh /opt/scripts/install_shiny_server.sh
49RUN /opt/scripts/install_shiny_server.sh
50
51# Allows trafic from all ips
52RUN sed -i "s/3838/3838 0.0.0.0/g" /etc/shiny-server/shiny-server.conf
53
54# Grant shiny to created bookmark state directory in /var/lib/
55# Grant shiny to write logs in /var/log/
56# Grant shiny to write in /srv/shiny-server/www/
57RUN chown shiny:shiny /var/lib/shiny-server \
58&& mkdir -p /var/log/shiny-server \
59&& chown shiny:shiny /var/log/shiny-server \
60&& mkdir -p /srv/shiny-server/www \
61&& chown shiny:shiny /srv/shiny-server/www

Line 2: The base image

1# Build image from base image `python`
2FROM python:${PYTHON_VERSION}

The FROM ... indicate which base image should be used, the image contains:

  • an OS with python

  • python’s “shiny” package

  • the shiny-server

In the gitlab-ci.yml, we decide to provide selected version of Python base image based on several python version and compatible with kubernetes architecture deployed at Pasteur. Actually (July 2023), three versions are available and mentionned in the gitlab-ci.yml`, as you can see extract below :

1...
2    PYTHON_VERSION: [
3      "3.9-slim-bullseye",
4      "3.10-slim-bullseye",
5      "3.11-slim-bullseye",
6    ]
7...

For more details about gitlab-ci.yml, please see the dedicated pages for gitlab-ci.yml.

Lines 5-7 : Environment variables

4# Define geographic location during installation
5ENV TZ=Etc/UTC
6ENV DEBIAN_FRONTEND=noninteractive
7ENV SHINY_LOG_STDERR=1

Some environment variables need to be defined in the container and set parameters needed for shiny-server installation or configuration.

Here, we set :

  • line 5 : the System Timezone TZ to Etc/UT

  • line 6 : the debconf frontends (interaction with the debconf interface) DEBIAN_FRONTEND to noninteractive. noninteractive permit to have zero interaction while installing or upgrading the system via apt.

  • line 7 : the standard error output SHINY_LOG_STDERR to a non-empty value 1 . To saving individual process logs to disk, Shiny Server can also relay those logs to its own stderr (which, by default, is saved to /var/log/shiny-server.log).

Line 9 : Listen port for Shiny-server

9EXPOSE 3838

By default, Shiny Server listens on port 3838, so the default application will be available at http://<server-address>:3838. We decide to conserve the default port mentionned in the /etc/shiny-server/shiny-server.conf, configuration file of shiny-server.

The EXPOSE instruction informs Docker that the container listens on the specified network ports at runtime. So, here we indicate the 3838 listen port.

Warning

If you change the default value of listen port, you must modify it also the /etc/shiny-server/shiny-server.conf.
More details below in the section line 52

Line 10 : Run Shiny-server

10CMD ["shiny-server"]

The main purpose of a CMD is to provide defaults for an executing container. Here, CMD ["shiny-server"] permits to launch the shiny-server, normally run by the command line shiny-server. Thanks to CMD, the shiny-server will be launch by default when the container will be run.

Note

Threre can only be one CMD instruction in a Dockerfile.

Lines 13-39 : The system dependencies

12# Install dependencies
13RUN apt-get update \
14&& apt-get install -y \
15    software-properties-common \
16    wget \
17    nano \
18    wget \
19    curl \
20    lsb-release \
21    libcurl4-openssl-dev \
22    gcc \
23    gfortran \
24    g++ \
25    libreadline-dev \
26    zlib1g-dev \
27    libbz2-dev \
28    liblzma-dev \
29    libpcre2-dev \
30    libcairo-dev \
31    libpng-dev \
32    libicu-dev \
33    libxml2-dev \
34    libssl-dev \
35    make \
36    libtiff-dev \
37    xorg-dev \
38    libx11-dev \
39&& rm -rf /var/lib/apt/lists/*

Shiny-server require some system dependencies. Here, we use the apt-get command line in order to install all dependencies requiered in the container.

Note

You can add system packages will be requiered by your future shiny app but don’t delete one in these previous list.

Lines 42-43 : Installation Shiny package for python

41# Upgrade pip and install wheel, shiny :
42RUN pip install --no-cache-dir --upgrade pip \
43&& python -m pip install wheel shiny

One of the requirements for developp Shiny app with Python, after get Python on your container, is the shiny package. The lines 42-43 permit to install the packages from PyPI, as mentionned on officiel documentation (Installing shiny for Python)

Lines 46-49 : Installation Shiny Server

45# Install shiny server
46ARG SHINY_SERVER_VERSION=latest
47RUN mkdir /opt/scripts
48COPY ./scripts/install_shiny_server.sh /opt/scripts/install_shiny_server.sh
49RUN /opt/scripts/install_shiny_server.sh
Here, we install the shiny-server. In order to make easily, we write a bash script contained all instructions and parameters needed.
The line 46 permits to set a value of Shiny-server version chosen for installation.
In this part of the Dockefile, we only create a folder named /opt/scripts in line 47. We indicate to copy the bash script named /scripts/install_shiny_server.sh in the folder created previously, /opt/scripts/install_shiny_server.sh in line 48.
The last instruction line 49 just run the bash script.

More details about the bash script in the helper scripts section

Line 52 : Custom Shiny-server configuration file

51# Allows trafic from all ips
52RUN sed -i "s/3838/3838 0.0.0.0/g" /etc/shiny-server/shiny-server.conf

In order to access correctly to your app, we set again the listen port in the specific configuration file of shiny-server. The line 52 permits to set the chosen values in the /etc/shiny-server/shiny-server.conf

Lines 57-61 : Permissions

54# Grant shiny to created bookmark state directory in /var/lib/
55# Grant shiny to write logs in /var/log/
56# Grant shiny to write in /srv/shiny-server/www/
57RUN chown shiny:shiny /var/lib/shiny-server \
58&& mkdir -p /var/log/shiny-server \
59&& chown shiny:shiny /var/log/shiny-server \
60&& mkdir -p /srv/shiny-server/www \
61&& chown shiny:shiny /srv/shiny-server/www

By default, Shiny Server will create and store data in /var/lib/shiny-server/. instruction line 57 permits to allow the user shiny to write in this dedicated folder.

All information related to Shiny Server itself, rather than a particular Shiny application, is logged in the global system log stored in /var/log/shiny-server.log. This log should be checked often to ensure Shiny Server is performing as expected. Any errors and warnings that Shiny Server needs to communicate will be written here. In order to access these log files, we grant the user shiny to write in the dedicated folder (lines 58-59).

In order to share easily data with the futur user of the shiny app, we add instruction to allow user shiny to access at /srv/shiny-server/www folder.

Specific point in gitlab-ci

Due to the use of two language - R or Python -, we define two distinct Dockerfile with their own configuraton. To distinct them, we just add an extension : .r for R base image and .python for Python base image. By default, the gitlab-ci.yml want to find a file named Dockerfile not Dockerfile.r or Dockerfile.python. In order the CI use the two files previously written, we add a command line to indicate it line 31 in the gitlab-ci.yml, as you can see extract below :

...
script:
    ...
    - envsubst < ./Dockerfile.$LANGUAGE > Dockerfile
    ...
...

For more details about gitlab-ci.yml, please see the dedicated pages for gitlab-ci.yml.

Note

More details about the content of Dockerfile in Dockerfile reference