This repository contains an experimental study relying on the open-source version of the test_FEMBEM solver test suite with the following structure:

• benchmarks
  • definitions.csv
  • run.sh
• figures
  • short-pipe.png
• public
• .gitignore
• .gitlab-ci.yml
• README.md
• container.def
• plot.R
• references.bib
• study.tex

In this case, we do not rely on Guix and literate programming using Org mode to ensure reproducibility of the research study. To build a software environment clode enough to the original for redoing the experiments, we can use either the combination of native system package manager and manual builds, as detailed in README.md, or the accompanying pre-built Singularity container defined in container.def.

As of redoing the experiments defined in definitions.csv using the dedicated run.sh shell script, we must settle for the instructions and explanations in the README.md file and comments in associated source code files.

The plot.R R script allows for generating figures based on experimental results into the figures folder. Finally, study.tex and references.bib represent the LaTeX source of the study manuscript and the referenced bibliography, respectively.

Note that the public folder is used by the continuous integration engine for publishing repository's static webpage hosted on GitLab pages.

This repository contains the same research study as in Baseline setup repository. However, here we do rely on Guix and literate programming using Org mode to ensure reproducibility of the study. See the structure of the repository below:

• benchmarks
  • definitions.org
  • run.org
• figures
  • short-pipe.png
• public
• styles/RR
• .gitignore
• .gitlab-ci.yml
• README.md
• channels.org
• macros.org
• manifests.org
• plot.org
• publish.el
• readtheorginria.setup
• references.bib
• reproducing-guidelines.org
• study-article.org
• study-research-report.org
• study.org

The first thing we can observe is that all the source code files are written in Org this time. The Org syntax allow us to combine formatted text with blocks of source code. The latter can then be extracted (or tangled in the Org terminology) OrgTangle into corresponding source files, e.g. run.org can be tangled into run.sh and so on. It is also possible to evaluate code blocks directly from within Org files OrgEval. Note that an Org file can also be easily exported OrgExport to various output formats such as PDF (through LaTeX), HTML, ODF, plain text, etc. We further detail the Org format later in Section 5.4.

We added some more files to the repository too. channels.org and manifests.org represent the specification of the Guix software environment of the study (see Section 5.3). study.org contains the study manuscript. study-article.org and study-research-report.org are wrappers for study.org representing an article version (as in Baseline setup) and an Inria research report version of the study manuscript. reproducing-guidelines.org combines the literate descriptions of all the sources in the repository and thus provides an exhaustive documentation of experiments, source code and procedures involved in the construction of related software environments, execution of benchmarks as well as gathering and post-processing of results.

readtheorginria.setup and the files in styles contain the HTML and LaTeX templates used for publishing the study manuscripts and the reproducing guidelines. We do not report further on these.

Finally, publish.el is an Emacs Lisp script for exporting the Org files into different output formats and publishing them into the public folder. As it is not associated with the core subject, we will consider this script as a black box during the hands-on session. Do not hesitate to ask more details about it though.

Note that macros.org provides Org macros OrgMacros reused throughout the Org documents in the repository. This is similar to custom LaTeX commands.

The hands-on session will be based on this repository. The master branch contains the complete configuration we should have built by the end of the session. The level0 branch represents the starting point for the participants to be completed during the session. For the participants joining us later or wanting to skip one or more phases, there are the other levelX branches corresponding to different levels of completion of the hands-on session.

Here, we assume that we are running a third-party Linux distribution such as Debian GNU/Linux or Ubuntu. We can install the Guix package manager on top of that distribution without interferring with our primary package manager. To do so, we use an installing shell script that needs to be run with superuser privileges.

cd /tmp
wget https://git.savannah.gnu.org/cgit/guix.git/plain/etc/guix-install.sh
chmod +x guix-install.sh
sudo ./guix-install.sh

Then, we just need to follow on-screen instructions.

After the installation, we proceed with a short sequence of commands to ensure a smooth user experience with Guix onward. At the beginning, we install our first package using Guix, i.e. glibc-locales to allow the latter to switch locales.

guix install glibc-locales

Then, to be able to acquire new versions of installed packages, we will need to pull new version of Guix first. The following command can take a while to execute, especially when run for the first time.

guix pull

Once the process finishes, we need to follow the hint the command gives us and add the following lines to our .bash_profile or .bashrc to always get access to the most recent Guix generated by guix pull.

GUIX_PROFILE="$HOME/.config/guix/current"
. "$GUIX_PROFILE/etc/profile"

We also have to tell to our shell to use this new Guix.

hash guix

Finally, we can update our installed packages.

guix upgrade

To get information on the generation (version in Guix terminology) of Guix being used, we can use:

guix describe

Let us enter our first Guix environment containing four packages, bash, cowsay, emacs and emacs-org, using the guix shell command and launch a shell inside of that environment. We should not forget the --pure switch which prevents existing environment variables from pulluting the target Guix environment. Note that the --norc option to bash prevents the shell from loading our .bashrc file which could pollute the final environment despite passing the --pure option to guix shell.

guix shell --pure bash cowsay emacs emacs-org -- bash --norc

We can simply type exit to get back to our original shell. Also, we do not have to run an interactive shell inside of the environment. We can directly execute a given command like for example:

guix shell --pure bash cowsay emacs emacs-org -- cowsay "Hello world!"

The above should give us the following output:

 ______________ 
< Hello world! >
 -------------- 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

We choose to write the source code of scripts and various configuration files allowing us to design and automatize numerical experiments in respect of the paradigm known as literate programming Knuth84. The idea of this approach is to associate source code with an explanation of its purpose written in a natural language.

There are numerous software tools designed for literate programming. We rely on Org mode for the Emacs text editor Dominik18,emacs which defines the Org markup language allowing to combine formatted text, images and figures with traditional source code. Files containing documents written in Org mode should end with the .org extension.

Extracting a compilable or interpretable source code from an Org document is called tangling OrgTangle. It is also possible to evaluate a particular source code block directly from the Emacs editor OrgEval while editing. For example, this can be particularly useful for the visualization of experimental results.

Eventually, an Org document can be exported to various output formats OrgExport such as LaTeX or Beamer, HTML and so on.

Listing 1 shows an example of Org syntax. In this exceprt, there is some formatted text followed by a Python code block. The line starting with #+PROPERTY specifies that all of the source code blocks in that particular Org file should be tangled into a Python script file named rss.py. Figure 1 shows an HTML output corresponding to Listing 1.

#+PROPERTY: header-args :tangle rss.py ...
...
Memory usage statistics of a particular process are
stored in ~/proc/<pid>/statm~ where ~<pid>~ is the
process identifier (PID). In this file, the field
=VmRSS= holds the amount of real memory used by the
process at instant $t$. See the associated function
below.

#+BEGIN_SRC python
def rss(pid):
    with open("/proc/%d/statm" % pid, "r") as f:
        line = f.readline().split();
        VmRSS = int(line[1])
        return VmRSS
#+END_SRC
...

We will now compose an Org file by ourselves. Let us enter our previous Guix environment, open Emacs inside and create a file named hello.org.

guix time-machine -C my-channels.scm -- shell --pure -m my-manifest.scm -- \
     emacs --no-init-file hello.org

Then, we will try to describe the below short shell script in that Org file following the aforementionned example. Note that tangling of hello.org should produce a shell script file named hello.sh.

MESSAGE="Hello world!"

if test "$1" != "";
then
  MESSAGE="$1"
fi

cowsay "$MESSAGE"

If all went well, we should have something similar to this in our hello.org file.

#+PROPERTY: header-args :tangle hello.sh

This file describes a simple shell script ~hello.sh~. It begins by defining a
default greeting message.

#+BEGIN_SRC shell
MESSAGE="Hello world!"
#+END_SRC

However, if the user provides a custom message, we prefer to show this one
instead.

#+BEGIN_SRC shell
if test "$1" != "";
then
  MESSAGE="$1"
fi
#+END_SRC

Finally, let the cow say our message!

#+BEGIN_SRC shell
cowsay "$MESSAGE"
#+END_SRC

To save our modifications to hello.org, we can use C-x C-s. In order for the #+PROPERTY setting to take effect, we need to refresh the current buffer by selecting Org > Refresh/Reload > Refresh setup current buffer from the Emacs application menu.

Then, to tangle the shell script from our Org file, we can use this sequence of keystrokes C-c C-v C-t. To execute our shell script in our Guix environment, we close Emacs using C-x C-c and fire the following command.

guix time-machine -C my-channels.scm -- shell --pure -m my-manifest.scm -- \
     bash hello.sh "I can do Guix and Org now!"

We are now ready for the core part of the hands-on session. We are going to make the research study from the Baseline setup repository better reproducible thanks to Guix and the literate programming approach in Org mode.

Before we begin, we need to connect to our Inria GitLab account, and fork the Advanced setup repository we are going to work with. It already contains all of the files required for it to work but some of them need to be completed.

Then, we make a local clone of our fork using the git clone command and navigate to the root directory of the clone. Now, depending on where in the hands-on session we want to join, we checkout the right branch to start from:

• git checkout level0: the channel definition (the very beginning),
• git checkout level1: the manifest definition,
• git checkout level2: the creation of a literate description of a source code file in Org,
• git checkout level3: the reproduction of the study in a Guix environment.

Note that whenever we need to edit an Org file during the hands-on session, we can use the following command.

guix shell --pure emacs emacs-org -- emacs --no-init-file <file-name>.org