Setting up a Docker development environment

Using Docker

Download and prepare a Docker

The following setup is based on the Hackathon setup here.

Prerequisites

Docker: you should be able to run Docker containers. On MacOS you should have the latest version of Docker, which finally has built in virtualization. Be sure to use the most recent version of upgrade-hackathon-setup, as well, as several Mac related issues were fixed, and Mac is now fully supported with no (known) caveats. Also ensure that you're logged in on Docker by running the command docker login, supplying your Docker ID (username, not email address!) and password.
Fuse: You should have a very recent version of Fuse (or OSXFuse on MacOS)
CVMFS: you should have access to /cvmfs/lhcb.cern.ch. On a Mac you will need OSXFuse and CernVM-FS.

Mounting CVMFS manuallyClick to expand

If you want to mount CVMFS manually, such as for accessing it for other tasks, this is how you can do that:

Set your proxy, for example:

local:~ $ echo "CVMFS_HTTP_PROXY=DIRECT" | sudo tee -a /etc/cvmfs/default.local

Mount a CernVM-FS repository using the following steps:

local:~ $ sudo mkdir /cvmfs/lhcb.cern.ch
local:~ $ sudo mount -t cvmfs lhcb.cern.ch /cvmfs/lhcb.cern.ch
local:~ $ sudo mkdir /cvmfs/lhcbdev.cern.ch
local:~ $ sudo mount -t cvmfs lhcbdev.cern.ch /cvmfs/lhcbdev.cern.ch

If you are on Mac, you will also need to add the /cvmfs directory to the shared directory list in the Docker applet.

Quick start

To get started, get the tools with:

local:~ $ git clone ssh://git@gitlab.cern.ch:7999/lhcb/upgrade-hackathon-setup.git hackathon

Using your ssh key inside the containerClick to expand

If you have loaded your identity, you can share to the container. To load it, type:

local:~ $ test -z "$SSH_AGENT_PID" && eval $(ssh-agent)
local:~ $ ssh-add

This is also useful, since it keeps you from having to type a passphrase every time you try to use git. The test -z command is just ensuring that you don't run the agent multiple times. If you are using multiple identities on one account, you can tell ssh-add to add the correct one.

Then from the hackathon directory just created invoke:

local:hackathon $ ./lb-docker-run -c x86_64-centos7-gcc7-opt

which will pull the latest image of the CentOS7 Docker image we use to build our software and start an interactive shell in the special directory /workspace, mapped to the local hackathon directory.

Note about the options of the `lb-docker-run` commandClick to expand

This command has quite a few options. You can see them with:

local:hackathon $ ./lb-docker-run -h

The persistent home option used above may require that you own /<username> on your system as well.

The --ssh-agent option forwards your ssh identity to the container (if one was added). You can also try the --wrap option to set up defaults differently, and to run in the same directory instead of /workspace.

Building the whole stack the first time may take a lot of time, so you can optionally get a pre-built image with:

local:hackathon $ make pull-build

At this point you can build the software stack with (will build automatically in parallel):

local:hackathon $ make

If you didn't pull the pre-built image, this command will checkout the code and build from scratch.

Using CVMFSClick to expand

Using CVMFS requires a few changes compared to working on LxPlus, where you have access to AFS. If you run an lb-dev or lb-run command on the nightlies, you should add --nightly-cvmfs to the command. For example:

local:hackathon $ lb-run --nightly-cvmfs --nightly lhcb-head Kepler HEAD $SHELL

Would drop you into a shell with the CVMFS nightly head of the Kepler package.

Building a project

This is how you would run MiniBrunel:

local:~ $ . /cvmfs/lhcb.cern.ch/lib/lhcb/LBSCRIPTS/dev/InstallArea/scripts/LbLogin.sh -c x86_64-slc6-gcc49-opt
local:~ $ lb-dev --nightly-cvmfs --nightly lhcb-future 282 Brunel/future
local:~ $ cd BrunelDev_future
local:BrunelDev_future $ git lb-use -q Brunel
local:BrunelDev_future $ git lb-use -q Rec
local:BrunelDev_future $ git lb-checkout Brunel/future Rec/Brunel
local:BrunelDev_future $ make
local:BrunelDev_future $ ./run gaudirun.py Rec/Brunel/options/MiniBrunel.py

Setting up with Docker