Docker setup
Docker ⤴ is a lightweight platform for automating the deployment of applications within containers. The platform also offers a centralized repository for sharing and storing containers. Download Docker Desktop ⤴ from the Docker Website. Follow the default installation process and open the Docker Desktop app after finishing installation.
What are Containers?
Containers are executable units of software containing anything from a small process to a larger application, along with its libraries and dependencies. Start with the Apptainer: the container system for secure HPC tutorial to get to know about using containers.
Finding Containers
Apptainer (previous Singularity) is commonly used for High-Performance Computing (HPC) environments whereas Docker is usually run locally as it needs admin privileges. Docker container images can be searched from the Docker Hub ⤴ or directly from the Docker Desktop app.
Apptainer (previous Singularity) is able to support Docker containers, which means that you can run Docker containers using Apptainer. But, Docker does not natively support Apptainer/Singularity containers. However, there are some workarounds that allow you to run Apptainer containers inside a Docker container, such as using Apptainer as an entry point for the Docker container.
Pulling a container image
The simplest way to pull (download) a container image is using the Docker Desktop app.
A. Open the app:
Let’s try pulling image of the ncbi-blast-plus container through the app.
The app is connected to docker hub and you can directly search for images from the search bar at the top. Click on Images tab to see the list of search results.
Pull the image by clicking on the desired container and then button.
Alternatively, you can copy the pull command by clicking on the icon and run it in a terminal.
When you pull an image in Docker, it downloads a snapshot of the filesystem and configuration settings that make up the application environment. This image is stored in your Docker host’s local storage.
The image alone is inert – it doesn’t run or do anything by itself.
The Run button is used to create and start a container from an image.
When you run the image, Docker does the following:
- It creates a new container instance. This is a runnable instance of the image with its own isolated environment.
- It sets up an isolated filesystem for the container, which is derived from the image but allows for changes without affecting the underlying image.
- It initializes the environment according to any configurations specified in the Dockerfile (like environment variables) or provided in the run command (like port mapping, volume attachment, etc.).
- It executes the default command specified in the image or any command you specify in the run command.
B. You can also search for the Docker containers online from the Docker Hub website ⤴.
Use the search bar at the top to search and then click on the desired container.
Now copy the pull command by clicking on it.
Open a terminal and paste the command: (assuming you have docker installed already)
docker pull biocontainers/ncbi-blast-plus
Press . The image will be downloaded.
Using Docker containers in command line
Docker can be used through the app or via command line (use in terminal).
Open the terminal and enter the following to check the version of docker (and if it installed properly).
docker --version
Docker version 20.10.16, build aa7e414
Run following command to see full list of commands for docker.
docker --help
see the expected output (Docker help message)
Usage: docker [OPTIONS] COMMAND A self-sufficient runtime for containers Options: --config string Location of client config files (default "/Users/abadacz/.docker") -c, --context string Name of the context to use to connect to the daemon (overrides DOCKER_HOST env var and default context set with "docker context use") -D, --debug Enable debug mode -H, --host list Daemon socket(s) to connect to -l, --log-level string Set the logging level ("debug"|"info"|"warn"|"error"|"fatal") (default "info") --tls Use TLS; implied by --tlsverify --tlscacert string Trust certs signed only by this CA (default "/Users/abadacz/.docker/ca.pem") --tlscert string Path to TLS certificate file (default "/Users/abadacz/.docker/cert.pem") --tlskey string Path to TLS key file (default "/Users/abadacz/.docker/key.pem") --tlsverify Use TLS and verify the remote -v, --version Print version information and quit Management Commands: builder Manage builds buildx* Docker Buildx (Docker Inc., v0.8.2) compose* Docker Compose (Docker Inc., v2.2.3) config Manage Docker configs container Manage containers context Manage contexts image Manage images manifest Manage Docker image manifests and manifest lists network Manage networks node Manage Swarm nodes plugin Manage plugins sbom* View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0) scan* Docker Scan (Docker Inc., v0.17.0) secret Manage Docker secrets service Manage services stack Manage Docker stacks swarm Manage Swarm system Manage Docker trust Manage trust on Docker images volume Manage volumes Commands: attach Attach local standard input, output, and error streams to a running container build Build an image from a Dockerfile commit Create a new image from a container's changes cp Copy files/folders between a container and the local filesystem create Create a new container diff Inspect changes to files or directories on a container's filesystem events Get real time events from the server exec Run a command in a running container export Export a container's filesystem as a tar archive history Show the history of an image images List images import Import the contents from a tarball to create a filesystem image info Display system-wide information inspect Return low-level information on Docker objects kill Kill one or more running containers load Load an image from a tar archive or STDIN login Log in to a Docker registry logout Log out from a Docker registry logs Fetch the logs of a container pause Pause all processes within one or more containers port List port mappings or a specific mapping for the container ps List containers pull Pull an image or a repository from a registry push Push an image or a repository to a registry rename Rename a container restart Restart one or more containers rm Remove one or more containers rmi Remove one or more images run Run a command in a new container save Save one or more images to a tar archive (streamed to STDOUT by default) search Search the Docker Hub for images start Start one or more stopped containers stats Display a live stream of container(s) resource usage statistics stop Stop one or more running containers tag Create a tag TARGET_IMAGE that refers to SOURCE_IMAGE top Display the running processes of a container unpause Unpause all processes within one or more containers update Update configuration of one or more containers version Show the Docker version information wait Block until one or more containers stop, then print their exit codes Run docker COMMAND --help for more information on a command. To get more help with docker, check out our guides at https://docs.docker.com/go/guides/
Run Docker container using Apptainer or Singularity
If you are unable to use Docker, for example you may be on an HPC, you can run Docker containers through Apptainer/Singularity.
module load apptainer
apptainer pull docker://user/image:tag
or
module load singularity
singularity pull docker://user/image:tag
Check out the official guide ⤴ to apptainer pull command for more details.
Check out the official guide ⤴ singularity pull command for more details.
Let’s test the blast-plus container image we pulled by doing a simple blast between two fasta files. I put my fasta files in a folder named blast on my desktop.
Run the image in the Docer Desktop
First we need to Run the image to convert it into a container.
When you Run an image in Docker, you’re not just making a local copy of the container; you’re actually initializing and starting a new container instance based on the image’s blueprint, ready to serve whatever application or task the image is set up to perform. This step is critical because it translates the static image into an active process capable of performing tasks or serving applications.
This can be done via the Docker Desktop app.
A dialogue box will open; click on the optional settings and add path to your data in Host path and type \data in Container path under Volumes section.
Click Run to initiate a new container.
Go to the Containers tab (top-left) and find the running container. Click on it to open details.
Make sure the container is running. You should see a button, if not click on the button. Then go to the terminal to use the blast and all its options.
Run the image in the command line
I had trouble at container step as it would not run and exit immediately.
This could happen due to a number of reasons including memory problems or problems with a path.
There is an easy fix for in-GUI-related problems, i.e., make the container (run the image) in a terminal. You will need the image ID for this, so copy it from Images tab in the Docker Desktop.
copied image-ID: eaeed4320e5as0b1d136533f02
You can also find images and their details including image ID from the terminal.
Open a terminal (for example, PowerShell on windows) and enter docker images to list all images.
To initiate a container in the command line, follow the syntax:
docker run -v path-to-your-blast-files:/data -it image-ID bash
by replacing the path-to-your-blast-files with the path to your input files and pasting the copied image-ID. It might look something like this:
docker run -v C:\Users\abc\Desktop\blast:/data -it eaeed4320e5as0b1d136533f02 bash
The :/data after your path is the default container path (keep it as is) and the image-ID is usually a long string of numbers and letters. This will initiate an interactive run of the docker container.
You can now use the blast programs in the terminal directly:
blastn -query testq.fa - subject testsub.fa -outfmt 6 -out Blast_out
Running this command successfully generated the output file, which I named Blast_out.
ls Blast_out testq.fa testsub.fa
You should now see that the container is running in Docker Desktop. You can also use the app’s built-in terminal to run the blast commands.