Many find it confusing, how to debug application in docker containers. In fact, it is reasonably straight forward, it just isn’t documented properly, especially in .Net land. Let’s go over what’s needed:

.Net Core application built in debug mode inside the container

application built in debug mode inside the container VSDBG installed in the docker container

installed in the docker container OpenSSH and correct port/login setup in the container

For your local machine you have many options:

Visual Studio 2017, using in-built SSH functionality Visual Studio Code, using MIEngine and Putty Visual Studio 2017, using MIEngine and Putty

vsdbg vs. clrdbg

Both are debugging tools for .Net Core applications.

As part of the new architecture, clrdbg has been replaced with a new executable named vsdbg. When you see references to clrdbg, just know it’s an out-of-date document.

Configuring Base Container

Our container will run both the OpenSSH server and our application simultaneously. This violates the usual mantra of “one container, one process”, but that is just the cost of doing business. We are setting up the application to run continuously — the debugger does not start the application or kill it when it detaches. I find that such setup is better suited to debugging real-world problems, where your application might run for hours before hitting some invalid state / issue.

We will assume that this particular container we are setting up is just used for testing, and with OpenSSH we prioritise convenience over security.

To configure it, create a file called sshd_config with contents as follows:

# This is ssh server system wide configuration file.

#

# /etc/sshd_config Port 2222

ListenAddress 0.0.0.0

LoginGraceTime 180

X11Forwarding yes

Ciphers aes128-cbc,3des-cbc,aes256-cbc

MACs hmac-sha1,hmac-sha1-96

StrictModes yes

SyslogFacility DAEMON

PasswordAuthentication yes

PermitEmptyPasswords no

PermitRootLogin yes # Nessesary for VS2017, but not for VSCode!

Subsystem sftp internal-sftp

Although the rest of this example uses Password Authentication, we will we will additionally create an SSH key that we can use. If you don’t need an SSH key, skip this step and delete the corresponding line form the dockerfile.

Create file authorized_keys and place the ssh key into it. In my case, the file looks like this, the entire key must be in a single line.

ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAkeoEOUi0HcXu6QZ0X1Qu4Msq5kVYXRjmNFXSUUbG3hYtgiCT9cq0k45phlKbrafFjEuo7BiaboHRz6JLdkYCBQT9aviTHtElpqmNUtQsaDvVGBBrNZ8vUQT0PXq/lYAi6onUxZLNbykYlBBuIgNVQhWqnmsHNXaFxKsG0BJADQeEzYHjJuEuhAtPgiWSP5ViR5hrZk0qdZMLiS8PHcw0XCQMcM2qU0v10PzSIVUUyAwRRgGWWfXmy2P0XcwAg95LVfosCaeFsaW9rb35/kYPRNlf6QxP/9BOsjwhxVWAV7eYWYZlkwqAsQY1g2qhZ5Nzy1N+JwaPhmirE5xO4zdVwQ==

Now we will create a container that will be used as a base container for our future exploits:

FROM microsoft/aspnetcore:2.0 AS base # Install the SSHD server

RUN apt-get update \

&& apt-get install -y --no-install-recommends openssh-server \

&& mkdir -p /run/sshd # Set password to 'Docker!'. Change as needed.

RUN echo "root:Docker!" | chpasswd #Copy settings file. See elsewhere to find them.

COPY sshd_config /etc/ssh/sshd_config

COPY authorized_keys root/.ssh/authorized_keys # Install Visual Studio Remote Debugger

RUN apt-get install zip unzip

RUN curl -sSL https://aka.ms/getvsdbgsh | bash /dev/stdin -v latest -l ~/vsdbg EXPOSE 2222

Configured base container is available at dockerhub as clumsypilot/dotnetdebug: asp-runtime-2.0

Configure Debugable application container

We will assume that you have a debugable application. I have just created an ASP.net core application from a template and left it as-is. I have called it DebugSample and it produces a dotnet executable DebugSample.dll

The first thing we need is a script that will start both the application and OpenSSH like the example give on dockerDocumentation. StartSSHAndApp.sh

#!/bin/bash

# Start the first process

dotnet /app/DebugSample.dll </dev/null &>/dev/null &s

tatus=$?

if [ $status -ne 0 ]; then

echo "Failed to start ASP Server: $status"

exit $status

fi /usr/sbin/sshd -D

if [ $status -ne 0 ]; then

echo "Failed to start Ssh Server: $status"

exit $status

fi # Naive check runs to see if either of the processes exited.# This illustrates part of the heavy lifting you need to do if you want to run# more than one service in a container. The container exits with an error# if it detects that either of the processes has exited.# Otherwise it loops forever, waking up every 5 seconds while sleep 5; do

ps aux |grep dotnet |grep -q -v grep

PROCESS_1_STATUS=$?

ps aux |grep sshd |grep -q -v grep

PROCESS_2_STATUS=$?

# If the greps above find anything, they exit with 0 status

# If they are not both 0, then something is wrong

if [ $PROCESS_1_STATUS -ne 0 -o $PROCESS_2_STATUS -ne 0 ]; then

echo "One of the processes has already exited."

exit 1

fi

done

Following this, create a container that will run this script. You can substitute base container for the we’ve created in previous step.

FROM clumsypilot/dotnetdebug:asp-runtime-2.0 AS base

WORKDIR /app

MAINTAINER Vladimir Vladimir@akopyan.me

FROM microsoft/aspnetcore-build:2.0 AS build

WORKDIR /src

COPY ./DebugSample .

RUN dotnet restore FROM build AS publish

RUN dotnet publish -c Debug -o /app FROM base AS final

COPY --from=publish /app /app

COPY ./StartSSHAndApp.sh /app EXPOSE 5000 CMD /app/StartSSHAndApp.sh #If you wish to only have SSH running and start #your service when you start debugging#then use just the SSH server, you don't need the script

#CMD ["/usr/sbin/sshd", "-D"]

A container built in this manner is avaliable at dockerhub as clumsypilot/dotnetdebug:asp-debug-sample

Basic Commands and References

Changing root password for SSH

If you want to use the base container, but you want it to be secure, change the root password

echo "root:Docker!" | chpasswd

Key File location

root/.ssh/authorized_keys

If want to secure the base container with your own Key, or use password, replace or delete it. If the key is updated, it is immediately applied — no need to restart OpenSSH.

Inspecting the container

You might want to inspect the container. You can attach to the running container as a separate Bash process

docker exec -it <id of running container> bash

VSDBG doesn’t start

Error from pipe program 'plink.exe': bash: /root/vsdbg/vsdbg: cannot execute binary file: Exec format error

To check executable architecture:

file /root/vsdbg/vsdbg

file is not usually installed in docker containers by default, so you might have to use apt-get.

Starting container in Docker for connect-ability

docker run -d -p 2222:2222 -p 5000:5000 clumsypilot/dotnetdebug:asp-debug-sample

Port 2222 is used for debugging, the port 5000 is for the web-server in asp-debug-sample. You might use a different one.

Connecting to a pod in a Kubernetes cluster

kubectl port-forward <POD-NAME> 2222

Now you can use LocalHost as destination address in your connectivity settings!

Detached Start of a program in Linux

dotnet program.dll </dev/null &>/dev/null &

Starts the program in a detached manner and redirects all Stdin/Stdout to null

Script Doesn’t work

Make sure you have linux line endings — if you create the script file in windows, you will have windwos line endings and bash won’t run the script and start your application and OpenSSH.

VSCODE gets Stuck

If you get password or username wrong, VSCODE gets stuck and hangs on:

Starting: “plink.exe” -l root -pw Docker! localhost -P 2222 “~/vsdbg/vsdbg — interpreter=vscode”

Visual Studio 2017 Debugging

Debugging in VS2017 is the most straight forward way of remote debugging, but also has an annoying issue that’s not documented anywhere (more on that later)