Forwarding Docker logs to Logstash

Here at Sandtable we’re as excited about Docker as the next guy. Shippable lightweight containers, yay! We’re running services in Docker containers using Marathon on Mesos in AWS EC2. Now that’s a buzzstack™.

It’s actually going rather well. But when things are not going so well, we turn to logs. We’re already fans of the ELK stack: Elasticsearch + Logstash + Kibana, so naturally we wondered whether we could forward logs from Docker containers to Logstash, all to be explored in the slick new Kibana dashboard.

Here’s our approach.

The Setup

The Approach

We started with a constraint: we didn’t want to change the configuration of Docker containers currently running. Consequently, solutions that required mounting volumes or modifying the Docker images didn’t suit us.

After some searching, we figured we could take a similar approach to that Jason Wilder did but using Logstash instead of Fluentd.

To achieve this we decided to use filebeat (that replaced logstash-forwarder) and Jason’s docker-gen tool (thanks Jason!).

Filebeat is used to transmit data to logstash.

Docker-gen is neat little tool used to automatically generate the configuration file for filebeat given the running containers. Docker-gen watches for Docker events (for example, a new container is started, or a container is stopped), regenerates the configuration, and restarts filebeat.


Note that we do not provide details of the configuration of the Docker images for the ELK stack. See their respective pages for more details.

Step 1: Configure logstash

Filebeat requires the Beats input plugin for Logstash.
The “input” in the logstash configuration looks like this:

input {
  beats {
    port => "5044"
filter {

For more configuration options for the Beats input plugin, see the official documentation.

Step 2: Install docker-gen

# wget
# mkdir /opt/docker-gen
# tar xvzf docker-gen-linux-amd64-0.7.2.tar.gz -C /opt/docker-gen
# mkdir /etc/docker-gen

Step 3: Create the docker-gen template for filebeat

This is the template for the filebeat configuration file.
Create the file /etc/docker-gen/filebeat.tmpl (replace the text in <> by your values):

{{/* this is a docker-gen template. See docker-gen for more details */}}
  {{ range $key, $value := . }}
    - paths:
        - /var/lib/docker/containers/{{ $value.ID }}/{{ $value.ID }}-json.log
        type: {{ $value.Image.Repository }}
  {{ end }}
    hosts: ["<your_logstash_domain>:5044"]
logging.level: warning


This will create as many prospectors as running containers.

For more configuration options, see the official filebeat documentation.

Step 4: Automatically start docker-gen during boot

Option 1: Using Systemd

Create the file: /etc/systemd/system/docker-gen.service :

Description=A file generator that renders templates using Docker Container meta-data.
Documentation= docker.socket

ExecStart=/usr/bin/docker-gen -notify "/etc/init.d/filebeat restart > /dev/null 2>&1 &" -notify-output -watch /etc/docker-gen/filebeat.tmpl /etc/filebeat/filebeat.yml


To activate it, run:
# systemctl enable docker-gen

Option 2: Using Upstart

Create the file /etc/init/docker-gen.conf :


description "docker-gen upstart script"

start on startup
stop on shutdown

  exec /opt/docker-gen/docker-gen -notify "/etc/init.d/filebeat restart > /dev/null 2>&1 &" -notify-output -watch /etc/docker-gen/filebeat.tmpl /etc/filebeat/filebeat.yml
end script

# Log to /var/log/upstart/<job>.log
console log


Step 5: Install filebeat

When writing this blog, the latest version of filebeat is 5.1.2. You can also refer to the official documentation for installation instructions.
Example for Ubuntu installation:
$ curl -L -O
$ sudo dpkg -i filebeat-5.1.2-amd64.deb

We don’t touch at the default configuration file as docker-gen will auto-generate this file.

Step 6: Start docker-gen

# start docker-gen

And that’s all folks. Docker-gen will then create the filebeat configuration file based on the template and restart the service.


Check the log file:


Filebeat send the logs to syslog by default. You can change the log level by modifying the logging level value in the filebeat configuration file (in our case we will modify the docker-gen template /etc/docker-gen/filebeat.tmpl)


Last update: January 20th 2017 – Thanks to Richard Laing for his contribution!


  1. Richard says:

    Hi there I was wished to enquirer about updating the upstart script for 16.04 ? Also in my ELK setup I currently use TLS for the filebeat forwarder, could take a look at this config and tell me if it would work?

    hosts: [“YYYY.XXXX.5044”]
    timeout: 15
    worker: 1
    loadbalance: true
    index: filebeat
    certificate_authorities: [“/etc/ssl/certs/filebeat.crt”]