LogDNA Agent

Mezmo, formerly LogDNA, enables enterprises to ingest all of their log data to a single platform, normalize it, and route it to the appropriate teams so that they can take meaningful action in real time. Join us at the Mezmo community forum.

The LogDNA agent is a resource-efficient log collection client that forwards logs to [LogDNA]. This version of the agent is written in [Rust] to ensure maximum performance, and when coupled with LogDNA's web application, provides a powerful log management tool for distributed systems, including [Kubernetes] clusters. Supported platforms include Linux, Windows and Mac.

:information_source: Information

As part of the new company name change, we will be changing the name of our agent to mezmo-agent. This will happen over a series of releases. In 3.6 we will be introducing environment variables prefixed with MZ in addition to the existing prefix of LOGDNA. In 3.7 we will release both the agent binaries and yaml files with mezmo in the names. Here again we will do this in tandem with the logdna name. In our 4.0 version, we will fully deprecate all previous references to LOGDNA_ and logdna.

[LogDNA]: https://logdna.com [Rust]: https://www.rust-lang.org/ [Kubernetes]: https://kubernetes.io/

Table of Contents

• Managing Deployments

- Installing on Kubernetes - Using manifest files - Using Helm - Installing on OpenShift - Installing on Linux - Running as Non-Root - Additional Installation Options - Installing on Windows

• Building

- Building Docker image

• Configuration

- Options - Configuring the Environment - Configuring Lookback - Configuring Journald - Configuring regex for redaction and exclusion or inclusion - Resource Limits - Exposing Agent Metrics

Managing Deployments

The agent is supported for Kubernetes 1.9+ and Red Hat OpenShift 4.5+ environments.

Installing on Kubernetes

You can use the manifest YAML files in this repository or use Helm package manager to deploy the agent in your Kubernetes cluster.

Using manifest files

NOTE: The Kubernetes manifest YAML files in this repository (and referenced in this documentation) describe the version of the LogDNA Agent in the current commit tree and no other version of the LogDNA agent. If you apply the Kubernetes manifest found in the current tree then your cluster will be running the version described by the current commit which may not be ready for general use.

Follow the full instructions in the documentation to deploy on Kubernetes using resource files.

Using Helm

Visit the documentation to install the agent on Kubernetes using Helm.

Installing on OpenShift

Follow the instructions for deploying the agent on Red Hat OpenShift.

Installing on Linux

Refer to the documentation for deploying the agent on Linux.

Installing on Windows

Refer to the documentation for installing the agent on Windows.

Running as Non-Root

By default the agent will run as root. Below are environment-specific instructions for running the agent as a non-root user.

• Running as Non-Root on Kubernetes
• Running as Non-Root on OpenShift

If you configure the LogDNA Agent to run as non-root, review the documentation about enabling "statefulness" for the agent.

Additional Installation Options

More information about managing your deployments is documented for Kubernetes or OpenShift. This includes topics such as

• Version specific upgrade paths
• Collecting system logs through Journald

Installing on Windows

Refer to the documentation for installing the agent on Windows.

Building the LogDNA Agent

Obtain the source code to build the agent from our GitHub repository. You can either download the source files as a .zip file, or clone the repository.

• Use the following commands to clone and then cd into this repository:

git clone https://github.com/logdna/logdna-agent-v2.git
cd logdna-agent-v2

Next, select how you want to build the agent: as a docker image or as a Linux binary.

Building Docker image

To build a Docker image of the agent, ensure the Docker command is installed properly, verify that the Docker engine is running, and then run the following command:

make build-image

The resulting image can be found by listing the images:

foo@bar:~$ docker images
REPOSITORY              TAG                 IMAGE ID            CREATED             SIZE
logdna-agent-v2         dcd54a0             e471b3d8a409        22 seconds ago      135MB

You can also obtain the image and review our tagging scheme on DockerHub.

Building Agent Binary on Linux

The agent requires rustc cargo-make to build. If rustc and cargo are installed, simply run the following command to build the agent:

cargo build --release

The compiled binary will be built to ./target/release/logdna-agent.

Helper scripts

Some make helper scripts are located under ./scripts folder:

* mk                    make "build" target - build Agent from rust sources
mk.lint               make "lint" target - run linting
mk.test               make "test" target - run unit tests
mk.integrationtest   make "integrationtest" target - run integration tests
mk.image              make "image" target - create Agent container image and publish it in local docker

Note:

• all targets are using "rust-xxxx" container image from "logdna/build-images" registry, will try to use local image first.
• to build arm64 image use:

$ ARCH=aarch64 scripts/mk.image

• "multi-arch build" in docker needs to be installed and configured before building arm64 on x86 platforms

Configuration

Options

The agent accepts configuration from three different sources: environment variables, command line arguments and/or a configuration YAML file. The default configuration yaml file is located at /etc/logdna/config.yaml. Unless noted, when an option says List of, this would mean a comma separated list for Environment variable and a YAML array list for YAML.

Some variables do not have a corresponding Yaml path because they are only applicable to k8s deployments and therefor would use environment variables.

For backward compatibility agent v1 configuration file format is still supported. For example - upgrade to Agent v2 on Windows can re-use v1 config file from previous installation as-is. The format of these v1 configuration parameter names are the same as the environment variables without LOGDNA and in lower case._

| Environment Variable Name | Yaml Path Name | Description | Default | | ------------------------------------------------------------------------ | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | LOGDNAINGESTIONKEY
Deprecated: LOGDNAAGENTKEY | http.ingestion_key | Required: The ingestion key associated with your LogDNA account | | | LOGDNACONFIGFILE
Deprecated: DEFAULTCONFFILE | | Path to the configuration yaml | /etc/logdna/config.yaml | | LOGDNA_HOST
Deprecated: LDLOGHOST | http.host | The host to forward logs to | logs.logdna.com | | LOGDNA_ENDPOINT
Deprecated: LDLOGPATH | http.endpoint | The endpoint to forward logs to | /logs/agent | | LOGDNAUSESSL
Deprecated: LDLOGSSL | http.use_ssl | Whether to use a SSL for sending logs | true | | LOGDNAUSECOMPRESSION
Deprecated: COMPRESS | http.use_compression | Whether to compress logs before sending | true | | LOGDNAGZIPLEVEL
Deprecated: GZIPCOMPRESSLEVEL | http.gzip_level | If compression is enabled, this is the gzip compression level to use | 2 | | LOGDNA_HOSTNAME | http.params.hostname | The hostname metadata to attach to lines forwarded from this agent | | | LOGDNA_IP | http.params.ip | The IP metadata to attach to lines forwarded from this agent | | | LOGDNA_TAGS | http.params.tags | Comma separated list of tags metadata to attach to lines forwarded from this agent | | | LOGDNA_MAC | http.params.mac | The MAC metadata to attach to lines forwarded from this agent | | | LOGDNALOGDIRS
Deprecated: LOG_DIRS | log.dirs[] | List of folders to recursively monitor for log events | /var/log/ | | LOGDNAEXCLUSIONRULES
Deprecated: LOGDNA_EXCLUDE | log.exclude.glob[] | List of glob patterns to exclude files from monitoring ¹ | /var/log/wtmp,/var/log/btmp,/var/log/utmp,
/var/log/wtmpx,/var/log/btmpx,/var/log/utmpx,
/var/log/asl/,/var/log/sa/,/var/log/sar,
/var/log/tallylog,/var/log/fluentd-buffers//,
/var/log/pods//* | | LOGDNAEXCLUSIONREGEXRULES
Deprecated: LOGDNAEXCLUDE_REGEX | log.exclude.regex[] | List of regex patterns to exclude files from monitoring | | | LOGDNAINCLUSIONRULES
Deprecated: LOGDNA_INCLUDE | log.include.glob[] | List of glob patterns to includes files for monitoring ¹ | *.log | | LOGDNAINCLUSIONREGEXRULES
Deprecated: LOGDNAINCLUDE_REGEX | log.include.regex[] | List of regex patterns to include files from monitoring | | | LOGDNALINEEXCLUSIONREGEX | log.lineexclusion_regex[] | List of regex patterns to exclude log lines. When set, the Agent will NOT send log lines that match any of these patterns. | | | LOGDNALINEINCLUSIONREGEX | log.lineinclusion_regex[] | List of regex patterns to include log lines. When set, the Agent will send ONLY log lines that match any of these patterns. | | | LOGDNAREDACTREGEX | log.lineredactregex | List of regex patterns used to mask matching sensitive information (such as PII) before sending it in the log line. | | | LOGDNAJOURNALDPATHS | journald.paths[] | List of paths (directories or files) of journald paths to monitor | | | MZSYSTEMDJOURNAL_TAILER | | True/False toggles journald on the agent | 'true' | | LOGDNA_LOOKBACK | log.lookback | The lookback strategy on startup | none | | LOGDNAK8SSTARTUP_LEASE | | Determines whether or not to use K8 leases on startup | never | | LOGDNAUSEK8SLOGENRICHMENT | | Determines whether the agent should query the K8s API to enrich log lines from other pods. | always | | LOGDNALOGK8SEVENTS | | Determines whether the agent should log Kubernetes resource events. This setting only affects tracking and logging Kubernetes resource changes via watches. When disabled, the agent may still query k8s metadata to enrich log lines from other pods depending on the value of LOGDNAUSEK8SLOG_ENRICHMENT setting value. | never | | LOGDNALOGMETRICSERVERSTATS | | Determines whether or not metrics usage statistics is enabled. | never | | LOGDNADBPATH | log.db_path | The directory in which the agent will store its state database. Note that the agent must have write access to the directory and be a persistent volume. | /var/lib/logdna | | LOGDNAMETRICSPORT | log.metricsport | The port number to expose a Prometheus endpoint target with the agent internal metrics. | | | LOGDNAINGESTTIMEOUT | http.timeout | The timeout of the API calls to the ingest API in milliseconds | 10000 | | LOGDNAINGESTBUFFERSIZE | http.bodysize | The size, in bytes, of the ingest data buffer used to batch log data with. | 2097152 | | LOGDNARETRYDIR | http.retry_dir | The directory used by the agent to store data temporarily while retrying calls to the ingestion API. | /tmp/logdna | | LOGDNARETRYDISKLIMIT | http.retrydisk_limit | The maximum amount of disk space the agent will use to store retry data. The value can be the total number of bytes or a human representation of space using suffixes "KB", "MB", "GB" or "TB", e.g. 10 MB If left unset, the agent will not limit disk usage. If set to 0, no retry data will be stored on disk. | | | LOGDNAMETAAPP | | Overrides/omits APP field in log line metadata. Examples | | | LOGDNAMETAHOST | | Overrides/omits HOST field in log line metadata. | | | LOGDNAMETAENV | | Overrides/omits ENV field in log line metadata. | | | LOGDNAMETAFILE | | Overrides/omits FILE field in log line metadata. | | | LOGDNAMETAK8SFILE | | Overrides/omits FILE field in k8s log line metadata. Follows LOGDNAMETA_FILE. | | | LOGDNAMETAJSON | | Overrides/omits META filed in log line metadata. | | | LOGDNAMETAANNOTATIONS | | Overrides specific kay-value-pairs inside ANNOTATIONS field in log line metadata. | | | LOGDNAMETALABELS | | Overrides specific kay-value-pairs inside LABELS field in log line metadata. | | | MZMETADATARETRY_DELAY | | The number of seconds to wait for Pod label availability before sending log lines. | 0 | | MZFLUSHDURATION | http.flush_duration | The number of milliseconds to wait before flushing captured logs to the ingestion API. | 5000 |

| Environment Variable Name | Yaml Path Name | Description | Default | | ------------------------------------------------------------------------ | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | LOGDNAINGESTIONKEY
Deprecated: LOGDNAAGENTKEY | http.ingestion_key | Required: The ingestion key associated with your LogDNA account | | | LOGDNACONFIGFILE
Deprecated: DEFAULTCONFFILE | | Path to the configuration yaml | /etc/logdna/config.yaml | | LOGDNA_HOST
Deprecated: LDLOGHOST | http.host | The host to forward logs to | logs.logdna.com | | LOGDNA_ENDPOINT
Deprecated: LDLOGPATH | http.endpoint | The endpoint to forward logs to | /logs/agent | | LOGDNAUSESSL
Deprecated: LDLOGSSL | http.use_ssl | Whether to use a SSL for sending logs | true | | LOGDNAUSECOMPRESSION
Deprecated: COMPRESS | http.use_compression | Whether to compress logs before sending | true | | LOGDNAGZIPLEVEL
Deprecated: GZIPCOMPRESSLEVEL | http.gzip_level | If compression is enabled, this is the gzip compression level to use | 2 | | LOGDNA_HOSTNAME | http.params.hostname | The hostname metadata to attach to lines forwarded from this agent | | | LOGDNA_IP | http.params.ip | The IP metadata to attach to lines forwarded from this agent | | | LOGDNA_TAGS | http.params.tags | Comma separated list of tags metadata to attach to lines forwarded from this agent | | | LOGDNA_MAC | http.params.mac | The MAC metadata to attach to lines forwarded from this agent | | | LOGDNALOGDIRS
Deprecated: LOG_DIRS | log.dirs[] | List of folders to recursively monitor for log events | /var/log/ | | LOGDNAEXCLUSIONRULES
Deprecated: LOGDNA_EXCLUDE | log.exclude.glob[] | List of glob patterns to exclude files from monitoring ¹ | /var/log/wtmp,/var/log/btmp,/var/log/utmp,
/var/log/wtmpx,/var/log/btmpx,/var/log/utmpx,
/var/log/asl/,/var/log/sa/,/var/log/sar,
/var/log/tallylog,/var/log/fluentd-buffers//,
/var/log/pods//* | | LOGDNAEXCLUSIONREGEXRULES
Deprecated: LOGDNAEXCLUDE_REGEX | log.exclude.regex[] | List of regex patterns to exclude files from monitoring | | | LOGDNAINCLUSIONRULES
Deprecated: LOGDNA_INCLUDE | log.include.glob[] | List of glob patterns to includes files for monitoring ¹ | *.log | | LOGDNAINCLUSIONREGEXRULES
Deprecated: LOGDNAINCLUDE_REGEX | log.include.regex[] | List of regex patterns to include files from monitoring | | | LOGDNALINEEXCLUSIONREGEX | log.lineexclusion_regex[] | List of regex patterns to exclude log lines. When set, the Agent will NOT send log lines that match any of these patterns. | | | LOGDNALINEINCLUSIONREGEX | log.lineinclusion_regex[] | List of regex patterns to include log lines. When set, the Agent will send ONLY log lines that match any of these patterns. | | | LOGDNAREDACTREGEX | log.lineredactregex | List of regex patterns used to mask matching sensitive information (such as PII) before sending it in the log line. | | | LOGDNAJOURNALDPATHS | journald.paths[] | List of paths (directories or files) of journald paths to monitor | | | MZSYSTEMDJOURNAL_TAILER | | True/False toggles journald on the agent | 'true' | | LOGDNA_LOOKBACK | log.lookback | The lookback strategy on startup | none | | LOGDNAK8SSTARTUP_LEASE | | Determines whether or not to use K8 leases on startup | never | | LOGDNAUSEK8SLOGENRICHMENT | | Determines whether the agent should query the K8s API to enrich log lines from other pods. | always | | LOGDNALOGK8SEVENTS | | Determines whether the agent should log Kubernetes resource events. This setting only affects tracking and logging Kubernetes resource changes via watches. When disabled, the agent may still query k8s metadata to enrich log lines from other pods depending on the value of LOGDNAUSEK8SLOG_ENRICHMENT setting value. | never | | LOGDNALOGMETRICSERVERSTATS | | Determines whether or not metrics usage statistics is enabled. | never | | LOGDNADBPATH | log.db_path | The directory in which the agent will store its state database. Note that the agent must have write access to the directory and be a persistent volume. | /var/lib/logdna | | LOGDNAMETRICSPORT | log.metricsport | The port number to expose a Prometheus endpoint target with the agent internal metrics. | | | LOGDNAINGESTTIMEOUT | http.timeout | The timeout of the API calls to the ingest API in milliseconds | 10000 | | LOGDNAINGESTBUFFERSIZE | http.bodysize | The size, in bytes, of the ingest data buffer used to batch log data with. | 2097152 | | LOGDNARETRYDIR | http.retry_dir | The directory used by the agent to store data temporarily while retrying calls to the ingestion API. | /tmp/logdna | | LOGDNARETRYDISKLIMIT | http.retrydisk_limit | The maximum amount of disk space the agent will use to store retry data. The value can be the total number of bytes or a human representation of space using suffixes "KB", "MB", "GB" or "TB", e.g. 10 MB If left unset, the agent will not limit disk usage. If set to 0, no retry data will be stored on disk. | | | LOGDNAMETAAPP | | Overrides/omits APP field in log line metadata. Examples |

---

README truncated. View on GitHub