How to Use Datree to Avoid Kubernetes Misconfigurations

Kubernetes is a complex system with many moving parts. Correct configuration rules are essential for your service to operate reliably. Errors can occur when you write Kubernetes manifests by hand without a comprehensive review process.

Datree is a rule-based tool that automatically finds problems in your manifests. You can use it to uncover policy violations without leaving your terminal, enabling a consistent approach to Kubernetes configuration.

In this article, you’ll learn how to use Datree’s CLI to perform on-demand manifest scans. The tool is free and open-source but backed by an online dashboard that lets you centrally manage policies shared by your entire team. This is free for individuals interacting with up to two Nodes while team plans start at $95/mo with a base allowance of five Nodes.

Installing the Datree CLI

First download and set up the Datree CLI using its installation script. This works on Linux and Mac:

$ curl https://get.datree.io | /bin/bash

Alternative installation instructions are available in the documentation if you’re using Windows or want to run Datree as a Docker container.

Check the CLI’s installed correctly by running the datree command without any arguments:

$ datree
Datree is a static code analysis tool for kubernetes files. Full code can be found at https://github.com/datreeio/datree
...

Now you can begin scanning your manifests for errors.

Performing a Policy Check

Copy the following YAML and save it as datree-demo.yaml in your working directory:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-deployment
  namespace: demo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: demo-app
  template:
    metadata:
      namespace: demo-deployment
      labels:
        app: demo-app
    spec:
      containers:
        - name: nginx
          image: nginx:latest
          readinessProbe:
            tcpSocket:
              port: 8080
          resources:
            requests:
              memory: "256Mi"
              cpu: "100m"
            limits:
              cpu: "500m"
          ports:
            - containerPort: 80

This YAML defines a valid Kubernetes Deployment object. Kubectl will apply it to your cluster without reporting any errors:

$ kubectl apply -f datree-demo.yaml
deployment/demo-deployment created

There could be problems with this configuration though. Running the Datree CLI will expose them. Use the datree test command to complete an analysis of your manifest:

$ datree test datree-demo.yaml
>> File: datree-demo.yaml

[V] YAML validation
[V] Kubernetes schema validation

[X] Policy check

❌  Ensure each container image has a pinned (tag) version  [1 occurrence]
    - metadata.name: demo-deployment (kind: Deployment)
💡  Incorrect value for key `image` - specify an image version to avoid unpleasant "version surprises" in the future

❌  Ensure each container has a configured liveness probe  [1 occurrence]
    - metadata.name: demo-deployment (kind: Deployment)
💡  Missing property object `livenessProbe` - add a properly configured livenessProbe to catch possible deadlocks

❌  Ensure each container has a configured memory limit  [1 occurrence]
    - metadata.name: demo-deployment (kind: Deployment)
💡  Missing property object `limits.memory` - value should be within the accepted boundaries recommended by the organization

(Summary)

- Passing YAML validation: 1/1

- Passing Kubernetes (1.20.0) schema validation: 1/1

- Passing policy check: 0/1

+-----------------------------------+-----+
| Enabled rules in policy "Default" | 21  |
| Configs tested against policy     | 1   |
| Total rules evaluated             | 21  |
| Total rules skipped               | 0   |
| Total rules failed                | 3   |
| Total rules passed                | 18  |
+-----------------------------------+-----+

Datree has uncovered three policy violations that could affect your cluster.

Interpreting Scan Results

Datree scans look at three aspects of each manifest:

• YAML validation – The first check validates your YAML for correctness. No further checks are run if your YAML file has syntax errors.
• Kubernetes schema validation – Checks whether the manifest contains a legal Kubernetes object. Common causes of these errors include invalid field values and incorrect object nesting.
• Policy checks – This is where Datree tests a valid Kubernetes object schema against common misconfigurations. Policies identify potential issues and missing optimizations so you can make your Kubernetes cluster more resilient.

Each report ends with a table that summarizes the number of manifests scanned, rules used, and failures detected.

Fixing the Example Manifest’s Errors

Scanning the example manifest surfaces three errors: the container’s image field isn’t using a pinned tag, there’s no livenessProbe, and no memory limit.

The first problem can be resolved by using an explicit image version such as nginx:1.23. The latest tag is risky because you could unintentionally receive breaking changes, such as 1.23 to 2.1.

The next error can be eliminated by adding a liveness probe. These allow Kubernetes to detect when your containers transition into a failed state. The control plane will automatically restart the container, reducing the probability of a service outage.

Add a new livenessProbe field above the readinessProbe:

livenessProbe:
  tcpSocket:
    port: 8080

Finally set a memory limit to address the last warning. Although the example manifest includes CPU and memory requests, as well as a CPU limit, there’s no hard cap on memory. The container could consume unlimited RAM, potentially creating an out of memory situation in your cluster.

The revised YAML should look like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-deployment
  namespace: demo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: demo-app
  template:
    metadata:
      namespace: demo-deployment
      labels:
        app: demo-app
    spec:
      containers:
        - name: nginx
          image: nginx:1.23  
          livenessProbe:
            tcpSocket:
              port: 8080
          readinessProbe:
            tcpSocket:
              port: 8080
          resources:
            requests:
              memory: "256Mi"
              cpu: "100m"
            limits:
              memory: "512Mi"
              cpu: "500m"
          ports:
            - containerPort: 80

Repeat the datree test command to verify that your deployment’s now passing the policy checks:

$ datree test datree-demo.yaml
(Summary)

- Passing YAML validation: 1/1

- Passing Kubernetes (1.20.0) schema validation: 1/1

- Passing policy check: 1/1

Customizing Rules

The example used so far relies on Datree’s built-in set of default policies. These cover many Kubernetes best practices, such as setting up probes, using resource limits, and avoiding deprecated APIs.

You can customize the policies by linking the Datree CLI to your online dashboard. Here you can disable policies that you don’t need and activate new custom rules to implement your organization’s routines.

The easiest way to sign in to Datree is to follow the link shown at the end of the Datree CLI’s output:

| See all rules in policy           | https://app.datree.io/login?t=bbY... |

The CLI automatically generates a unique token for your account. Click the link and then sign in to Datree with GitHub or Google.

You’ll be taken to the Policies dashboard that shows all the policies active on your account. Click the toggle buttons in the “Status” column to enable or remove policies. Your changes will immediately apply to new scans. The CLI automatically downloads your policy list before starting each test.

The dashboard also provides a history of the scans you’ve completed. Click the “History” tab in the left sidebar to retrieve previous scan results.

Scanning With a Specific Policy

Datree currently has 60 built-in rules which provide individual tests. Rules are combined into groups called policies. The Default policy is used automatically. It enables 21 of the 60 rules. Datree also comes with pre-configured policies for Argo and NSA Kubernetes configurations.

You can create your own policy with the blue “Create Policy” button in the online dashboard. Give your policy a name and enable one or more rules.

To start a scan with a specific policy, add the --policy CLI flag. This should be supplied the name of the policy you want to use.

$ datree test --policy NSA datree-demo.yaml

Datree also lets you implement custom tests by adding entirely new rules. While rule creation is outside the scope of this getting started guide, you could enforce that Deployments have specific labels, a minimum replica count, and use images from an approved registry. Rules are defined as JSON or YAML using JSON Schema logic.

Scanning Multiple Files

The datree test command accepts a file path or a glob pattern. You can scan a directory of manifests using the following syntax:

datree test demo-dir/*.yaml

Any invalid files matched by your glob will show as failing the Datree YAML validation check.

Authenticating Other CLI Instances

The Datree CLI connects to your account using an authentication token. A new token is generated automatically when you install the CLI. It sets up a fresh account the first time it’s used.

You’ll need to manually provide your existing authentication token if you install Datree on another machine. You can retrieve the value from the token field in your ~/.datree/config.yaml file. Alternatively, head to the online dashboard, click your profile picture in the top-right corner, choose Settings from the menu, and switch to the “Token Management” tab.

Back in your new CLI instance, use the following command to add your token:

$ datree config set token <TOKEN_VALUE>

The CLI will now use the policies configured in your account. Scans will start showing up on the History screen too.

Using Datree Without Account Access

You can disable the Datree CLI’s account connection features if you’re satisfied with the default rule set and don’t want scans to communicate with Datree’s servers:

$ datree config set offline local

This also removes support for Kubernetes schema validation.

You can stop individual scans from showing up in your account’s History page by setting the --no-record flag in the CLI.

$ datree test datree-demo.yaml --no-record

Summary

Datree automates detection of Kubernetes config errors by offering a simple CLI that’s centrally configured by an online dashboard. This ensures everyone in your team is testing their manifests against the same policies, reducing the risk that mistakes will reach your cluster. You can integrate Datree into your CI pipelines to prevent deployment of changes that contain a rule violation.

Datree’s also available as a Kubernetes admission webhook that will actively block non-compliant resources. Admission webhooks are responsible for deciding whether new objects can be added to a cluster; Datree will reject any objects that fail your policy tests. Setting up the webhook provides absolute confidence that misconfigured resources can’t be used, even if a user manually applies a manifest with Kubectl.