Skip to main content
Version: 6.x (Latest)

Build Images with kaniko

Using kaniko as build tool allows you to build images directly inside your Kubernetes cluster without a Docker daemon. DevSpace simply starts a build pod and builds the image using kaniko.

To set kaniko as default build tool, use the following configuration:

images:
  backend:
    image: john/appbackend
    kaniko: {}
Automatic Cleanup

After the build process completes, the build pod started for the kaniko build process will be deleted again.

Flags & Arguments​

cache​

The cache option expects a boolean that states if kaniko should make use of layer caching by pulling the previously build image and using the layers of this image as cache.

Default Value For cache​

cache: true

Example: Building Images with kaniko​

images:
  backend:
    image: john/appbackend
    kaniko:
      cache: true
  frontend:
    image: john/appfrontend
    kaniko:
      cache: false

Explanation:

  • The first image backend is built using kaniko and makes use of the build cache.
  • The second image frontend is built using kaniko and does not use the build cache.

snapshotMode​

The snapshotMode option expects a string that can have the following values:

  • full tells kaniko to do a full filesystem snapshot
  • time tells kaniko to do a filesystem snapshot based on mtime (default)
Limitations

One of the biggest limitations of the time mode is that kaniko might miss file metadata changes introduced by RUN statements, e.g. kaniko might skip a command such as RUN chown 333:333 -R /app. In cases where this is not an option, use full instead. Learn more about limitations related to kaniko snapshots using mtime.

Default Value For snapshotMode​

snapshotMode: time

Example: Building Images with kaniko​

images:
  backend:
    image: john/appbackend
    kaniko:
      snapshotMode: full
  frontend:
    image: john/appfrontend
    kaniko:
      snapshotMode: time

Explanation:

  • The first image backend is built using kaniko and creates full filesystem snapshots.
  • The second image frontend is built using kaniko and calculates filesystem snapshots only based on mtime.

insecure​

The insecure option expects a boolean stating if kaniko should allow to push to an insecure (plain HTTP instead of HTTPS) registry.

danger

This option should only be set to true for testing purposes.

Default Value For insecure​

insecure: false

Example: Push to Insecure Registry With kaniko​

images:
  backend:
    image: 123.456.789.0:5000/john/appbackend
    kaniko:
      insecure: false

Explanation: The image backend is built using kaniko and pushes to the insecure registry 123.456.789.0:5000 if allowed.

args​

The args option expects an array of strings that will be passed as arguments (and flags) when running the kaniko build command.

Kaniko Documentation

Take a look at the kaniko documentation for a full list of available flags.

Default Value For args​

args: []

Example: Passing Args to kaniko​

images:
  backend:
    image: john/appbackend
    kaniko:
      args:
      - --cache-dir=/tmp
      - --verbosity=debug

Explanation: The image backend is built using kaniko and the flags --cache-dir=/tmp --verbosity=debug are set when running the build command within the kaniko pod used for image building.

command​

The command option expects an array of strings that will replace the starting command for the kaniko build container

Default Value For command​

command: []

Example: Replacing the command for the kaniko build container​

images:
  backend:
    image: john/appbackend
    kaniko:
      command:
      - kaniko-custom

Build Pod Configuration​

generateName​

The generateName option expects a string that will be used as the generateName field of the build pod

Default value for generateName​

generateName: devspace-build-kaniko-

Example: Add generateName to kaniko build pods​

images:
  backend:
    image: 123.456.789.0:5000/john/appbackend
    kaniko:
      generateName: devspace-build-kaniko-backend-

annotations​

The annotations option expects a key/value map of extra annotations that will be added to the build pod

Default value for annotations​

annotations: {}

Example: Add annotations to kaniko build pods​

images:
  backend:
    image: 123.456.789.0:5000/john/appbackend
    kaniko:
      annotations:
        app: kaniko

labels​

The labels option expects a key/value map of extra labels that will be added to the build pod

Default value for labels​

labels: {}

Example: Add labels to kaniko build pods​

images:
  backend:
    image: 123.456.789.0:5000/john/appbackend
    kaniko:
      labels:
        app: kaniko

image​

The image option expects a string stating the image that should be used for the kaniko container within the build pod.

Default Value For image​

image: gcr.io/kaniko-project/executor:v1.8.1

Example: Use Different Kaniko Image/Version​

images:
  backend:
    image: 123.456.789.0:5000/john/appbackend
    kaniko:
      image: gcr.io/kaniko-project/executor:v1.6.0

env​

The env option provides extra environment variables that will be added to the build kaniko container.

Default Value For env​

env: {}

Example: Specify environment variables for the kaniko container​

images:
  backend:
    image: 123.456.789.0:5000/john/appbackend
    kaniko:
      env:
        AZURE_CLIENT_ID: XXXXXXXX

envFrom​

The envFrom option provides extra environment variables from configmap or secret that will be added to the build kaniko container.

Default Value For envFrom​

envFrom: {}

Example: Specify environment variables for the kaniko container using a secret​

images:
  backend:
    image: 123.456.789.0:5000/john/appbackend
    kaniko:
      envFrom:
        AZURE_CLIENT_ID:
          secretMapKeyRef:
            name: azure-creds
            key: AZURE_CLIENT_ID

initImage​

The initImage option expects a string stating the image that should be used for the init kaniko container within the build pod. (default is alpine)

initEnv​

The initEnv option provides extra environment variables that will be added to the build init container.

Default Value For initEnv​

initEnv: {}

Example: Specify environment variables for the kaniko container​

images:
  backend:
    image: 123.456.789.0:5000/john/appbackend
    kaniko:
      initImage: alpine
      initEnv:
        AZURE_CLIENT_ID: XXXXXXXX

pullSecret​

The pullSecret option expects a string with the name of a Kubernetes secret which is used by kaniko as pull/push secret (e.g. for pulling the base image defined in the FROM statement of the Dockerfile and for pushing the newly built image after the kaniko build process).

info

In most cases, DevSpace already makes sure that kaniko gets the correct pull secrets to push and pull to registries.

Default Value For pullSecret​

pullSecret: ""

Example: Pull Secret For kaniko​

images:
  backend:
    image: john/appbackend
    kaniko:
      pullSecret: custom-pull-secret

Explanation: The image backend is built using kaniko. Kaniko uses the Kubernete secret, custom-pull-secret, to pull images from registries that require authentication.

Creating Pull Secrets using pullSecrets

If you use the pullSecret option, you must create the secret before DevSpace tries to build your images. You can automate this by using the pullSecrets feature as shown in this example:

pullSecrets:
- registry: my-registry.com:5000
  username: ${REGISTRY_USER}
  password: ${REGISTRY_PASSWORD}
  secret: custom-pull-secret
Building AWS ECR images on EKS

If your EKS instance is configured with access to ECR (see instance role permissions for AWS EKS and ECR), your pull secret referenced by the pullSecret option, could be created with this kubectl command:

kubectl create secret generic custom-pull-secret --type=kubernetes.io/dockerconfigjson --from-literal .dockerconfigjson='{ "credsStore":"ecr-login" }'

The resulting Kubernetes secret would look like this:

apiVersion: v1
kind: Secret
type: kubernetes.io/dockerconfigjson
metadata:
  name: custom-pull-secret
data:
  .dockerconfigjson: eyAiY3JlZHNTdG9yZSI6ImVjci1sb2dpbiIgfQ==

If you define pullSecret: custom-pull-secret as shown in the example above, DevSpace will automatically mount this secret into the kaniko container and kaniko will be able to pull from and push to ECR.

skipPullSecretMount​

The skipPullSecretMount option expects a boolean and if true will skip mounting the pull secret to the kaniko build container.

Default Value For skipPullSecretMount​

skipPullSecretMount: false

additionalMounts​

The additionalMounts option expects an array of mount options that allow to mount Kubernetes Secrets and ConfigMaps into the kaniko container within the build pod.

Default Value For additionalMounts​

additionalMounts: []

Example: Mount ConfigMaps & Secrets For Kaniko​

images:
  backend:
    image: john/appbackend
    kaniko:
      additionalMounts:
      - mountPath: /some/configmap/dir
        configMap:
          name: my-configmap
      - mountPath: /some/secret/dir
        secret:
          name: my-secret
          items:
          - key: aws-token
            path: token.json

Explanation: The above configuration creates a kaniko container which mounts the following volumes:

  • All keys within the ConfigMap my-configmap will be mounted as files within the folder /some/configmap/dir. The filenames will be the keys within the ConfigMap.
  • The key aws-token within the Secret my-secret will be mounted as the file /some/secret/dir/token.json.

resources​

The resources option expects a Kubernetes resource object, so that the kaniko pod can specify requests and limits for resources such as memory and cpu.

Default Value For resources​

resources: {}

Example: Resource Limits For Kaniko​

images:
  backend:
    image: john/appbackend
    kaniko:
      resources:
        limits:
          memory: "256Mi"
          cpu: "500m"

namespace​

The namespace option expects a string stating a namespace that should be used to start the kaniko build pod in.

Hard-Coding Namespaces Discouraged

Unless you really know what you are doing, it is discouraged to hard-code namespaces within devspace.yaml because that makes it harder to share the project and its configuration with others.

Default Value For namespace​

namespace: "" # defaults to the default namespace of the current kube-context

Example: Different Namespace For kaniko​

images:
  backend:
    image: john/appbackend
    kaniko:
      namespace: build-namespace

Explanation: The image backend is built using kaniko. The build pod starts to run the kaniko build process and creates the namespace build-namespace within the cluster that the current kube-context points to.

serviceAccount​

The service account to use for the build pod.

nodeSelector​

A key value map of the node selector to use for the build pod.

tolerations​

Array of node Tolerations that should be used to schedule the build pod.


Config Reference​

kaniko required ​

Kaniko if kaniko is specified, DevSpace will build the image in-cluster with kaniko

cache required boolean false ​

Cache tells DevSpace if a cache repository should be used. defaults to false

snapshotMode required string ​

SnapshotMode tells DevSpace which snapshot mode kaniko should use. defaults to time

image required string ​

Image is the image name of the kaniko pod to use

initImage required string ​

InitImage to override the init image of the kaniko pod

args required string[] ​

Args for additional arguments that should be passed to kaniko

command required string[] ​

Command to replace the starting command for the kaniko container

namespace required string ​

Namespace is the namespace where the kaniko pod should be run

insecure required boolean false ​

Insecure allows pushing to insecure registries

pullSecret required string ​

PullSecret is the pull secret to mount by default

skipPullSecretMount required boolean false ​

SkipPullSecretMount will skip mounting the pull secret

nodeSelector required <nodeSelector_name>:string ​

NodeSelector is the node selector to use for the kaniko pod

tolerations required object[] ​

Tolerations is a tolerations list to use for the kaniko pod

Key required string ​
Operator required string ​
Value required string ​
Effect required string ​
TolerationSeconds required integer ​

serviceAccount required string ​

ServiceAccount the service account to use for the kaniko pod

generateName required string ​

GenerateName is the optional prefix that will be set to the generateName field of the build pod

annotations required <annotation_name>:string ​

Annotations are extra annotations that will be added to the build pod

labels required <label_name>:string ​

Labels are extra labels that will be added to the build pod

initEnv required <initEnv_name>:string ​

InitEnv are extra environment variables that will be added to the build init container

env required <env_name>:string ​

Env are extra environment variables that will be added to the build kaniko container Will populate the env.value field.

envFrom required <envFrom_name>:object ​

EnvFrom are extra environment variables from configmap or secret that will be added to the build kaniko container Will populate the env.valueFrom field.

additionalMounts required object[] ​

AdditionalMounts are additional mounts that will be added to the build pod

secret required ​

The secret that should be mounted

name required string ​

Name of the secret in the pod's namespace to use. More info: https://kubernetes.io/docs/concepts/storage/volumes#secret +optional

items required object[] ​

If unspecified, each key-value pair in the Data field of the referenced Secret will be projected into the volume as a file whose name is the key and content is the value. If specified, the listed keys will be projected into the specified paths, and unlisted keys will not be present. If a key is specified which is not present in the Secret, the volume setup will error unless it is marked optional. Paths must be relative and may not contain the '..' path or start with '..'. +optional

####### key required string {#images-kaniko-additionalMounts-secret-items-key}

The key to project.

####### path required string {#images-kaniko-additionalMounts-secret-items-path}

The relative path of the file to map the key to. May not be an absolute path. May not contain the path element '..'. May not start with the string '..'.

####### mode required integer {#images-kaniko-additionalMounts-secret-items-mode}

Optional: mode bits to use on this file, must be a value between 0 and 0777. If not specified, the volume defaultMode will be used. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set. +optional

defaultMode required integer ​

Optional: mode bits to use on created files by default. Must be a value between 0 and 0777. Defaults to 0644. Directories within the path are not affected by this setting. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set. +optional

configMap required ​

The configMap that should be mounted

name required string ​

Name of the configmap +optional

items required object[] ​

If unspecified, each key-value pair in the Data field of the referenced ConfigMap will be projected into the volume as a file whose name is the key and content is the value. If specified, the listed keys will be projected into the specified paths, and unlisted keys will not be present. If a key is specified which is not present in the ConfigMap, the volume setup will error unless it is marked optional. Paths must be relative and may not contain the '..' path or start with '..'. +optional

####### key required string {#images-kaniko-additionalMounts-configMap-items-key}

The key to project.

####### path required string {#images-kaniko-additionalMounts-configMap-items-path}

The relative path of the file to map the key to. May not be an absolute path. May not contain the path element '..'. May not start with the string '..'.

####### mode required integer {#images-kaniko-additionalMounts-configMap-items-mode}

Optional: mode bits to use on this file, must be a value between 0 and 0777. If not specified, the volume defaultMode will be used. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set. +optional

defaultMode required integer ​

Optional: mode bits to use on created files by default. Must be a value between 0 and 0777. Defaults to 0644. Directories within the path are not affected by this setting. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set. +optional

readOnly required boolean false ​

Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false. +optional

mountPath required string ​

Path within the container at which the volume should be mounted. Must not contain ':'.

subPath required string ​

Path within the volume from which the container's volume should be mounted. Defaults to "" (volume's root). +optional

resources required ​

Resources are the resources that should be set on the kaniko pod

requests required <request_name>:string ​

Requests are the requests part of the resources

limits required <limit_name>:string ​

Limits are the limits part of the resources