This page shows how to install a custom resource into the Kubernetes API by creating a CustomResourceDefinition.
You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using Minikube, or you can use one of these Kubernetes playgrounds:
To check the version, enter kubectl version
.
Make sure your Kubernetes cluster has a master version of 1.7.0 or higher.
Read about custom resources.
When you create a new CustomResourceDefinition (CRD), the Kubernetes API Server
reacts by creating a new RESTful resource path, either namespaced or cluster-scoped,
as specified in the CRD’s scope
field. As with existing built-in objects, deleting a
namespace deletes all custom objects in that namespace.
CustomResourceDefinitions themselves are non-namespaced and are available to all namespaces.
For example, if you save the following CustomResourceDefinition to resourcedefinition.yaml
:
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
# name must match the spec fields below, and be in the form: <plural>.<group>
name: crontabs.stable.example.com
spec:
# group name to use for REST API: /apis/<group>/<version>
group: stable.example.com
# version name to use for REST API: /apis/<group>/<version>
version: v1
# either Namespaced or Cluster
scope: Namespaced
names:
# plural name to be used in the URL: /apis/<group>/<version>/<plural>
plural: crontabs
# singular name to be used as an alias on the CLI and for display
singular: crontab
# kind is normally the CamelCased singular type. Your resource manifests use this.
kind: CronTab
# shortNames allow shorter string to match your resource on the CLI
shortNames:
- ct
And create it:
kubectl create -f resourcedefinition.yaml
Then a new namespaced RESTful API endpoint is created at:
/apis/stable.example.com/v1/namespaces/*/crontabs/...
This endpoint URL can then be used to create and manage custom objects.
The kind
of these objects will be CronTab
from the spec of the
CustomResourceDefinition object you created above.
Please note that it might take a few seconds for the endpoint to be created.
You can watch the Established
condition of your CustomResourceDefinition
to be true or watch the discovery information of the API server for your
resource to show up.
After the CustomResourceDefinition object has been created, you can create
custom objects. Custom objects can contain custom fields. These fields can
contain arbitrary JSON.
In the following example, the cronSpec
and image
custom fields are set in a
custom object of kind CronTab
. The kind CronTab
comes from the spec of the
CustomResourceDefinition object you created above.
If you save the following YAML to my-crontab.yaml
:
apiVersion: "stable.example.com/v1"
kind: CronTab
metadata:
name: my-new-cron-object
spec:
cronSpec: "* * * * */5"
image: my-awesome-cron-image
and create it:
kubectl create -f my-crontab.yaml
You can then manage your CronTab objects using kubectl. For example:
kubectl get crontab
Should print a list like this:
NAME AGE
my-new-cron-object 6s
Note that resource names are not case-sensitive when using kubectl, and you can use either the singular or plural forms defined in the CRD, as well as any short names.
You can also view the raw YAML data:
kubectl get ct -o yaml
You should see that it contains the custom cronSpec
and image
fields
from the yaml you used to create it:
apiVersion: v1
items:
- apiVersion: stable.example.com/v1
kind: CronTab
metadata:
clusterName: ""
creationTimestamp: 2017-05-31T12:56:35Z
deletionGracePeriodSeconds: null
deletionTimestamp: null
name: my-new-cron-object
namespace: default
resourceVersion: "285"
selfLink: /apis/stable.example.com/v1/namespaces/default/crontabs/my-new-cron-object
uid: 9423255b-4600-11e7-af6a-28d2447dc82b
spec:
cronSpec: '* * * * */5'
image: my-awesome-cron-image
kind: List
metadata:
resourceVersion: ""
selfLink: ""
When you delete a CustomResourceDefinition, the server will uninstall the RESTful API endpoint and delete all custom objects stored in it.
kubectl delete -f resourcedefinition.yaml
kubectl get crontabs
Error from server (NotFound): Unable to list "crontabs": the server could not find the requested resource (get crontabs.stable.example.com)
If you later recreate the same CustomResourceDefinition, it will start out empty.
Finalizers allow controllers to implement asynchronous pre-delete hooks. Custom objects support finalizers just like built-in objects.
You can add a finalizer to a custom object like this:
apiVersion: "stable.example.com/v1"
kind: CronTab
metadata:
finalizers:
- finalizer.stable.example.com
Finalizers are arbitrary string values, that when present ensure that a hard delete of a resource is not possible while they exist.
The first delete request on an object with finalizers merely sets a value for the
metadata.deletionTimestamp
field instead of deleting it. Once this value is set,
entries in the finalizer
list can only be removed.
This triggers controllers watching the object to execute any finalizers they handle. This will be represented via polling update requests for that object, until all finalizers have been removed and the resource is deleted.
The time period of polling update can be controlled by metadata.deletionGracePeriodSeconds
.
It is the responsibility of each controller to removes its finalizer from the list.
Kubernetes will only finally delete the object if the list of finalizers is empty, meaning all finalizers are done.
Validation of custom objects is possible via OpenAPI v3 schema. Additionally, the following restrictions are applied to the schema:
default
, nullable
, discriminator
, readOnly
, writeOnly
, xml
,
deprecated
and $ref
cannot be set.uniqueItems
cannot be set to true.additionalProperties
cannot be set to false.This feature is beta in v1.9.
You can disable this feature using the CustomResourceValidation
feature gate on
the kube-apiserver:
--feature-gates=CustomResourceValidation=false
The schema is defined in the CustomResourceDefinition. In the following example, the CustomResourceDefinition applies the following validations on the custom object:
spec.cronSpec
must be a string and must be of the form described by the regular expression.spec.replicas
must be an integer and must have a minimum value of 1 and a maximum value of 10.Save the CustomResourceDefinition to resourcedefinition.yaml
:
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: crontabs.stable.example.com
spec:
group: stable.example.com
version: v1
scope: Namespaced
names:
plural: crontabs
singular: crontab
kind: CronTab
shortNames:
- ct
validation:
# openAPIV3Schema is the schema for validating custom objects.
openAPIV3Schema:
properties:
spec:
properties:
cronSpec:
type: string
pattern: '^(\d+|\*)(/\d+)?(\s+(\d+|\*)(/\d+)?){4}$'
replicas:
type: integer
minimum: 1
maximum: 10
And create it:
kubectl create -f resourcedefinition.yaml
A request to create a custom object of kind CronTab
will be rejected if there are invalid values in its fields.
In the following example, the custom object contains fields with invalid values:
spec.cronSpec
does not match the regular expression.spec.replicas
is greater than 10.If you save the following YAML to my-crontab.yaml
:
apiVersion: "stable.example.com/v1"
kind: CronTab
metadata:
name: my-new-cron-object
spec:
cronSpec: "* * * *"
image: my-awesome-cron-image
replicas: 15
and create it:
kubectl create -f my-crontab.yaml
you will get an error:
The CronTab "my-new-cron-object" is invalid: []: Invalid value: map[string]interface {}{"apiVersion":"stable.example.com/v1", "kind":"CronTab", "metadata":map[string]interface {}{"name":"my-new-cron-object", "namespace":"default", "deletionTimestamp":interface {}(nil), "deletionGracePeriodSeconds":(*int64)(nil), "creationTimestamp":"2017-09-05T05:20:07Z", "uid":"e14d79e7-91f9-11e7-a598-f0761cb232d1", "selfLink":"", "clusterName":""}, "spec":map[string]interface {}{"cronSpec":"* * * *", "image":"my-awesome-cron-image", "replicas":15}}:
validation failure list:
spec.cronSpec in body should match '^(\d+|\*)(/\d+)?(\s+(\d+|\*)(/\d+)?){4}$'
spec.replicas in body should be less than or equal to 10
If the fields contain valid values, the object creation request is accepted.
Save the following YAML to my-crontab.yaml
:
apiVersion: "stable.example.com/v1"
kind: CronTab
metadata:
name: my-new-cron-object
spec:
cronSpec: "* * * * */5"
image: my-awesome-cron-image
replicas: 5
And create it:
kubectl create -f my-crontab.yaml
crontab "my-new-cron-object" created
Custom resources support /status
and /scale
subresources.
This feature is alpha in v1.10 and may change in backward incompatible ways.
Enable this feature using the CustomResourceSubresources
feature gate on
the kube-apiserver:
--feature-gates=CustomResourceSubresources=true
When the CustomResourceSubresources
feature gate is enabled, only the properties
construct
is allowed in the root schema for custom resource validation.
The status and scale subresources can be optionally enabled by defining them in the CustomResourceDefinition.
When the status subresource is enabled, the /status
subresource for the custom resource is exposed.
.status
and .spec
JSONPaths respectively inside of a custom resource.PUT
requests to the /status
subresource take a custom resource object and ignore changes to anything except the status stanza.PUT
requests to the /status
subresource only validate the status stanza of the custom resource.PUT
/POST
/PATCH
requests to the custom resource ignore changes to the status stanza..metadata.generation
.When the scale subresource is enabled, the /scale
subresource for the custom resource is exposed.
The autoscaling/v1.Scale
object is sent as the payload for /scale
.
To enable the scale subresource, the following values are defined in the CustomResourceDefinition.
SpecReplicasPath
defines the JSONPath inside of a custom resource that corresponds to Scale.Spec.Replicas
.
.spec
and with the dot notation are allowed.SpecReplicasPath
in the custom resource,
the /scale
subresource will return an error on GET.StatusReplicasPath
defines the JSONPath inside of a custom resource that corresponds to Scale.Status.Replicas
.
.status
and with the dotation are allowed.StatusReplicasPath
in the custom resource,
the status replica value in the /scale
subresource will default to 0.LabelSelectorPath
defines the JSONPath inside of a custom resource that corresponds to Scale.Status.Selector
.
.status
and with the dotation are allowed.LabelSelectorPath
in the custom resource,
the status selector value in the /scale
subresource will default to the empty string.In the following example, both status and scale subresources are enabled.
Save the CustomResourceDefinition to resourcedefinition.yaml
:
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: crontabs.stable.example.com
spec:
group: stable.example.com
version: v1
scope: Namespaced
names:
plural: crontabs
singular: crontab
kind: CronTab
shortNames:
- ct
# subresources describes the subresources for custom resources.
subresources:
# status enables the status subresource.
status: {}
# scale enables the scale subresource.
scale:
# specReplicasPath defines the JSONPath inside of a custom resource that corresponds to Scale.Spec.Replicas.
specReplicasPath: .spec.replicas
# statusReplicasPath defines the JSONPath inside of a custom resource that corresponds to Scale.Status.Replicas.
statusReplicasPath: .status.replicas
# labelSelectorPath defines the JSONPath inside of a custom resource that corresponds to Scale.Status.Selector.
labelSelectorPath: .status.labelSelector
And create it:
kubectl create -f resourcedefinition.yaml
After the CustomResourceDefinition object has been created, you can create custom objects.
If you save the following YAML to my-crontab.yaml
:
apiVersion: "stable.example.com/v1"
kind: CronTab
metadata:
name: my-new-cron-object
spec:
cronSpec: "* * * * */5"
image: my-awesome-cron-image
replicas: 3
and create it:
kubectl create -f my-crontab.yaml
Then new namespaced RESTful API endpoints are created at:
/apis/stable.example.com/v1/namespaces/*/crontabs/status
and
/apis/stable.example.com/v1/namespaces/*/crontabs/scale
A custom resource can be scaled using the kubectl scale
command.
For example, the following command sets .spec.replicas
of the
custom resource created above to 5:
kubectl scale --replicas=5 crontabs/my-new-cron-object
crontabs "my-new-cron-object" scaled
kubectl get crontabs my-new-cron-object -o jsonpath='{.spec.replicas}'
5
Categories is a list of grouped resources the custom resource belongs to (eg. all
).
You can use kubectl get <category-name>
to list the resources belonging to the category.
This feature is beta and available for custom resources from v1.10.
The following example adds all
in the list of categories in the CustomResourceDefinition
and illustrates how to output the custom resource using kubectl get all
.
Save the following CustomResourceDefinition to resourcedefinition.yaml
:
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: crontabs.stable.example.com
spec:
group: stable.example.com
version: v1
scope: Namespaced
names:
plural: crontabs
singular: crontab
kind: CronTab
shortNames:
- ct
# categories is a list of grouped resources the custom resource belongs to.
categories:
- all
And create it:
kubectl create -f resourcedefinition.yaml
After the CustomResourceDefinition object has been created, you can create custom objects.
Save the following YAML to my-crontab.yaml
:
apiVersion: "stable.example.com/v1"
kind: CronTab
metadata:
name: my-new-cron-object
spec:
cronSpec: "* * * * */5"
image: my-awesome-cron-image
and create it:
kubectl create -f my-crontab.yaml
You can specify the category using kubectl get
:
kubectl get all
and it will include the custom resources of kind CronTab
:
NAME AGE
crontabs/my-new-cron-object 3s