Pipelines

Our operator supports deploying Haystack pipelines through the Kubernetes API using the operator concept.

The operator consumes the customer resource definition (CRD) and provisions the underlying resources to deploy the haystack pipeline.

The operator is responsible for managing the lifecycle of the pipeline and ensuring that the pipeline is running as expected.

The latest CRD definition can be found on our github page here.

Operator APIs

The apiVersion for the operator is pipelines.baler.gatecastle.com/v1.

The supported resources are:

  • kind: Haystack

Haystack

The Haystack kind is used to define the haystack pipeline that needs to be deployed and additional configurations for controlling scheduling, scaling, etc.

Haystack is a namespaced resource, which means that it can only be deployed in a specific namespace.

In order to specify namespace, you can use the metadata.namespace field in the Haystack resource. Or you can deploy the Haystack resource in a specific namespace via specific the namespace in your kubectl command using the -n flag.

See more about the possible configurations here.

Manifest structure

The structure is as follows:

apiVersion: pipelines.baler.gatecastle.com/v1
kind: Haystack
metadata:
  name: my-haystack-pipeline
  annotations:
    # Add any annotations here, the operator
    # will preserve these annotations.
    # There are some reserved annotations that
    # control the behavior of the operator.
    # See more about these annotations [here]().
    description: "My haystack pipeline"
spec:
    # Here you place your haystack pipeline yaml definition.
    # See more about the haystack pipeline yaml [here](https://haystack.deepset.ai/tutorials/20_using_haystack_with_rest_api).
    version: ...
    components: ...
    pipelines: ...

You can find more examples at our github page.

Deployment

kubectl apply -f my-haystack-pipeline.yaml

You can use the kubectl utility to deploy the haystack pipeline, just as you would deploy any other Kubernetes resource.

Status

The operator will update the status of the Haystack resource to reflect the current state of the pipeline.

You can check the status of the pipeline using the following command:

kubectl get haystack my-haystack-pipeline

The status will show the current state of the pipeline and any errors that might have occurred during the deployment.

You can also describe the resource to get more detailed information:

kubectl describe haystack my-haystack-pipeline

Deleting

You can delete the haystack pipeline using the following command:

kubectl delete haystack my-haystack-pipeline

The operator will clean up all the resources associated with the pipeline and update the status of the Haystack resource to reflect the deletion.

Updating

You can update the haystack pipeline by modifying the Haystack resource and applying the changes using the kubectl apply command.

The operator will detect the changes and update the pipeline accordingly.

Note that some changes might require a restart of the pipeline, in which case the operator will restart the pipeline automatically.

Annotations

The operator supports a number of annotations that control the behavior of the operator. These annotations are used to configure the operator and provide additional information about the pipeline.

The supported annotations are:

auto-provision-documentstore.pipelines.baler.gatecastle.com/enabled

  • Type: string

  • Allowed Values: "true", "false"

  • Comment: If set to "true", the Datastore will be automatically provisioned when a pipeline is created.

  • Default Value: "false"

This property controls whether a Datastore should be automatically provisioned upon the creation of a pipeline.

preserve-documentstore.pipelines.baler.gatecastle.com/enabled

  • Type: string

  • Allowed Values: "true", "false"

  • Comment: If set to "true", the Datastore will be preserved when a pipeline is deleted.

  • Default Value: "false"

This setting determines if the Datastore should be kept even after the deletion of a pipeline.

compute.pipelines.baler.gatecastle.com/type

  • Type: string

  • Allowed Values: "cpu", "gpu"

  • Comment: The type of compute resource to use for the pipeline.

  • Default Value: "cpu"

Specifies the computing resource type for pipeline execution.

compute.pipelines.baler.gatecastle.com/request-cpu

  • Type: string

  • Pattern: ^[0-9]+m$

  • Comment: The amount of CPU to request for the pipeline, in millicores.

  • Default Value: "1000m"

Defines the CPU resources to be allocated for a pipeline, measured in millicores (where 1000m equals 1 CPU core).

compute.pipelines.baler.gatecastle.com/request-memory

  • Type: string

  • Pattern: ^[0-9]+[KMG]i$

  • Comment: The amount of memory to request for the pipeline, with units in KiB, MiB, or GiB.

  • Default Value: "1Gi"

Indicates the memory allocation for a pipeline, using binary multiples (KiB, MiB, GiB).

compute.pipelines.baler.gatecastle.com/request-gpu

  • Type: string

  • Pattern: ^[0-9]+$

  • Comment: The number of GPUs to request for the pipeline.

  • Default Value: "0"

Determines how many GPU units are requested for the pipeline.

compute.pipelines.baler.gatecastle.com/node-selector

  • Type: string

  • Pattern: ^$|^(?:[a-zA-Z0-9_\-]+:[a-zA-Z0-9_\-]+;)*(?:[a-zA-Z0-9_\-]+:[a-zA-Z0-9_\-]+)?$

  • Comment: Specifies node labels for pod assignment, formatted as 'key1:value1;key2:value2;...'. An empty string signifies no specific node selection.

  • Default Value: ""

Allows for specifying node labels to control on which nodes a pipeline’s pods should be scheduled.

compute.pipelines.baler.gatecastle.com/tolerations

  • Type: string

  • Pattern: ^([a-zA-Z0-9_\-]+:[a-zA-Z0-9_\-]+:[a-zA-Z0-9_\-]*:[a-zA-Z0-9_\-]+;)*([a-zA-Z0-9_\-]+:[a-zA-Z0-9_\-]+:[a-zA-Z0-9_\-]*:[a-zA-Z0-9_\-]+)$

  • Comment: Tolerations for the pod, formatted as 'key:operator:value:effect', with multiple tolerations separated by semicolons (';'). The value can be empty.

  • Example: "key1:Equal:value1:NoSchedule;key2:Exists::NoExecute"

  • Default Value: ""

Defines tolerations for pods to allow scheduling on nodes with matching taints, facilitating fine-grained control over pod placement.

compute.pipelines.baler.gatecastle.com/service-account

  • Type: string

  • Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$

  • Comment: The name of the service account to use for the pipeline.

  • Default Value: ""

Specifies the service account to be used for the pipeline pod deployment in the Kubernetes cluster.

compute.pipelines.baler.gatecastle.com/image-pull-secrets

  • Type: string

  • Pattern: ^([^;]+(;[^;]+)*)?$

  • Comment: The name of the image pull secrets to use for the pipeline image pull.

  • Default Value: ""

Pipeline Statuses

The operator maintains the status of the pipeline in the status field of the Haystack resource. The phase field indicates the current state of the pipeline.

The possible values for the phase field are:

  • Pending: The pipeline is in the process of being created. (Underlying resources are being provisioned)

  • Running: The pipeline is running and ready to accept requests.

  • Failed: The pipeline has encountered an error and is not running.

  • Error: The pipeline is in an error state and requires manual intervention.

In a production setup you can monitor the status of the pipeline to set up your alerting and monitoring systems.

Supported Haystack runtimes

These versions will be supported by the operator if you specify them under the spec.version field in the Haystack resource.

Note that haystack pipeline definitions may not be backward compatible, so you should ensure that the pipeline definition is compatible with the specified version.