Introduction
Container Object Storage Interface (COSI)
Container Object Storage Interface (COSI) is a set of abstractions for provisioning and management of object storage. It aims to be a common layer of abstraction across multiple object storage vendors, such that workloads can request and automatically be provisioned with object storage buckets.
The goals of this project are:
- Automate object storage provisioning, access and management
- Provide a common layer of abstraction for consuming object storage
- Facilitate lift and shift of workloads across object storage providers (i.e. prevent vendor lock-in)
Why another standard?
Kubernetes abstracts file/block storage via the CSI standard. The primitives for file/block storage do not extend well to object storage. Here is the extremely concise and incomplete list of reasons why:
- Unit of provisioned storage - Bucket instead of filesystem mount or blockdevice.
- Access is over the network instead of local POSIX calls.
- No common protocol for consumption across various implementations of object storage.
- Management policies and primitives - for instance, mounting and unmounting do not apply to object storage.
The existing primitives in CSI do not apply to objectstorage. Thus the need for a new standard to automate the management of objectstorage.
Community, discussion, contribution, and support
You can reach the maintainers and interact with community through the following channels:
- #sig-storage-cosi slack channel
- container-object-storage-interface mailing list
Community Meeting
- Thursday at 10:30AM - 11:00AM PT
- SIG Storage Zoom Meeting Room (Password is
77777
) - Meeting Notes
Quick Start
COSI components are out-of-tree and need manual installation.
To install COSI Custom Resource Definitions and COSI Controller, issue the following command:
kubectl create -k 'https://github.com/kubernetes-sigs/container-object-storage-interface//?ref=v0.2.0'
COSI Management Tasks
This section provides details for some of the operations that need to be performed when managing COSI components.
Administrative Tasks
Installing Custom Resources and Controller
Refer to Quickstart Guide for installation instructions.
Installing Driver
Refer to Installing Driver for detailed steps.
Creating BucketClasses and BucketAccessClasses
These resources define storage classes and access policies for object storage.
---
apiVersion: objectstorage.k8s.io/v1alpha1
kind: BucketAccessClass
metadata:
name: example-accessclass
driverName: cosi.example.com
authenticationType: Key
parameters:
foo: bar
---
apiVersion: objectstorage.k8s.io/v1alpha1
kind: BucketClass
metadata:
name: example-class
driverName: cosi.example.com
deletionPolicy: Delete
parameters:
foo: bar
User Tasks
Creating BucketClaims
A BucketClaim
requests a new bucket provisioned by the COSI driver.
apiVersion: objectstorage.k8s.io/v1alpha1
kind: BucketClaim
metadata:
name: example-claim
spec:
bucketClassName: example-class
protocols: [ 'S3' ]
Creating BucketAccesses
A BucketAccess
grants access to a previously created bucket claim.
apiVersion: objectstorage.k8s.io/v1alpha1
kind: BucketAccess
metadata:
name: example-access
spec:
bucketClaimName: example-claim
protocol: S3
bucketAccessClassName: example-accessclass
credentialsSecretName: example-secret
Using the COSI-Provisioned Object Storage Credentials
Applications can access COSI-provisioned object storage credentials using Kubernetes Secrets.
spec:
template:
spec:
containers:
- volumeMounts:
- mountPath: /conf
name: example-secret-vol
volumes:
- name: example-secret-vol
secret:
secretName: example-secret
items:
- key: BucketInfo
path: BucketInfo
Installing COSI Driver
There is no single and definitive guide for installing a COSI driver. The installation process may vary depending on the specific driver you are using and the platform it is being deployed on.
General Installation Steps
- Check Compatibility: Ensure the driver is compatible with your Kubernetes version and COSI API version.
- Refer to Driver Documentation: Each driver may have unique requirements and steps. Refer to the documentation of each driver for specific installation details. Many drivers provide Helm charts or Kubernetes manifests for easier deployment.
Troubleshooting
This document provides troubleshooting steps for common issues encountered when using COSI components including Custom Resource Definitions (CRDs), Custom Resources (CRs), the COSI Controller, and Drivers with Sidecars.
CRD Issues
Symptoms
- CRDs fail to apply to the cluster.
- Resources are not recognized by
kubectl
.
Possible Causes & Resolution
- Configuration Misalignment
- Check: Validate whether the installation followed official documentation steps.
- Resolve: Reinstall CRDs by strictly following the quick-start guide, step-by-step.
Custom Resource (CR) Issues
Symptoms
- BucketClaim/BucketAccess CRs are not processed.
- Status
bucketReady
oraccessGranted
conditions remainfalse
.
Possible Causes & Resolution
-
Misconfigured CR Spec
- Check: Validate required fields (e.g.,
bucketName
,driverName
). - Fix: Refer to the CR examples in the driver documentation.
- Check: Validate required fields (e.g.,
-
Controller Not Responding
- Check: Verify the controller pods is running (
kubectl get pods -n container-object-storage-system
). - Fix: Inspect controller logs for errors.
- Check: Verify the controller pods is running (
Controller Issues
Symptoms
- Controller pod crashes or enters
CrashLoopBackOff
. - No events generated for CRs.
Possible Causes & Resolution
-
Missing Permissions
- Check: Review RBAC roles for the controller service account.
- Fix: Ensure the controller has permissions to manage CRDs and watch resources.
-
Reconciliation Failures
- Check: Look for
Reconcile
errors in controller logs. - Fix: Validate driver connectivity or CR configurations (e.g., invalid bucket class parameters).
- Check: Look for
Driver with Sidecar Issues
Symptoms
- Sidecar fails to communicate with the driver.
- Bucket provisioning times out.
Possible Causes & Resolution
-
Sidecar-Driver Communication Failure
- Check: Ensure the sidecar and driver share the communication socket (e.g. shared
volumeMounts
). - Fix: Adjust driver manifests and
COSI_ENDPOINT
for both driver and sidecar.
- Check: Ensure the sidecar and driver share the communication socket (e.g. shared
-
Resource Conflicts
- Check: Multiple drivers using the same driver name.
- Fix: Ensure unique
driverName
values per driver instance.
-
Sidecar Liveness Probe Failures
- Check: Inspect sidecar logs for health check errors.
- Fix: Adjust liveness/readiness probe thresholds in the sidecar deployment.
FAQs
-
Q: Why is my CRD not recognized after applying?
A: Ensure the CRD is compatible with your Kubernetes cluster version and the COSI controller is running. -
Q: The driver isn't responding to provisioning requests. What should I check?
A: Verify driver-sidecar communication. -
Q: Why is my bucket stuck in
ready: false
state?
A: Check storage quotas, driver availability, and controller logs for reconciliation errors.
Monitoring
// TODO: add once metrics are in place
Developer guide
The Developer Guide provides an in-depth overview of developing for the Container Object Storage Interface (COSI). This chapter is designed for developers who want to contribute to COSI, build drivers, or integrate object storage solutions with Kubernetes using the COSI specification.
Developing "core" COSI
With “core” COSI we refer to the common set of API and controllers that are required to run any COSI driver.
Before your first contribution, you should follow the Kubernetes Contributor Guide.
To further understand the COSI architecture, please refer to KEP-1979: Object Storage Support.
Before contributing a Pull Request, ensure a GitHub issue exists corresponding to the change.
Local code development
For new contributors, use make help
, and use Core targets as needed.
These targets ensure changes build successfully, pass basic checks, and are ready for end-to-end tests run in COSI's automated CI.
Other more advanced targets are available and also described in make help
output.
Some specific workflows are documented below.
Building and deploying COSI controller changes locally
export CONTROLLER_TAG="$MY_REPO"/cosi-controller:latest # replace MY_REPO with desired dev repo
make -j prebuild
make build.controller
docker push "$CONTROLLER_TAG"
make deploy
Developing a COSI Driver
Overview
The starting point for developing a COSI driver is the COSI Driver Sample. This repository provides a foundational implementation that you can build upon.
Implementing the Servers
A COSI driver requires implementing two main servers:
Identity Server
The IdentityServer
provides driver metadata and implements the following interface:
type IdentityServer interface {
DriverGetInfo(context.Context, *cosi.DriverGetInfoRequest) (*cosi.DriverGetInfoResponse, error)
}
Provisioner Server
The ProvisionerServer
handles bucket provisioning and access management:
type ProvisionerServer interface {
DriverCreateBucket(context.Context, *cosi.DriverCreateBucketRequest) (*cosi.DriverCreateBucketResponse, error)
DriverDeleteBucket(context.Context, *cosi.DriverDeleteBucketRequest) (*cosi.DriverDeleteBucketResponse, error)
DriverGrantBucketAccess(context.Context, *cosi.DriverGrantBucketAccessRequest) (*cosi.DriverGrantBucketAccessResponse, error)
DriverRevokeBucketAccess(context.Context, *cosi.DriverRevokeBucketAccessRequest) (*cosi.DriverRevokeBucketAccessResponse, error)
}
Entrypoint
The driver entrypoint initializes logging, parses flags, and starts the gRPC server:
package main
func main() {
klog.InitFlags(nil)
flag.Parse()
if err := run(context.Background()); err != nil {
klog.ErrorS(err, "Exiting on error")
os.Exit(1)
}
}
func run(ctx context.Context) error {
ctx, stop := signal.NotifyContext(ctx,
os.Interrupt,
syscall.SIGINT,
syscall.SIGTERM,
)
defer stop()
cfg := config.Load() // placeholder
identityServer := &driver.IdentityServer{Name: "cosi.example.com"}
provisionerServer := &driver.ProvisionerServer{
Config: cfg,
}
server, err := grpcServer(identityServer, provisionerServer)
if err != nil {
return fmt.Errorf("gRPC server creation failed: %w", err)
}
cosiEndpoint, ok := os.LookupEnv("COSI_ENDPOINT")
if !ok {
cosiEndpoint = "unix:///var/lib/cosi/cosi.sock"
}
lis, cleanup, err := listener(ctx, cosiEndpoint)
if err != nil {
return fmt.Errorf("failed to create listener for %s: %w", cosiEndpoint, err)
}
defer cleanup()
var wg sync.WaitGroup
wg.Add(1)
go shutdown(ctx, &wg, server)
if err = server.Serve(lis); err != nil {
return fmt.Errorf("gRPC server failed: %w", err)
}
wg.Wait()
return nil
}
Creating Listener
The listener sets up a gRPC connection for handling requests:
func listener(
ctx context.Context,
cosiEndpoint string,
) (net.Listener, func(), error) {
endpointURL, err := url.Parse(cosiEndpoint)
if err != nil {
return nil, nil, fmt.Errorf("unable to parse COSI endpoint: %w", err)
}
listenConfig := net.ListenConfig{}
if endpointURL.Scheme == "unix" {
_ = os.Remove(endpointURL.Path) // Cleanup stale socket
}
listener, err := listenConfig.Listen(ctx, endpointURL.Scheme, endpointURL.Path)
if err != nil {
return nil, nil, fmt.Errorf("unable to create listener: %w", err)
}
cleanup := func() {
if endpointURL.Scheme == "unix" {
if err := os.Remove(endpointURL.Path); err != nil {
klog.ErrorS(err, "Failed to remove old socket")
}
}
}
return listener, cleanup, nil
}
Creating gRPC Server
The gRPC server registers both the IdentityServer
and ProvisionerServer
:
func grpcServer(
identity cosi.IdentityServer,
provisioner cosi.ProvisionerServer,
) (*grpc.Server, error) {
server := grpc.NewServer()
if identity == nil || provisioner == nil {
return nil, errors.New("provisioner and identity servers cannot be nil")
}
cosi.RegisterIdentityServer(server, identity)
cosi.RegisterProvisionerServer(server, provisioner)
return server, nil
}
Graceful Shutdown
To ensure clean shutdown, implement a graceful termination mechanism:
const (
gracePeriod = 5 * time.Second
)
func shutdown(
ctx context.Context,
wg *sync.WaitGroup,
g *grpc.Server,
) {
<-ctx.Done()
defer wg.Done()
defer klog.Info("Stopped")
klog.Info("Shutting down")
dctx, stop := context.WithTimeout(context.Background(), gracePeriod)
defer stop()
c := make(chan struct{})
if g != nil {
go func() {
g.GracefulStop()
c <- struct{}{}
}()
for {
select {
case <-dctx.Done():
klog.Info("Forcing shutdown")
g.Stop()
return
case <-c:
return
}
}
}
}
Developing client applications
No official client libraries
We do not provide official client libraries for interacting with COSI secrets. Instead, we encourage users to build their own clients using standard tools and APIs.
- Different users have different needs, and maintaining an official client library might limit their ability to customize or optimize for specific use cases.
- Providing and maintaining client libraries across multiple languages is a significant effort, requiring continuous updates and support.
- By relying on standard APIs, users can integrate directly with COSI without additional abstraction layers that may introduce unnecessary complexity.
Stability and breaking changes
We follow a strict versioning policy to ensure stability while allowing for necessary improvements.
- Patch Releases (
v1alphaX
): No breaking changes are introduced between patch versions. - Version Upgrades (
v1alpha1
tov1alpha2
): Breaking changes, including format modifications, may occur between these versions.
For more details, refer to the Kubernetes Versioning and Deprecation Policy.
Existing Guides
For guidance on developing clients, refer to our language-specific documentation:
If additional client guides are needed, we welcome contributions from the community.
Developing Client Apps in Go
Configuration Structure
The Config struct is the primary configuration object for the storage package. It encapsulates all necessary settings for interacting with different storage providers. This design ensures that all configuration details are centralized and easily maintainable, allowing your application to switch storage backends with minimal code changes.
The nested Spec
struct defines both generic and provider-specific parameters:
BucketName
: Specifies the target storage container or bucket. This value directs where the data will be stored or retrieved.AuthenticationType
: Indicates the method of authentication (for example, "key"). This ensures that the correct credentials are used when accessing a storage provider.Protocols
: An array of strings that informs the system which storage protocols (e.g., "s3" or "azure") are supported. The factory uses this to determine the appropriate client to initialize.SecretS3
/SecretAzure
: These fields hold pointers to the respective secret structures needed for authenticating with S3 or Azure services. Their presence is conditional on the protocols configured.
// import "example.com/pkg/storage"
package storage
type Config struct {
Spec Spec `json:"spec"`
}
type Spec struct {
BucketName string `json:"bucketName"`
AuthenticationType string `json:"authenticationType"`
Protocols []string `json:"protocols"`
SecretS3 *s3.SecretS3 `json:"secretS3,omitempty"`
SecretAzure *azure.SecretAzure `json:"secretAzure,omitempty"`
}
Azure Secret Structure
The SecretAzure
struct holds authentication credentials for accessing Azure-based storage services. It is essential when interacting with Azure Blob storage, as it contains a shared access token along with an expiration timestamp. The inclusion of the ExpiryTimestamp
allows your application to check token validity.
While current COSI implementation doesn't auto-renew tokens, the
ExpiryTimestamp
provides hooks for future refresh logic.
// import "example.com/pkg/storage/azure"
package azure
type SecretAzure struct {
AccessToken string `json:"accessToken"`
ExpiryTimestamp time.Time `json:"expiryTimeStamp"`
}
S3 Secret Structure
The SecretS3
struct holds authentication credentials for accessing S3-compatible storage services. This struct includes the endpoint, region, and access credentials required to securely interact with the S3 service. By isolating these values into a dedicated structure, the design helps maintain clear separation between configuration types, thus enhancing code clarity.
// import "example.com/pkg/storage/s3"
package s3
type SecretS3 struct {
Endpoint string `json:"endpoint"`
Region string `json:"region"`
AccessKeyID string `json:"accessKeyID"`
AccessSecretKey string `json:"accessSecretKey"`
}
Factory
The factory pattern1 is used to instantiate the appropriate storage backend based on the provided configuration. We will hide the implementation behind the interface.
The factory function examines the configuration’s Protocols
array and validates the AuthenticationType
along with the corresponding secret. It then returns a concrete implementation of the Storage interface. This method of instantiation promotes extensibility, making it easier to support additional storage protocols in the future, as the COSI specification evolves.
Here is a minimal interface that supports only basic Delete
/Get
/Put
operations:
type Storage interface {
Delete(ctx context.Context, key string) error
Get(ctx context.Context, key string, wr io.Writer) error
Put(ctx context.Context, key string, data io.Reader, size int64) error
}
Our implementation of factory method can be defined as following:
// import "example.com/pkg/storage"
package storage
import (
"fmt"
"slices"
"strings"
"example.com/pkg/storage/azure"
"example.com/pkg/storage/s3"
)
func New(config Config, ssl bool) (Storage, error) {
if slices.ContainsFunc(config.Spec.Protocols, func(s string) bool { return strings.EqualFold(s, "s3") }) {
if !strings.EqualFold(config.Spec.AuthenticationType, "key") {
return nil, fmt.Errorf("invalid authentication type for s3")
}
s3secret := config.Spec.SecretS3
if s3secret == nil {
return nil, fmt.Errorf("s3 secret missing")
}
return s3.New(config.Spec.BucketName, *s3secret, ssl)
}
if slices.ContainsFunc(config.Spec.Protocols, func(s string) bool { return strings.EqualFold(s, "azure") }) {
if !strings.EqualFold(config.Spec.AuthenticationType, "key") {
return nil, fmt.Errorf("invalid authentication type for azure")
}
azureSecret := config.Spec.SecretAzure
if azureSecret == nil {
return nil, fmt.Errorf("azure secret missing")
}
return azure.New(config.Spec.BucketName, *azureSecret)
}
return nil, fmt.Errorf("invalid protocol (%v)", config.Spec.Protocols)
}
Clients
As we alredy defined the factory and uppermost configuration, let's get into the details of the clients, that will implement the Storage
interface.
S3
In the implementation of S3 client, we will use MinIO client library, as it's more lightweight than AWS SDK.
// import "example.com/pkg/storage/s3"
package s3
import (
"context"
"fmt"
"io"
"net/http"
"time"
"github.com/minio/minio-go/v7"
"github.com/minio/minio-go/v7/pkg/credentials"
)
type Client struct {
s3cli *minio.Client
bucketName string
}
func New(bucketName string, s3secret SecretS3, ssl bool) (*Client, error) {
s3cli, err := minio.New(s3secret.Endpoint, &minio.Options{
Creds: credentials.NewStaticV4(s3secret.AccessKeyID, s3secret.AccessSecretKey, ""),
Region: s3secret.Region,
Secure: ssl,
})
if err != nil {
return nil, fmt.Errorf("unable to create client: %w", err)
}
return &Client{
s3cli: s3cli,
bucketName: bucketName,
}, nil
}
func (c *Client) Delete(ctx context.Context, key string) error {
return c.s3cli.RemoveObject(ctx, c.bucketName, key, minio.RemoveObjectOptions{})
}
func (c *Client) Get(ctx context.Context, key string, wr io.Writer) error {
obj, err := c.s3cli.GetObject(ctx, c.bucketName, key, minio.GetObjectOptions{})
if err != nil {
return err
}
_, err = io.Copy(wr, obj)
return err
}
func (c *Client) Put(ctx context.Context, key string, data io.Reader, size int64) error {
_, err := c.s3cli.PutObject(ctx, c.bucketName, key, data, size, minio.PutObjectOptions{})
return err
}
Azure Blob
In the implementation of Azure client, we will use Azure SDK client library. Note, that the configuration is done with NoCredentials
client, as the Azure secret contains shared access signatures (SAS)2.
// import "example.com/pkg/storage/azure"
package azure
import (
"context"
"errors"
"fmt"
"io"
"time"
"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob"
)
type Client struct {
azCli *azblob.Client
containerName string
}
func New(containerName string, azureSecret SecretAzure) (*Client, error) {
azCli, err := azblob.NewClientWithNoCredential(azureSecret.AccessToken, nil)
if err != nil {
return nil, fmt.Errorf("unable to create client: %w", err)
}
return &Client{
azCli: azCli,
containerName: containerName,
}, nil
}
func (c *Client) Delete(ctx context.Context, blobName string) error {
_, err := c.azCli.DeleteBlob(ctx, c.containerName, blobName, nil)
return err
}
func (c *Client) Get(ctx context.Context, blobName string, wr io.Writer) error {
stream, err := c.azCli.DownloadStream(ctx, c.containerName, blobName, nil)
if err != nil {
return fmt.Errorf("unable to get download stream: %w", err)
}
_, err = io.Copy(wr, stream.Body)
return err
}
func (c *Client) Put(ctx context.Context, blobName string, data io.Reader, size int64) error {
_, err := c.azCli.UploadStream(ctx, c.containerName, blobName, data, nil)
return err
}
Summing up
Once all components are in place, using the storage package in your application becomes straightforward. The process starts with reading a JSON configuration file, which is then decoded into the Config
struct. The factory method selects and initializes the appropriate storage client based on the configuration, enabling seamless integration with either S3 or Azure storage.
import (
"encoding/json"
"os"
"example.com/pkg/storage"
)
func example() {
f, err := os.Open("/opt/cosi/BucketInfo")
if err != nil {
panic(err)
}
defer f.Close()
var cfg storage.Config
if err := json.NewDecoder(f).Decode(&cfg); err != nil {
panic(err)
}
client, err := storage.New(cfg, true)
if err != nil {
panic(err)
}
// use client Put/Get/Delete
// ...
}
Drivers
- Platform will take you to platform documentation;
- COSI Driver Name will take you to driver repository.
Platform | COSI Driver Name | Description | Compatible with COSI Version(s) |
---|---|---|---|
Akamai Cloud Object Storage | objectstorage.cosi.linode.com | A Kubernetes Container Object Storage Interface (COSI) Driver for Linode | v1alpha1 |
Azure Blob | blob.cosi.azure.com | This driver allows Kubernetes to use Azure Blob Storage using the Container Storage Object Interface (COSI) infrastructure | v1alpha1 |
Ceph Rados Gateway | ceph.objectstorage.k8s.io | COSI driver for Ceph Object Store aka RGW | v1alpha1 |
Dell ObjectScale | cosi.dellemc.com | COSI Driver for Dell ObjectScale | v1alpha1 |
HPE Alletra Storage MP X10000 | cosi.hpe.com | A Kubernetes Container Object Storage Interface (COSI) driver for HPE Alletra Storage MP X10000 | v1alpha1 |
Scality RING and ARTESCA Object Storage | cosi.scality.com | Scality COSI Driver integrates Scality RING, ARTESCA, and other AWS S3 and IAM compatible object storage with Kubernetes | v1alpha1 |
SeaweedFS | seaweedfs.objectstorage.k8s.io | COSI driver implementation for SeaweedFS | v1alpha1 |
Deprecated drivers
Deprecated drivers are no longer maintained or recommended for use with COSI; users should migrate to supported alternatives to ensure compatibility and security.
Name | COSI Driver Name | Description | Compatible with COSI Version(s) |
---|---|---|---|
S3GW | s3gw.objectstorage.k8s.io | COSI driver for s3gw | v1alpha1 |
MinIO | minio.objectstorage.k8s.io | Sample Driver that provides reference implementation for Container Object Storage Interface (COSI) API. | pre-alpha |
API Reference
API Reference provides API Schema definitions for COSI's objectstorage.k8s.io
API group.
API Reference
Packages
objectstorage.k8s.io/v1alpha1
AuthenticationType
Underlying type: string
Appears in:
Field | Description |
---|---|
Key | |
IAM |
Bucket
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
kind string | Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds | ||
apiVersion string | APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources | ||
metadata ObjectMeta | Refer to Kubernetes API documentation for fields of metadata . | ||
spec BucketSpec | |||
status BucketStatus |
BucketAccess
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
kind string | Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds | ||
apiVersion string | APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources | ||
metadata ObjectMeta | Refer to Kubernetes API documentation for fields of metadata . | ||
spec BucketAccessSpec | |||
status BucketAccessStatus |
BucketAccessClass
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
kind string | Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds | ||
apiVersion string | APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources | ||
metadata ObjectMeta | Refer to Kubernetes API documentation for fields of metadata . | ||
driverName string | DriverName is the name of driver associated with this BucketAccess | ||
authenticationType AuthenticationType | AuthenticationType denotes the style of authentication It can be one of Key - access, secret tokens based authentication IAM - implicit authentication of pods to the OSP based on service account mappings | ||
parameters object (keys:string, values:string) | Parameters is an opaque map for passing in configuration to a driver for granting access to a bucket |
BucketAccessSpec
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
bucketClaimName string | BucketClaimName is the name of the BucketClaim. | ||
protocol Protocol | Protocol is the name of the Protocol that this access credential is supposed to support If left empty, it will choose the protocol supported by the bucket. If the bucket supports multiple protocols, the end protocol is determined by the driver. | ||
bucketAccessClassName string | BucketAccessClassName is the name of the BucketAccessClass | ||
credentialsSecretName string | CredentialsSecretName is the name of the secret that COSI should populate with the credentials. If a secret by this name already exists, then it is assumed that credentials have already been generated. It is not overridden. This secret is deleted when the BucketAccess is delted. | ||
serviceAccountName string | ServiceAccountName is the name of the serviceAccount that COSI will map to the OSP service account when IAM styled authentication is specified |
BucketAccessStatus
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
accountID string | AccountID is the unique ID for the account in the OSP. It will be populated by the COSI sidecar once access has been successfully granted. | ||
accessGranted boolean | AccessGranted indicates the successful grant of privileges to access the bucket |
BucketClaim
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
kind string | Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds | ||
apiVersion string | APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources | ||
metadata ObjectMeta | Refer to Kubernetes API documentation for fields of metadata . | ||
spec BucketClaimSpec | |||
status BucketClaimStatus |
BucketClaimSpec
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
bucketClassName string | Name of the BucketClass | ||
protocols Protocol array | Protocols are the set of data API this bucket is required to support. The possible values for protocol are: - S3: Indicates Amazon S3 protocol - Azure: Indicates Microsoft Azure BlobStore protocol - GCS: Indicates Google Cloud Storage protocol | ||
existingBucketName string | Name of a bucket object that was manually created to import a bucket created outside of COSI If unspecified, then a new Bucket will be dynamically provisioned |
BucketClaimStatus
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
bucketReady boolean | BucketReady indicates that the bucket is ready for consumpotion by workloads | ||
bucketName string | BucketName is the name of the provisioned Bucket in response to this BucketClaim. It is generated and set by the COSI controller before making the creation request to the OSP backend. |
BucketClass
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
kind string | Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds | ||
apiVersion string | APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources | ||
metadata ObjectMeta | Refer to Kubernetes API documentation for fields of metadata . | ||
driverName string | DriverName is the name of driver associated with this bucket | ||
deletionPolicy DeletionPolicy | DeletionPolicy is used to specify how COSI should handle deletion of this bucket. There are 2 possible values: - Retain: Indicates that the bucket should not be deleted from the OSP - Delete: Indicates that the bucket should be deleted from the OSP once all the workloads accessing this bucket are done | Retain | |
parameters object (keys:string, values:string) | Parameters is an opaque map for passing in configuration to a driver for creating the bucket |
BucketSpec
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
driverName string | DriverName is the name of driver associated with this bucket | ||
bucketClassName string | Name of the BucketClass specified in the BucketRequest | ||
bucketClaim ObjectReference | Name of the BucketClaim that resulted in the creation of this Bucket In case the Bucket object was created manually, then this should refer to the BucketClaim with which this Bucket should be bound | ||
protocols Protocol array | Protocols are the set of data APIs this bucket is expected to support. The possible values for protocol are: - S3: Indicates Amazon S3 protocol - Azure: Indicates Microsoft Azure BlobStore protocol - GCS: Indicates Google Cloud Storage protocol | ||
parameters object (keys:string, values:string) | |||
deletionPolicy DeletionPolicy | DeletionPolicy is used to specify how COSI should handle deletion of this bucket. There are 2 possible values: - Retain: Indicates that the bucket should not be deleted from the OSP (default) - Delete: Indicates that the bucket should be deleted from the OSP once all the workloads accessing this bucket are done | Retain | |
existingBucketID string | ExistingBucketID is the unique id of the bucket in the OSP. This field should be used to specify a bucket that has been created outside of COSI. This field will be empty when the Bucket is dynamically provisioned by COSI. |
BucketStatus
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
bucketReady boolean | BucketReady is a boolean condition to reflect the successful creation of a bucket. | ||
bucketID string | BucketID is the unique id of the bucket in the OSP. This field will be populated by COSI. |
DeletionPolicy
Underlying type: string
Appears in:
Field | Description |
---|---|
Retain | |
Delete |
Protocol
Underlying type: string
Appears in:
Field | Description |
---|---|
S3 | |
Azure | |
GCP |
API Reference
Packages
objectstorage.k8s.io/v1alpha2
Package v1alpha2 contains API Schema definitions for the objectstorage v1alpha2 API group.
Resource Types
- Bucket
- BucketAccess
- BucketAccessClass
- BucketAccessClassList
- BucketAccessList
- BucketClaim
- BucketClaimList
- BucketClass
- BucketClassList
- BucketList
Bucket
Bucket is the Schema for the buckets API
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
apiVersion string | objectstorage.k8s.io/v1alpha2 | ||
kind string | Bucket | ||
kind string | Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds | ||
apiVersion string | APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources | ||
metadata ObjectMeta | Refer to Kubernetes API documentation for fields of metadata . | ||
spec BucketSpec | spec defines the desired state of Bucket | ||
status BucketStatus | status defines the observed state of Bucket |
BucketAccess
BucketAccess is the Schema for the bucketaccesses API
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
apiVersion string | objectstorage.k8s.io/v1alpha2 | ||
kind string | BucketAccess | ||
kind string | Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds | ||
apiVersion string | APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources | ||
metadata ObjectMeta | Refer to Kubernetes API documentation for fields of metadata . | ||
spec BucketAccessSpec | spec defines the desired state of BucketAccess | ||
status BucketAccessStatus | status defines the observed state of BucketAccess |
BucketAccessClass
BucketAccessClass is the Schema for the bucketaccessclasses API
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
apiVersion string | objectstorage.k8s.io/v1alpha2 | ||
kind string | BucketAccessClass | ||
kind string | Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds | ||
apiVersion string | APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources | ||
metadata ObjectMeta | Refer to Kubernetes API documentation for fields of metadata . | ||
spec BucketAccessClassSpec | spec defines the desired state of BucketAccessClass | ||
status BucketAccessClassStatus | status defines the observed state of BucketAccessClass |
BucketAccessClassList
BucketAccessClassList contains a list of BucketAccessClass
Field | Description | Default | Validation |
---|---|---|---|
apiVersion string | objectstorage.k8s.io/v1alpha2 | ||
kind string | BucketAccessClassList | ||
kind string | Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds | ||
apiVersion string | APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources | ||
metadata ListMeta | Refer to Kubernetes API documentation for fields of metadata . | ||
items BucketAccessClass array |
BucketAccessClassSpec
BucketAccessClassSpec defines the desired state of BucketAccessClass
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
foo string | foo is an example field of BucketAccessClass. Edit bucketaccessclass_types.go to remove/update |
BucketAccessClassStatus
BucketAccessClassStatus defines the observed state of BucketAccessClass.
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
conditions Condition array | conditions represent the current state of the BucketAccessClass resource. Each condition has a unique type and reflects the status of a specific aspect of the resource. Standard condition types include: - "Available": the resource is fully functional - "Progressing": the resource is being created or updated - "Degraded": the resource failed to reach or maintain its desired state The status of each condition is one of True, False, or Unknown. |
BucketAccessList
BucketAccessList contains a list of BucketAccess
Field | Description | Default | Validation |
---|---|---|---|
apiVersion string | objectstorage.k8s.io/v1alpha2 | ||
kind string | BucketAccessList | ||
kind string | Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds | ||
apiVersion string | APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources | ||
metadata ListMeta | Refer to Kubernetes API documentation for fields of metadata . | ||
items BucketAccess array |
BucketAccessSpec
BucketAccessSpec defines the desired state of BucketAccess
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
foo string | foo is an example field of BucketAccess. Edit bucketaccess_types.go to remove/update |
BucketAccessStatus
BucketAccessStatus defines the observed state of BucketAccess.
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
conditions Condition array | conditions represent the current state of the BucketAccess resource. Each condition has a unique type and reflects the status of a specific aspect of the resource. Standard condition types include: - "Available": the resource is fully functional - "Progressing": the resource is being created or updated - "Degraded": the resource failed to reach or maintain its desired state The status of each condition is one of True, False, or Unknown. |
BucketClaim
BucketClaim is the Schema for the bucketclaims API
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
apiVersion string | objectstorage.k8s.io/v1alpha2 | ||
kind string | BucketClaim | ||
kind string | Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds | ||
apiVersion string | APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources | ||
metadata ObjectMeta | Refer to Kubernetes API documentation for fields of metadata . | ||
spec BucketClaimSpec | spec defines the desired state of BucketClaim | ||
status BucketClaimStatus | status defines the observed state of BucketClaim |
BucketClaimList
BucketClaimList contains a list of BucketClaim
Field | Description | Default | Validation |
---|---|---|---|
apiVersion string | objectstorage.k8s.io/v1alpha2 | ||
kind string | BucketClaimList | ||
kind string | Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds | ||
apiVersion string | APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources | ||
metadata ListMeta | Refer to Kubernetes API documentation for fields of metadata . | ||
items BucketClaim array |
BucketClaimReference
BucketClaimReference is a reference to a BucketClaim object.
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
name string | name is the name of the BucketClaim being referenced. | MinLength: 1 | |
namespace string | namespace is the namespace of the BucketClaim being referenced. If empty, the Kubernetes 'default' namespace is assumed. namespace is immutable except to update '' to 'default'. | MinLength: 0 | |
uid UID | uid is the UID of the BucketClaim being referenced. Once set, the UID is immutable. |
BucketClaimSpec
BucketClaimSpec defines the desired state of BucketClaim
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
bucketClassName string | bucketClassName selects the BucketClass for provisioning the BucketClaim. This field is used only for BucketClaim dynamic provisioning. If unspecified, existingBucketName must be specified for binding to an existing Bucket. | ||
protocols ObjectProtocol array | protocols lists object storage protocols that the provisioned Bucket must support. If specified, COSI will verify that each item is advertised as supported by the driver. | ||
existingBucketName string | existingBucketName selects the name of an existing Bucket resource that this BucketClaim should bind to. This field is used only for BucketClaim static provisioning. If unspecified, bucketClassName must be specified for dynamically provisioning a new bucket. |
BucketClaimStatus
BucketClaimStatus defines the observed state of BucketClaim.
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
boundBucketName string | boundBucketName is the name of the Bucket this BucketClaim is bound to. Once set, this is immutable. | ||
readyToUse boolean | readyToUse indicates that the bucket is ready for consumption by workloads. | ||
protocols ObjectProtocol array | protocols is the set of protocols the bound Bucket reports to support. BucketAccesses can request access to this BucketClaim using any of the protocols reported here. | ||
error TimestampedError | error holds the most recent error message, with a timestamp. This is cleared when provisioning is successful. |
BucketClass
BucketClass defines a named "class" of object storage buckets. Different classes might map to different object storage protocols, quality-of-service levels, backup policies, or any other arbitrary configuration determined by storage administrators. The name of a BucketClass object is significant, and is how users can request a particular class.
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
apiVersion string | objectstorage.k8s.io/v1alpha2 | ||
kind string | BucketClass | ||
kind string | Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds | ||
apiVersion string | APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources | ||
metadata ObjectMeta | Refer to Kubernetes API documentation for fields of metadata . | ||
spec BucketClassSpec | spec defines the BucketClass. spec is entirely immutable. |
BucketClassList
BucketClassList contains a list of BucketClass
Field | Description | Default | Validation |
---|---|---|---|
apiVersion string | objectstorage.k8s.io/v1alpha2 | ||
kind string | BucketClassList | ||
kind string | Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds | ||
apiVersion string | APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources | ||
metadata ListMeta | Refer to Kubernetes API documentation for fields of metadata . | ||
items BucketClass array |
BucketClassSpec
BucketClassSpec defines the BucketClass.
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
driverName string | driverName is the name of the driver that fulfills requests for this BucketClass. | MinLength: 1 | |
deletionPolicy BucketDeletionPolicy | deletionPolicy determines whether a Bucket created through the BucketClass should be deleted when its bound BucketClaim is deleted. Possible values: - Retain: keep both the Bucket object and the backend bucket - Delete: delete both the Bucket object and the backend bucket | Enum: [Retain Delete] | |
parameters object (keys:string, values:string) | parameters is an opaque map of driver-specific configuration items passed to the driver that fulfills requests for this BucketClass. |
BucketDeletionPolicy
Underlying type: string
BucketDeletionPolicy configures COSI's behavior when a Bucket resource is deleted.
Validation:
- Enum: [Retain Delete]
Appears in:
Field | Description |
---|---|
Retain | BucketDeletionPolicyRetain configures COSI to keep the Bucket object as well as the backend bucket when a Bucket resource is deleted. |
Delete | BucketDeletionPolicyDelete configures COSI to delete the Bucket object as well as the backend bucket when a Bucket resource is deleted. |
BucketList
BucketList contains a list of Bucket
Field | Description | Default | Validation |
---|---|---|---|
apiVersion string | objectstorage.k8s.io/v1alpha2 | ||
kind string | BucketList | ||
kind string | Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds | ||
apiVersion string | APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources | ||
metadata ListMeta | Refer to Kubernetes API documentation for fields of metadata . | ||
items Bucket array |
BucketSpec
BucketSpec defines the desired state of Bucket
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
driverName string | driverName is the name of the driver that fulfills requests for this Bucket. | MinLength: 1 | |
deletionPolicy BucketDeletionPolicy | deletionPolicy determines whether a Bucket should be deleted when its bound BucketClaim is deleted. This is mutable to allow Admins to change the policy after creation. Possible values: - Retain: keep both the Bucket object and the backend bucket - Delete: delete both the Bucket object and the backend bucket | Enum: [Retain Delete] | |
parameters object (keys:string, values:string) | parameters is an opaque map of driver-specific configuration items passed to the driver that fulfills requests for this Bucket. | ||
protocols ObjectProtocol array | protocols lists object store protocols that the provisioned Bucket must support. If specified, COSI will verify that each item is advertised as supported by the driver. | ||
bucketClaim BucketClaimReference | bucketClaim references the BucketClaim that resulted in the creation of this Bucket. For statically-provisioned buckets, set the namespace and name of the BucketClaim that is allowed to bind to this Bucket. | ||
existingBucketID string | existingBucketID is the unique identifier for an existing backend bucket known to the driver. Use driver documentation to determine how to set this value. This field is used only for Bucket static provisioning. This field will be empty when the Bucket is dynamically provisioned from a BucketClaim. |
BucketStatus
BucketStatus defines the observed state of Bucket.
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
readyToUse boolean | readyToUse indicates that the bucket is ready for consumption by workloads. | ||
bucketID string | bucketID is the unique identifier for the backend bucket known to the driver. Once set, this is immutable. | ||
protocols ObjectProtocol array | protocols is the set of protocols the Bucket reports to support. BucketAccesses can request access to this BucketClaim using any of the protocols reported here. | ||
bucketInfo object (keys:string, values:string) | BucketInfo reported by the driver, rendered in the COSI_ BucketAccess Secret. e.g., COSI_S3_ENDPOINT, COSI_AZURE_STORAGE_ACCOUNT. This should not contain any sensitive information. | ||
error TimestampedError | Error holds the most recent error message, with a timestamp. This is cleared when provisioning is successful. |
CosiEnvVar
Underlying type: string
A CosiEnvVar defines a COSI environment variable that contains backend bucket or access info.
Vars marked "Required" will be present with non-empty values in BucketAccess Secrets.
Some required vars may only be required in certain contexts, like when a specific
AuthenticationType is used.
Some vars are only relevant for specific protocols.
Non-relevant vars will not be present, even when marked "Required".
Vars are used as data keys in BucketAccess Secrets.
Vars must be all-caps and must begin with COSI_
.
Appears in:
ObjectProtocol
Underlying type: string
ObjectProtocol represents an object protocol type.
Appears in:
Field | Description |
---|---|
S3 | ObjectProtocolS3 represents the S3 object protocol type. |
Azure | ObjectProtocolS3 represents the Azure Blob object protocol type. |
GCS | ObjectProtocolS3 represents the Google Cloud Storage object protocol type. |
TimestampedError
TimestampedError contains an error message with timestamp.
Appears in:
Field | Description | Default | Validation |
---|---|---|---|
time Time | time is the timestamp when the error was encountered. | ||
message string | message is a string detailing the encountered error. NOTE: message will be logged, and it should not contain sensitive information. |