# Working with Buckets in CloudDisk
A bucket is a container for storing objects. To store an object, you create a bucket and then upload it to a bucket.
A CloudDisk bucket is just a mapping of a storage target, and it can have different names. Multiple CloudDisk buckets can correspond to the same storage target.
## Create a Bucket
Before creating a bucket, make sure that you already have a bucket on the cloud, otherwise you need to [apply for it](#apply-for-a-bucket) first.
### Apply for a Bucket
note
We provide a bucket of type *non-prod testing* in non-production environments for testing. It has the following limitations:
- Objects are retained only for about one year.
- Multipart upload is not supported.
- Object tags are not supported. Tags passed in `X-Mo-Tag` on upload are ignored, and the tag APIs are not available.
- Listing object versions is not supported.
- Pre-signed URLs are only available as insureMO links; no native link of the underlying storage is returned.
See [Supported Actions by Storage](#supported-actions-by-storage) for the full list. If you need any of the features above, you still need to apply for a separate bucket on the cloud.
Storage resources on the cloud need to be created manually, so you need to submit an application to create a bucket on the cloud.
Send an email to *siteops@insuremo.com* with below form:
| Title | Example | Description |
| ---------------------- | ---------------------------------------------------- | --------------------------------------------------------------------------------- |
| **Project Code** | *1234567 XXX project* | - |
| **Bucket Name** | *imo-kic-ptdev-tenant-clouddisk-bucket* | A bucket name formatted with {env_name}-{tenant/workspace_name}-clouddisk-bucket. |
| **Site / Env** | *ptdev* | The site or environment that the bucket will be used on. |
| **Type of Documents** | *Policy PDF* | Briefly describe the documents that will be stored. |
| **Size of Documents** | *1Mb per document
20Mb per day
20Gb for all* | Estimated document size. |
| **Other Requirements** | *None* | Such as enable versioning, configure lifecycle rule, specify the region. |
### Naming a Bucket
Unlike the bucket name in the cloud, the bucket name in CloudDisk is more like an alias. A bucket name must be unique within a tenant. Different tenants can use the same bucket name.
The following naming rules apply to general-purpose buckets.
- Bucket names must be between 6 (minimum) and 63 (maximum) characters long.
- Bucket names can consist only of lowercase letters, numbers, underscores (`_`), and hyphens (`-`).
- Bucket names must begin with a letter and end with a letter or number.
- Bucket names must not start with the prefix `mo-`.
- A bucket name must be unique within a tenant.
### Bucket Integrate with Cloud Storages
CloudDisk supports the following cloud storages:
- [Amazon S3](https://aws.amazon.com/s3/)
- S3-compatible storage (MinIO, Ceph, OCI Storage, Google Cloud Storage, etc)
- [Alibaba Cloud OSS](https://www.aliyun.com/product/oss)
- [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs/)
- SFTP (Private Cloud Only)
- [Microsoft Graph](https://learn.microsoft.com/graph/overview) (`msgraph`)
- SFTP (Private Cloud Only)
- Database (Private Cloud Only)
#### Properties by Storage
Property descriptions for each cloud storage. *Required* means the property must be set when creating the bucket.
| Property | S3 | OSS | Azblob | Msgraph | SFTP |
| --------------------------------- | -------------------------------------- | --------------------------- | ----------------------------------------------------------------- | ----------------------------- | ------------------------------------ |
| `source_type` | `s3` | `oss` | `azblob` | `msgraph` | `sftp` |
| `source_bucket` | S3 Bucket Name. *Required* | OSS Bucket Name. *Required* | Container. *Required* | Drive ID. *Required* | - |
| `source_user` | Access Key | Access Key. *Required* | Account Name | Client ID. *Required* | Username. *Required* |
| `source_secret` | Secret Key | Secret Key. *Required* | Account Key / Connection String. *Required* | Client Secret. *Required* | Password. *Required* |
| `source_region` | Region, e.g. `us-east-1` | Region, e.g. `cn-shanghai`. *Required* | Client Type, *connection-string* (default) or *shared-key-cred* | - | - |
| `source_endpoint` | Endpoint | Endpoint, e.g. `oss-cn-shanghai.aliyuncs.com` | Service Url, e.g. `https://.blob.core.windows.net` | - | Server address, e.g. `127.0.0.1:22`. *Required* |
| `source_prefix` | Object prefix | Object prefix | Blob prefix | Folder prefix | Directory prefix |
| `source_properties.tenant_id` | - | - | - | Azure AD Tenant ID. *Required* | - |
A *Database* bucket (`source_type`: `database`) stores objects in the database of the private cloud site and needs no property other than `source_type`.
The S3 storage also accepts the following optional properties in `source_properties`:
| Property | Description |
| -------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| `aws_use_path_style` | *true* means use path style, *false* means use virtual-hosted style. Default *false*. |
| `aws_role_arn` | ARN of an IAM role to assume. When set, CloudDisk accesses the bucket with the temporary credentials of this role. |
| `aws_role_region` | Region of the STS endpoint used to assume `aws_role_arn`. |
| `aws_retry_max_attempts` | Maximum number of attempts for each request to the storage, including the first one. Default `1`, which means no retry. |
| `aws_checksum_calculation` | When to calculate and validate request/response checksums: *when_supported* or *when_required*. Leave it empty to use the default of the SDK. Set *when_required* if your S3-compatible storage rejects checksum headers. |
For MinIO bucket, region should be `us-east-1` and use path style should be `true`.
For OCI storage bucket, endpoint should contain namespace and use path style should be `true`.
#### Supported Actions by Storage
Not every storage supports every action. Calling an unsupported action returns the error code `e_clouddisk_source_action_not_support`. See [CloudDisk Error Codes](./InsureMO_clouddisk_error_codes.md) for all error codes.
| Action | S3 | OSS | Azblob | Msgraph | SFTP | Database | Non-prod testing |
| ------------------------------------------------------ | -- | --- | ------ | ------- | ---- | -------- | ---------------- |
| List Objects | ✓ | ✓ | ✓ | ✓ | ✓ ¹ | ✓ | ✓ |
| List Object Versions | ✓ | ✓ | ✓ | ✓ ² | ✗ | ✗ | ✗ |
| Get Object | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Put Object | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Delete Object | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Delete Objects (batch) | ✓ ³ | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ ³ |
| Exist Object | ✓ | ✓ | ✗ | ✓ | ✗ | ✓ | ✓ |
| Copy Object | ✓ | ✓ | ✗ | ✓ | ✗ | ✗ | ✓ |
| Get Object Metadata | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ | ✓ |
| Get / Put / Delete Object Tags | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ |
| Multipart Upload (create / upload part / complete / abort) | ✓ | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ |
| Presign Object - insureMO link | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Presign Object - native link of the storage | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ |
¹ SFTP returns at most 200 objects per request and does not support paging with `continuation_token`.
² Msgraph returns only the current version of each object.
³ Some S3-compatible storages, such as MinIO, may not support batch deletion. Delete objects one by one with *Delete Object* instead. The *non-prod testing* bucket may also be backed by such a storage in some environments.
The *S3* column describes Amazon S3. An S3-compatible storage (MinIO, Ceph, OCI Storage, Google Cloud Storage, etc.) may not fully support every action or header listed for S3; verify the actions you rely on against your storage before going live.
#### Supported Headers by Storage
Headers that a storage does not support are ignored; the request itself still succeeds.
| Header | Used on | S3 | OSS | Azblob | Msgraph | SFTP | Database |
| --------------------- | ------------ | -- | --- | ------ | ------- | ---- | -------- |
| `Cache-Control` | Put Object | ✓ | ✓ | ✓ | ✗ | ✗ | ✓ |
| `Content-Disposition` | Put Object | ✓ | ✓ | ✓ | ✗ | ✗ | ✓ |
| `Content-Encoding` | Put Object | ✓ | ✓ | ✓ | ✗ | ✗ | ✓ |
| `Content-Language` | Put Object | ✓ | ✓ | ✓ | ✗ | ✗ | ✓ |
| `Content-Type` | Put Object | ✓ | ✓ | ✓ | ✓ | ✗ | ✓ |
| `X-Mo-Meta-Xxx` | Put Object | ✓ | ✓ | ✓ | ✗ | ✗ | ✓ |
| `X-Mo-Tag` | Put Object | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ |
| `Range` | Get Object | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ |
| `If-None-Match` | Get Object | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ |
| `If-Modified-Since` | Get Object | ✓ | ✓ | ✓ | ✗ | ✗ | ✗ |
A *non-prod testing* bucket supports the same headers as the storage behind it, except that `X-Mo-Tag` is ignored.
### Operations
Using insureMO Portal (Console)
1. Log in to insureMO Portal, [find CloudDisk card](./InsureMO_clouddisk_welcome.md#find-clouddisk-on-insuremo-portal) or open https://portal.insuremo.com/#/home/non-insurance-services/clouddisk-v3.
2. Click **Create**.
3. Fill the form in **Create Bucket** dialog and click **Submit**.
Using insureMO API
You can call [CreateBucket](https://docs.insuremo.com/tech/pro/sdk/api/clouddisk?id=CreateBucket) API to create a bucket.
Below is an example of using `curl` command to create a bucket.
```sh
# set gateway url and access token
insuremo_gw_url="https://portal-gw.insuremo.com" \
insuremo_access_token="Bearer your-access-token" \
curl -X POST \
"$insuremo_gw_url/mo-fo/1.0/clouddisk/v3/bucket/add-one" \
-H "Content-Type: application/json" \
-H "Authorization: $insuremo_access_token" \
--data '{
"description": "description of this bucket",
"name": "your-bucket-name",
"source_type": "default",
// options ...
}'
```
Using insureMO SDK for JAVA
```java
import com.insuremo.sdk.core.client.SDKClient;
import com.insuremo.sdk.core.exception.ApiException;
import com.insuremo.sdk.services.clouddisk.ClouddiskSdkClient;
import com.insuremo.sdk.services.clouddisk.bundle.CreateBucketResponseBundle;
import com.insuremo.sdk.services.clouddisk.model.CreateBucketRequest;
public class BucketApi_createBucket {
public static void main(String[] args) {
System.out.println("----- Example for BucketApi.createBucket -----");
System.out.println("----- building api -----");
// spec gateway url, default http://gateway.mo-fo
String gateway = "https://portal-gw.insuremo.com";
String accessToken = "your-access-token";
SDKClient sdkClient = SDKClient.builder().
defaultConfigs().
gateway(gateway).
accessToken(accessToken).
build();
ClouddiskSdkClient client = new ClouddiskSdkClient(sdkClient);
System.out.println("----- createBucket -----");
try {
CreateBucketResponseBundle responseBundle = client.bucketApi().newCreateBucketRequestBuilder().
body(new CreateBucketRequest().
sourceType("default").
name("sdk-example").
description("for sdk example testing")).
doRequest();
System.out.println(responseBundle.getBody());
} catch (ApiException e) {
System.out.println("error when call api");
System.out.println(e.getResponseBody());
System.out.println(e.getCode());
throw e;
}
System.out.println("----- Done. -----");
}
}
```
## Delete a Bucket
Deleting a bucket in CloudDisk does not actually delete the bucket on cloud storage. This means that if you re-create the same bucket, you will also be able to see the previously uploaded objects.
If you want to delete a bucket and objects on the cloud, you need to call the API `Delete Objects` [/clouddisk/v3/object/delete](https://docs.insuremo.com/tech/pro/sdk/api/clouddisk?id=DeleteObject) to delete objects first and then delete the bucket on the cloud.
If you want to delete a bucket in CloudDisk that is used for non-production testing, you need to delete the objects in it first, and then delete the bucket.
Using insureMO Portal (Console)
Find the bucket you will delete, and click **Delete** on the right side. Click **Ok** for double-checking.
