S3 object storage

S3-compatible buckets on our own Ceph storage, billed hourly on the quota you set. This page covers how to create one, how the keys work, and every limit we could find in the code that enforces them.

What it is

A bucket is object storage that speaks the S3 API, served by Ceph RADOS Gateway. Each bucket gets its own access key and secret key, and a quota in GB that you choose when you create it and can change later. You reach it with any S3 client or SDK by pointing it at our endpoint and signing with Signature Version 4. Billing is hourly against your prepaid balance and is calculated from the quota you set, not from the bytes you actually store.

Getting started

  1. Create an account and confirm your email address. While email verification is pending, no cloud resource can be created, buckets included.
  2. Add funds. Creation is refused with "Insufficient funds" unless your balance covers at least the first hour of the bucket you asked for.
  3. Open Cloud, then Buckets in the panel, at /panel/cloud/buckets/, and use Add bucket. Pick a name, a quota in GB, and whether the bucket is publicly readable. Through the API the same call is POST /api/cloud/buckets/ with name, quota_gb and public_read; it answers 202 and provisions in the background.
  4. Wait for the bucket to leave the provisioning state. Provisioning creates the bucket, its own storage user and its key pair, and applies the public-read policy if you asked for one.
  5. Read your credentials. On the bucket page in the panel use Reveal credentials, or call POST /api/cloud/buckets/{id}/credentials/reveal/. The answer carries the bucket name, the endpoint, the region, the access key and the secret key, and is sent with Cache-Control: no-store so nothing caches it.
  6. Point your S3 client at https://s3.pidginhost.cloud with region us-east-1 and the key pair you just read. Anything that speaks S3 works: the AWS CLI, boto3, rclone, s3cmd.
  7. List what you have with GET /api/cloud/buckets/, or from the bucket list in the panel.

The name you type is not the final bucket name. Your account id is prefixed to it, so a bucket you call photos is created as c42-photos if your account id is 42. That prefixed name is the one your S3 client must use, and it is the one the reveal call returns.

Limits

These apply per account unless the row says otherwise. Where a value is a platform-wide setting rather than a per-account quota, the last column says so, because there is nothing support can raise for you alone.

Limit Value Can support lift it?
Buckets per account 5 Yes, on request
Stored data per account No separate cap on total bytes. Your ceiling is the bucket limit above times the quota of each bucket Yes, indirectly: support raises the bucket limit, you resize the buckets
Size of one bucket The quota you choose, within the range and step the create form offers You resize it yourself, in the panel or with POST /api/cloud/buckets/{id}/resize/
Shrinking a bucket below what it already stores Refused No, delete objects first
Bucket name rules 3 to 63 characters including the account prefix, lowercase letters, digits and hyphens only, first and last character a letter or a digit, no dots No, the name has to stay valid for the storage layer
Credential reveals through the API 10 per hour No, it is a platform-wide setting
Credential rotations through the API 5 per hour No, it is a platform-wide setting
Pre-signed URLs generated from the panel 60 per hour per user No, it is a platform-wide setting
Lifetime of a pre-signed URL Between 60 seconds and 7 days No, it is fixed in the panel
File upload through the panel file browser 100 MB per file No, use an S3 client for larger objects, which is not subject to this cap
Objects deleted in one panel request 1000 objects No, delete in batches
CORS rules per bucket 100 rules No
Lifecycle rules per bucket 1000 rules No

Keys and credentials

  • One access key and secret key pair is created per bucket, at provisioning time, by the storage layer. You never choose them.
  • The secret key is stored encrypted and is shown only when you ask for it, through Reveal credentials in the panel or POST /api/cloud/buckets/{id}/credentials/reveal/.
  • Rotation replaces the pair. Use Rotate credentials in the panel or POST /api/cloud/buckets/{id}/credentials/rotate/. The new pair is saved first, then the old access key is deleted at the storage layer, so anything still using the old key stops working. Update your clients before you rotate.
  • Every reveal, every rotation and every pre-signed URL is recorded with the user, the impersonating administrator if there was one, the IP address, the browser user agent and the API token prefix. The last few entries are shown on the bucket page.

What the panel does that the API does not

The REST API covers listing, creating, reading, resizing, public-read visibility, credential reveal and rotation, and cancelling a bucket. The rest is panel only, on the bucket page:

  • Turning object versioning on and off.
  • Editing CORS rules and lifecycle rules.
  • Browsing, uploading and deleting objects.
  • Generating a pre-signed URL for a single object.

Everything in that list is also available directly through the S3 API on the bucket itself, with your own key pair and your own client. The panel is a convenience, not the only way in.

What happens if your balance runs out

  • Cloud resources are suspended 2 days after the balance goes negative. For a bucket, suspension disables its storage user and removes the public-read policy, so the keys stop working and public objects stop being served. The data stays.
  • They are destroyed 7 days after that point if the balance has not been topped up. For a bucket, that deletes the bucket and everything in it.
  • A final warning email goes out 2 days before anything is destroyed.
  • Topping up restores the service, re-enables the storage user and puts the public-read policy back if the bucket had one.
  • Destruction is not recoverable. There is no backup of a destroyed bucket on our side.

What needs support

  • Raising the number of buckets your account may hold. The limit in the table above is a per-account setting, and we change it on request.
  • Offering a quota outside the range the create form shows. The minimum, the maximum and the step come from the product catalogue, and both the panel and the API refuse anything outside them.
  • Anything to do with a bucket that has been destroyed. The bucket, its keys and its objects are gone, and we cannot bring them back, which is why the warning email matters.
  • Recovering a key pair you rotated away from. The previous access key is deleted at the storage layer and cannot be restored; rotate again and update your clients.
  • Nothing needs support for the rate limits on reveals, rotations and pre-signed URLs. Those are platform-wide settings, not per-account quotas, so there is no per-account switch to flip.

Where to go next

Something here still not clear?

If a limit on this page surprised you, tell us. That is a documentation bug and we fix it.