GCP Cloud Storage is an object storage service provided by Google Cloud Platform. GCP takes care of managing the storage hardware, so you don’t have to configure servers upfront. It offers unlimited storage space for any file. It is a highly available, scalable, durable, and performant service. It can be used to backup and restore, disaster recovery, archive solutions. It further acts as the backbone for storage to many GCP services. It’s equivalent to Amazon’s S3. You can read more about AWS S3 HERE
You can refer to the GCS Cloud Storage notes below if you are planning to appear for various GCP certification exams like:
- GCP Associate Cloud Engineer
- GCP Professional Cloud Architect
- GCP Professional Cloud Developer
- GCP Professional Data Engineer
- GCP Professional Cloud DevOps Engineer
- GCP Professional Machine Learning Engineer
In this article, we’ll talk about the following topics in detail:
Cloud Storage Bucket Naming Convention
- Must be unique across the world.
- Must contain lowercase characters, numbers, underscore, dashes, or dots.
- The name should consist of 3-63 characters.
- Must starts and end with characters or numbers.
- Cant begin with the term “goog.”
- Cant have misspelled words for “google.”
- If it contains a dot, the length can be up to 222 characters. The string length between dots can not exceed 63 characters.
- Cant represent IP address as the bucket name.
Cloud Storage Location Type
- Supports three location types
- Multi-region cloud storage
- Dual region cloud storage
- Region cloud storage
- It defines how the data is placed geographically and affects the availability, cost, and performance.
- Multi-region storage type puts the data in multiple regions and offers the highest availability.
- The dual region storage class provides the highest availability and low latency across the two regions.
- The single region storage class offers the lowest latency in a single region.
Cloud Storage Class
- You can define the storage class for each object.
- The storage bucket will have a default storage class. If you don’t specify the object’s storage class, Cloud Storage will create the object using buckets default storage class.
- The cost varies for each storage class.
- You can change the storage class of the object explicitly by requiring the objects.
- You can change storage class using the Object Lifecycle Management policy.
- Cloud storage supports the following storage classes:
- Standard Storage
- Nearline Storage
- Coldline Storage
- Archive Storage
- Standard storage is best for frequently used data, and there is no requirement of minimum storage duration.
- Nearline storage is best when the data is accessed once a month, and the minimum storage duration is 30 days.
- Coldline storage is best when data is used once in a quarter and isn’t available immediately. There is the requirement of a minimum of 90 days storage duration for using cold line storage.
- Archive storage is best for archive storage. Unlike cold line storage, You can access the data with milliseconds of latency.
Cloud Storage Access Control
- Cloud storage provides the option to prevent public access to the bucket and the objects.
- If you enable the option to prevent public access, you cant host web site on the same bucket.
- Cloud Storage provides types of access control.
- Uniform access control ensures all objects within the bucket have the same access permissions (IAM).
- Uniform access control becomes permanent in 90 days.
- Fine-grained access control allows you to have different access controls for individual objects within the storage bucket.
- You can define ACL in addition to bucket level permission when using fine-grained access control.
- Google recommends using uniform access control.
Cloud Storage Object Versioning
- It helps to recover data in case of accidental deletion or update.
- You can retrieve deleted objects using object versions.
- You can delete the object versions permanently by including the generation number with the delete request.
- You can also delete older object versions using Object Lifecycle Management.
- If you disable object versioning, the existing object version stays as is.
- You can configure the max number of object versions while enabling the object versioning. It includes the liver version as well as past versions.
- You can also have a duration after which noon the current object version gets deleted automatically.
Cloud Storage Retention Policy
- You can apply a retention policy to a bucket if it’s not already applied.
- Once applied, the retention policy will be applicable for all objects in the bucket.
- You cant delete, modify the objects till the retention duration is over.
- You can remove the retention policy or increase and decrease retention duration only if the retention policy is not locked.
- Once the retention policy is locked, you cant change it. You cant increase, decrease the retention duration.
- You will need to delete the entire bucket to delete the retention policy once it is locked.
- You can either apply object versioning or retention policy.
Cloud Storage Lifecycle Rule
- The lifecycle rule helps change the object storage class or delete the noncurrent version of objects when certain conditions are met.
- While setting the Lifecycle rule, you can select one of the actions below:
- Set storage class to Nearline
- Set Storage class to Coldline
- Set Storage Class to Archive
- Delete Object
- You can set one or more conditions from the below options:
- Created before
- Storage class matches
- Number of newer versions
- Days since becoming noncurrent
- Became noncurrent before
- Live State
- Days since custom date
- Custom time before
Cloud Storage Data Encryption
- Cloud storage allows you to encrypt the data at rest.
- It provides two ways to manage the keys.
- Google managed encryption key doesn’t require any configuration from your side. Google will take care of managing and rotating the encryption key.
- You can also use a Customer-managed encryption key (CMEK) created using Google Cloud Key Management Service.
- Cloud storage will encrypt the data before storing and decrypt it before providing it to the end-user.
gsutil is a command-line tool that you can use to create, manage and delete Storage Buckets or upload or download objects to the Storage bucket. In this section, we will look at gsutil commands. The following section assumes that gsutil is already installed and configured on your system.
- List Buckets
- Make bucket
gsutil mb gs://<NEW_BUCKET_NAME>
- Delete Bucket
gsutil rb gs://<BUCKET_NAME>
- List object in the directory
gsutil ls gs://<BUCKET_NAME>/dir_name
- Move/Rename object
gsutil mv gs://<BUCKET_NAME>/<SOURCE> gs://<BUCKET_NAME>/<DEST>
- Copy local files to GCS bucket
gsutil cp local.txt gs://<BUCKET_NAME>/
- Download GCS object to the current directory
gsutil cp gs://<BUCKET_NAME>/object.txt .
- Delete GCS object
gsutil rm gs://<BUCKET_NAME>/object.ext
This article looked at GCS buckets, location type, storage classes, access control mechanism, object versioning, retention policy, encryption types, and some gsutil commands.