This directory contains samples for Google Cloud Storage. Cloud Storage allows world-wide storage and retrieval of any amount of data at any time. You can use Google Cloud Storage for a range of scenarios including serving website content, storing data for archival and disaster recovery, or distributing large data objects to users via direct download.
Before running the samples, make sure you've followed the steps outlined in Quick Start.
Refer to the Authentication Set Up Guide for more detailed instructions.
-
Clone this repository and change to the sample directory you want to use.
git clone https://github.com/googleapis/python-storage.git -
Activate a venv if you have not already from the Quick Start.
source <your-venv>/bin/activate -
Install the dependencies needed to run the samples.
cd samples/snippets pip install -r requirements.txt
Before running the tests, make sure you've followed the steps outlined in Setup.
We use nox to instrument our tests.
pip install nox
You can run tests locally using your own gcs project or with a valid service account in project python-docs-samples-tests. This outlines the workflow of running tests locally using your own gcs project.
Refer to noxfile_config.py and a list of environment variables that can be set manually. Not every test needs all of these variables.
Below outlines some common environment variables used in the storage samples.
See Other Resources on how to create credentials, keys, and secrets.
export GOOGLE_CLOUD_PROJECT=[your-project-name]
export MAIN_GOOGLE_CLOUD_PROJECT=[your-project-name]
export BUILD_SPECIFIC_GCLOUD_PROJECT=[your-project-name]
export HMAC_KEY_TEST_SERVICE_ACCOUNT=[your-service-account]
export CLOUD_KMS_KEY=[your-kms-key]
export GOOGLE_APPLICATION_CREDENTIALS=[your-credentials]
If you are running a single test locally that does not use the environment variables, you can delete the noxfile_config.py file and simply set your GOOGLE_CLOUD_PROJECT
export GOOGLE_CLOUD_PROJECT=[your-project-name]
nox -s lint
nox -s py-3.9 -- snippets_test.py
nox -s py-3.9 -- snippets_test.py::test_list_blobs
There are restrictions on the testing projects used in Kokoro. For instance,
we change the service account based on different test sessions to avoid
hitting the maximum limit of HMAC keys on a single service account.
Another example is requester_pays_test.py needs to use a different Storage bucket, and looks for an environment variable REQUESTER_PAYS_TEST_BUCKET.
Please refer to noxfile_config.py , kokoro configs, and test files to see if there are special test configurations required.
List of Samples
- Activate HMAC Key
- Batch Request
- Add Bucket Conditional IAM Binding
- Add Bucket Default Owner
- Add Bucket IAM Member
- Add Bucket Label
- Add Bucket Owner
- Add File Owner
- Bucket Delete Default KMS Key
- Change Default Storage Class
- Change File Storage Class
- Compose File
- Configure Retries
- Copy File
- Copy File Archived Generation
- CORS Configuration
- Create Bucket
- Create Bucket Class Location
- Create Bucket Dual Region
- Create Bucket Notifications
- Create Bucket Turbo Replication
- Create HMAC Key
- Deactivate HMAC Key
- Define Bucket Website Configuration
- Delete Bucket
- Delete Bucket Notification
- Delete File
- Delete File Archived Generation
- Delete HMAC Key
- Disable Bucket Lifecycle Management
- Disable Default Event Based Hold
- Disable Requester Pays
- Disable Uniform Bucket Level Access
- Disable Versioning
- Download Byte Range
- Download Encrypted File
- Download File
- Download File Requester Pays
- Download Into Memory
- Download Public File
- Enable Bucket Lifecycle Management
- Enable Default Event Based Hold
- Enable Requester Pays
- Enable Uniform Bucket Level Access
- Enable Versioning
- FileIO Write-Read
- FileIO Pandas
- Generate Encryption Key
- Generate Signed Post Policy V4
- Generate Signed Url V2
- Generate Signed Url V4
- Generate Upload Signed Url V4
- Get Bucket Labels
- Get Bucket Metadata
- Get Default Event Based Hold
- Get HMAC Key
- Get Metadata
- Get Public Access Prevention
- Get Requester Pays Status
- Get Retention Policy
- Get RPO
- Get Service Account
- Get Uniform Bucket Level Access
- List Buckets
- List Bucket Notifications
- List File Archived Generations
- List Files
- List Files With Prefix
- List HMAC Keys
- Lock Retention Policy
- Make Public
- Move File
- Object CSEK To CMEK
- Object Get KMS Key
- Print Bucket ACL
- Print Bucket ACL For User
- Print File ACL
- Print File ACL For User
- Print PubSub Bucket Notification
- Release Event Based Hold
- Release Temporary Hold
- Remove Bucket Conditional IAM Binding
- Remove Bucket Default Owner
- Remove Bucket IAM Member
- Remove Bucket Label
- Remove Bucket Owner
- Remove Cors Configuration
- Remove File Owner
- Remove Retention Policy
- Rename File
- Rotate Encryption Key
- Set Bucket Default KMS Key
- Set Bucket Public IAM
- Set Event Based Hold
- Set Metadata
- Set Public Access Prevention Enforced
- Set Public Access Prevention Inherited
- Set RPO Async Turbo
- Set RPO Default
- Set Retention Policy
- Set Temporary Hold
- Upload Encrypted File
- Upload File
- Upload From Memory
- Upload With KMS Key
- View Bucket IAM Members
View the source code. To run this sample:
python storage_activate_hmac_key.py <ACCESS_ID> <PROJECT_ID>
View the source code. To run this sample:
python storage_batch_request.py <BUCKET_NAME> <PREFIX>
View the source code. To run this sample:
python storage_add_bucket_conditional_iam_binding.py <BUCKET_NAME> <ROLE> <TITLE> <DESCRIPTION> <EXPRESSION> <MEMBERS>
View the source code. To run this sample:
python storage_add_bucket_default_owner.py <BUCKET_NAME> <USER_EMAIL>
View the source code. To run this sample:
python storage_add_bucket_iam_member.py <BUCKET_NAME> <ROLE> <MEMBER>
View the source code. To run this sample:
python storage_add_bucket_label.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_add_bucket_owner.py <BUCKET_NAME> <USER_EMAIL>
View the source code. To run this sample:
python storage_add_file_owner.py <BUCKET_NAME> <BLOB_NAME> <USER_EMAIL>
View the source code. To run this sample:
python storage_bucket_delete_default_kms_key.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_change_default_storage_class.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_change_file_storage_class.py <BUCKET_NAME> <BLOB_NAME>
View the source code. To run this sample:
python storage_compose_file.py <BUCKET_NAME> <FIRST_BLOB_NAME> <SECOND_BLOB_NAME> <DESTINATION_BLOB_NAME>
View the source code. To run this sample:
python storage_configure_retries.py <BUCKET_NAME> <BLOB_NAME>
View the source code. To run this sample:
python storage_copy_file.py <BUCKET_NAME> <BLOB_NAME> <DESTINATION_BUCKET_NAME> <DESTINATION_BLOB_NAME>
View the source code. To run this sample:
python storage_copy_file_archived_generation.py <BUCKET_NAME> <BLOB_NAME> <DESTINATION_BUCKET_NAME> <DESTINATION_BLOB_NAME> <GENERATION>
View the source code. To run this sample:
python storage_cors_configuration.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_create_bucket.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_create_bucket_class_location.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_create_bucket_dual_region.py <BUCKET_NAME> <LOCATION> <REGION_1> <REGION_2>
View the source code. To run this sample:
python storage_create_bucket_notifications.py <BUCKET_NAME> <TOPIC_NAME>
View the source code. To run this sample:
python storage_create_bucket_turbo_replication.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_create_hmac_key.py <PROJECT_ID> <SERVICE_ACCOUNT_EMAIL>
View the source code. To run this sample:
python storage_deactivate_hmac_key.py <ACCESS_ID> <PROJECT_ID>
View the source code. To run this sample:
python storage_define_bucket_website_configuration.py <BUCKET_NAME> <MAIN_PAGE_SUFFIX> <NOT_FOUND_PAGE>
View the source code. To run this sample:
python storage_delete_bucket.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_delete_bucket_notification.py <BUCKET_NAME> <NOTIFICATION_ID>
View the source code. To run this sample:
python storage_delete_file.py <BUCKET_NAME> <BLOB_NAME>
View the source code. To run this sample:
python storage_delete_file_archived_generation.py <BUCKET_NAME> <BLOB_NAME> <GENERATION>
View the source code. To run this sample:
python storage_delete_hmac_key.py <ACCESS_ID> <PROJECT_ID>
View the source code. To run this sample:
python storage_disable_bucket_lifecycle_management.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_disable_default_event_based_hold.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_disable_requester_pays.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_disable_uniform_bucket_level_access.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_disable_versioning.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_download_byte_range.py <BUCKET_NAME> <SOURCE_BLOB_NAME> <START_BYTE> <END_BYTE> <DESTINATION_FILE_NAME> <>BASE64_ENCRYPTION_KEY
View the source code. To run this sample:
python storage_download_encrypted_file.py <BUCKET_NAME> <SOURCE_BLOB_NAME> <DESTINATION_FILE_NAME> <>BASE64_ENCRYPTION_KEY
View the source code. To run this sample:
python storage_download_file.py <BUCKET_NAME> <SOURCE_BLOB_NAME> <DESTINATION_FILE_NAME>
View the source code. To run this sample:
python storage_download_file_requester_pays.py <BUCKET_NAME> <PROJECT_ID> <SOURCE_BLOB_NAME> <DESTINATION_FILE_NAME>
View the source code. To run this sample:
python storage_download_into_memory.py <BUCKET_NAME> <BLOB_NAME>
View the source code. To run this sample:
python storage_download_public_file.py <BUCKET_NAME> <SOURCE_BLOB_NAME> <DESTINATION_FILE_NAME>
View the source code. To run this sample:
python storage_enable_bucket_lifecycle_management.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_enable_default_event_based_hold.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_enable_requester_pays.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_enable_uniform_bucket_level_access.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_enable_versioning.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_fileio_write_read.py <BUCKET_NAME> <BLOB_NAME>
View the source code. To run this sample:
python storage_fileio_pandas.py <BUCKET_NAME> <BLOB_NAME>
View the source code. To run this sample:
python storage_generate_encryption_key.py
View the source code. To run this sample:
python storage_generate_signed_post_policy_v4.py <BUCKET_NAME> <BLOB_NAME>
View the source code. To run this sample:
python storage_generate_signed_url_v2.py <BUCKET_NAME> <BLOB_NAME>
View the source code. To run this sample:
python storage_generate_signed_url_v4.py <BUCKET_NAME> <BLOB_NAME>
View the source code. To run this sample:
python storage_generate_upload_signed_url_v4.py <BUCKET_NAME> <BLOB_NAME>
View the source code. To run this sample:
python storage_get_bucket_labels.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_get_bucket_metadata.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_get_default_event_based_hold.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_get_hmac_key.py <ACCESS_ID> <PROJECT_ID>
View the source code. To run this sample:
python storage_get_metadata.py <BUCKET_NAME> <BLOB_NAME>
View the source code. To run this sample:
python storage_get_public_access_prevention.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_get_requester_pays_status.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_get_retention_policy.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_get_rpo.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_get_service_account.py
View the source code. To run this sample:
python storage_get_uniform_bucket_level_access.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_list_buckets.py
View the source code. To run this sample:
python storage_list_bucket_notifications.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_list_file_archived_generations.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_list_files.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_list_files_with_prefix.py <BUCKET_NAME> <PREFIX>
View the source code. To run this sample:
python storage_list_hmac_keys.py <PROJECT_ID>
View the source code. To run this sample:
python storage_lock_retention_policy.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_make_public.py <BUCKET_NAME> <BLOB_NAME>
View the source code. To run this sample:
python storage_move_file.py <BUCKET_NAME> <BLOB_NAME> <DESTINATION_BUCKET_NAME> <DESTINATION_BLOB_NAME>
View the source code. To run this sample:
python storage_object_csek_to_cmek.py <BUCKET_NAME> <BLOB_NAME> <ENCRYPTION_KEY> <KMS_KEY_NAME>
View the source code. To run this sample:
python storage_object_get_kms_key.py <BUCKET_NAME> <BLOB_NAME>
View the source code. To run this sample:
python storage_print_bucket_acl.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_print_bucket_acl_for_user.py <BUCKET_NAME> <USER_EMAIL>
View the source code. To run this sample:
python storage_print_file_acl.py <BUCKET_NAME> <BLOB_NAME>
View the source code. To run this sample:
python storage_print_file_acl_for_user.py <BUCKET_NAME> <BLOB_NAME> <USER_EMAIL>
View the source code. To run this sample:
python storage_print_pubsub_bucket_notification.py <BUCKET_NAME> <NOTIFICATION_ID>
View the source code. To run this sample:
python storage_release_event_based_hold.py <BUCKET_NAME> <BLOB_NAME>
View the source code. To run this sample:
python storage_release_temporary_hold.py <BUCKET_NAME> <BLOB_NAME>
View the source code. To run this sample:
python storage_remove_bucket_conditional_iam_binding.py <BUCKET_NAME> <ROLE> <TITLE> <DESCRIPTION> <EXPRESSION>
View the source code. To run this sample:
python storage_remove_bucket_default_owner.py <BUCKET_NAME> <USER_EMAIL>
View the source code. To run this sample:
python storage_remove_bucket_iam_member.py <BUCKET_NAME> <ROLE> <MEMBER>
View the source code. To run this sample:
python storage_remove_bucket_label.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_remove_bucket_owner.py <BUCKET_NAME> <USER_EMAIL>
View the source code. To run this sample:
python storage_remove_cors_configuration.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_remove_file_owner.py <BUCKET_NAME> <BLOB_NAME> <USER_EMAIL>
View the source code. To run this sample:
python storage_remove_retention_policy.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_rename_file.py <BUCKET_NAME> <BLOB_NAME> <NEW_NAME>
View the source code. To run this sample:
python storage_rotate_encryption_key.py <BUCKET_NAME> <BLOB_NAME> <BASE64_ENCRYPTION_KEY> <BASE64_NEW_ENCRYPTION_KEY>
View the source code. To run this sample:
python storage_set_bucket_default_kms_key.py <BUCKET_NAME> <KMS_KEY_NAME>
View the source code. To run this sample:
python storage_set_bucket_public_iam.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_set_event_based_hold.py <BUCKET_NAME> <BLOB_NAME>
View the source code. To run this sample:
python storage_set_metadata.py <BUCKET_NAME> <BLOB_NAME>
View the source code. To run this sample:
python storage_set_public_access_prevention_enforced.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_set_public_access_prevention_inherited.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_set_retention_policy.py <BUCKET_NAME> <RETENTION_PERIOD>
View the source code. To run this sample:
python storage_set_rpo_async_turbo.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_set_rpo_default.py <BUCKET_NAME>
View the source code. To run this sample:
python storage_set_temporary_hold.py <BUCKET_NAME> <BLOB_NAME>
View the source code. To run this sample:
python storage_upload_encrypted_file.py <BUCKET_NAME> <SOURCE_FILE_NAME> <DESTINATION_BLOB_NAME> <BASE64_ENCRYPTION_KEY>
View the source code. To run this sample:
python storage_upload_file.py <BUCKET_NAME> <SOURCE_FILE_NAME> <DESTINATION_BLOB_NAME>
View the source code. To run this sample:
python storage_upload_from_memory.py <BUCKET_NAME> <CONTENTS> <DESTINATION_BLOB_NAME>
View the source code. To run this sample:
python storage_upload_with_kms_key.py <BUCKET_NAME> <SOURCE_FILE_NAME> <DESTINATION_BLOB_NAME> <KMS_KEY_NAME>
View the source code. To run this sample:
python storage_view_bucket_iam_members.py <BUCKET_NAME>