BigQuery - Check if table already exists

I have a dataset in BigQuery. This dataset contains multiple tables. I am doing the following steps programmatically using the BigQuery API: Querying the tables in the dataset - Since my response...

Google Cloud Storage: How Much Total Space Am I Using?

I am using Google Cloud Storage. Where can I determine my total space used? I can't seem to find where in the dashboard it tells you the total usage in megabytes or gigabytes.

Force download on GCS via App Engine using Signed URL

I get my file via: require_once 'google/appengine/api/cloud_storage/CloudStorageTools.php'; use google\appengine\api\cloud_storage\CloudStorageTools; $public_link =...

NoSuchKey Error While Downloading File From Bucket

I tried to download a file from the bucket but I got an error like this (Signed URL link) NOT FOUND: <?xml version='1.0' encoding='UTF-8'?><Error><Code>NoSuchKey</Code><Message>The specified key...

How to get list of folders in a given bucket using Google Cloud API

I wanted to get all the folders inside a given Google Cloud bucket or folder using Google Cloud Storage API. For example if gs://abc/xyz contains three folders gs://abc/xyz/x1, gs://abc/xyz/x2 and...

ResumableUploadAbortException on gsutil

I keep seeing the following error when using gsutil ResumableUploadAbortException: Upload complete with 6275 additional bytes left in stream The command is very simple, something like gsutil cp...

Google cloud storage gzip compression, wrong headers

Just started using google cloud storage. I am having problem receiving gzip files from it. Stored file is gzip'ed but it gets decompressed on request. I have read all documentation, forums and...

Writing data to google cloud storage using python

I cannot find a way to to write a data set from my local machine into the google cloud storage using python. I have researched a a lot but didn't find any clue regarding this. Need help, thanks

Custom 301 redirects for static site using GCP Storage

I am considering whether to host my static website on an AWS S3 bucket or using Google Cloud Platform Storage, and, as I already use GCP and with it being generally cheaper, I would really like to...

ImportError: No module named google.oauth2

I wanted to add a google.cloud.storage dependency to my project so I tried to install this dependency with pip install --upgrade google-cloud-storage Running my app again with dev_appserver, it...

Comparison of loading from different file formats in BigQuery

We currently load most of our data into BigQuery either via csv or directly via the streaming API. However, I was wondering if there were any benchmarks available (or maybe a Google engineer could...

How do I switch to another serviceAccountKey.json via .firebaserc on Firebase Cloud Functions?

In my .firebaserc file, I provided two Firebase databases, one for dev and one for production: { "projects": { "default": "db-dev", "production": "db-prod" } } So when I'm deploying...

Docker build taking too long when installing grpcio via pip

I have a Dockerfile which installs a few packages via pip. Some of them are requiring grpcio, and it takes a few minutes only to build this part. Does anyone have a tip to speed up this...

SigningError with Firebase getSignedUrl()

I'm trying to use file.getSignedUrl() to get the download URL from Firebase Storage via Google Cloud Functions (Nodejs). I'm getting this error in the Cloud Functions console: { SigningError: A...

Is there an upload limit on Firebase?

I can't seem to upload files more than 5mb in firebase storage using firebase function with busboy. the error returns: ResumableUploadError: A resumable upload could not be performed....

Cloud Run Request Limit

Currently, Cloud Run has a request limit of 32 Mb per request, which makes it impossible to upload files like videos (which placed with no changes to GCP Storage). Meantime All Quotas page doesn't...

How to authenticate to GCP from a containerized Dockerfile

I am trying to build a new Docker image dynamically using a Cloud Build trigger job, however I fail to see how to safely retrieve my credentials to authenticate against GCP with a service...

What's the difference between Google Cloud Spanner and Cloud SQL?

I am novice in GCP stack so I am so confused about amount GCP technologies for storing data: https://cloud.google.com/products/storage Although google cloud spanner is not mentioned in the...

Firebase Cloud Storage: resource: the server responded with a status of 403 ()

I make a simple audio recording web app using Firebase Hosting. I would like to record audio on browser and upload it to Cloud Storage. When I deploy and access my app, I can record audio. However...

Google Cloud Container Registry/Artifact Registry Permissions

I'm trying to push containers to the Google Cloud Container Registry or the Google Cloud Artifact Registry on Windows 10 using the Google Cloud SDK. I'm getting a similar permissions error from...

(gcloud.app.deploy) HTTPError 403: <account> does not have storage.objects.get access to the Google Cloud Storage object

When I try to follow the example here: https://cloud.google.com/appengine/docs/flexible/python/quickstart, gcloud app deploy gives me the below error: ERROR: (gcloud.app.deploy) HTTPError 403:...

Heroku SSL connection error unsupported protocol

I have been using Heroku for a while to host my Discord bot. It has been connecting to a MySQL database hosted on ClearDB successfully. However, very recently, whenever I use the bot and it tries...

what is the difference between BigQuery and Storage on GCP?

Hi guys I am using GCP for the first time and while I walking through the a project's cloud function example with the mock data, I got confused about similarities/differences of each one and I...

Using apache-airflow-providers-snowflake on airflow (no module named Snowflake)

I have installed package apache-airflow-providers-snowflake on airflow on docker and i am getting error No module named Snowflake Please refer attachment (check the error mentioned for the...

How to access Artifacts for a Model Endpoint on Unified AI Platform when using Custom Containers for Prediction?

Because of certain VPC restrictions I am forced to use custom containers for predictions for a model trained on Tensorflow. According to the documentation requirements I have created a HTTP server...

gsutil timeout in every call refreshing access_token

This might be a duplicate but none of the previous answers match my conditions. I installed gsutil as part of the google-cloud-sdk following https://cloud.google.com/sdk/docs/install. I could...

Unable to validate google recaptcha enterprise. getting error: java.io.IOException: The Application Default Credentials are not available

Unable to validate google recaptcha enterprise. getting error: java.io.IOException: The Application Default Credentials are not available. java.io.IOException: The Application Default Credentials...

Create new csv file in Google Cloud Storage from cloud function

First time working with Google Cloud Storage. Below I have a cloud function which is triggered whenever a csv file gets uploaded to my-folder inside my bucket. My goal is to create a new csv file...

Loading dynamic schema JSON files into a BigQuery table using autodetect

This question is very similar to this one - but using the python API, and a couple of years later - and as the solution there is not complete, I thought it's best to open a new question. I have a...

ERROR: (gcloud.builds.submit) Error 403: <SERVICE ACCOUNT> does not have storage.objects.get access to the Google Cloud Storage object

I am trying to build a docker image from the Pub/Sub tutorial on Google (link here). I am running this command: gcloud builds submit --tag gcr.io/project_id/pubsub but for some reason I get the...