streamify

GCP

Initial Setup

Checkout this video

1. Create an account with your Google email ID
2. Setup your first project if you haven’t already
   • eg. “Streamify”, and note down the “Project ID” (we’ll use this later when deploying infra with TF)
3. Setup service account & authentication for this project
   • Grant Viewer role to begin with.
   • Download service-account-keys (.json) for auth. (Please do not share this key file publicly. Keep it secure!)
   • Rename the .json key file to google_credentials.json
4. Download SDK for local setup
5. Set environment variable to point to your downloaded GCP keys:

   export GOOGLE_APPLICATION_CREDENTIALS="<path/to/your/google_credentials.json>"
      
   # Refresh token/session, and verify authentication
   gcloud auth application-default login
   

Setup for Access

1. IAM Roles for Service account:
   • Go to the IAM section of IAM & Admin https://console.cloud.google.com/iam-admin/iam
   • Click the Edit principal icon for your service account.
   • Add these roles in addition to Viewer : Storage Admin + Storage Object Admin + BigQuery Admin
2. Enable these APIs for your project:
   • https://console.cloud.google.com/apis/library/iam.googleapis.com
   • https://console.cloud.google.com/apis/library/iamcredentials.googleapis.com
   • Note: You might have to enable a few APIs here and there like DataProc etc.
3. Please ensure GOOGLE_APPLICATION_CREDENTIALS environment variable is set.

   export GOOGLE_APPLICATION_CREDENTIALS="<path/to/your/service-account-authkeys>.json"
   

Installation Reference - DataTalks Club