Integrating Cloud Run functions with databases
A Cloud Run function usually needs to read or write data somewhere. It can integrate with Firestore, Cloud SQL, Spanner, Bigtable, BigQuery, and Memorystore (Google Cloud's in-memory cache). This page covers the two that need special wiring - Memorystore (reached over a private network) and Firestore (driven by events) - plus the two mechanisms a function uses to configure and authenticate those connections: environment variables and secrets.
- Firestore and Memorystore are covered below. BigQuery integration is covered separately - see BigQuery Remote Functions.
- For Cloud SQL, Spanner, and Bigtable, refer to the product documentation.
Memorystore
Memorystore is a highly available, scalable, and secure in-memory cache service. It's a fully managed service that automates provisioning, replication, failover, and patching, is integrated with IAM for secure access, and with Cloud Monitoring for monitoring and alerting.
- Redis - an open-source in-memory data structure store used as a database, cache, message broker, and streaming engine.
- Memcached - an open-source distributed memory object caching system.
Connecting to Memorystore (Redis)
A Memorystore instance lives on a VPC network, so a serverless function reaches it over Serverless VPC Access - the same connector mechanism used for any private resource.
- Determine the Redis instance's authorized VPC network.
- Create a Serverless VPC Access connector in the same region as the function, specifying the network, region, and IP address range. Ensure it reaches the Ready state before use.
- Attach the connector to the Redis instance's authorized VPC network.
- Deploy the function with the connector name and environment variables for the Redis host IP address and port. The function code reads those variables to instantiate a client that connects to Redis.
- Invoke the function by sending an HTTP GET request to its URL endpoint.
Environment variables
Environment variables are key-value pairs set for a function at deployment time and read by the function code at runtime (or used as configuration for the buildpack system). They're stored in the Cloud Run functions backend, bound to a single function, and exist within that function's lifecycle.
- Pass them inline with the gcloud CLI (
--set-env-vars FOO=bar,BAZ=boo) or in the Google Cloud console. - Or store them in a YAML file in source control and pass its name at deploy:
--env-vars-file env.yaml. - Add, update, or remove them at any time via the console or the gcloud CLI.
Read them in your function code with the runtime's standard mechanism:
| Runtime | Access environment variables with |
|---|---|
| Python | the os module - os.environ.get('FOO') |
| Node.js | the process.env property - process.env.FOO |
Firestore
Firestore is a fully managed serverless NoSQL document database with high availability, scalability (no maintenance windows or downtime), and multi-region replication. You store a set of key-value pairs as a document, and all documents are stored in collections.
You extend Firestore with Cloud Run functions by handling events triggered by changes in the database - adding server-side functionality without running your own servers. The trigger mechanics (event types, document path, snapshot delivery) are covered under Firestore trigger.
Reading and writing Firestore data
When a function is triggered, a snapshot of the data related to the event is available - the document state before and after the change. Each invocation is associated with a specific document, reachable as a DocumentReference (Firestore Node.js SDK) whose methods modify the document that triggered the function.
| In a Node.js Firestore function | Gives you |
|---|---|
change.after.data() | the document data after the update |
change.before.data() | the document data before the update |
change.after.ref | the DocumentReference - read or write the triggering document |
To read and write documents other than the one that triggered the function, use the Firebase Admin SDK.
Secrets
When function code accesses a database or API, it needs credentials - a database username/password or an API key. This sensitive information is stored in and accessed from Secret Manager, not baked into the code.
A secret is an object containing a collection of metadata (replication locations, labels, permissions) and its secret versions. A secret version stores the actual data - an API key or password - as a text string or binary blob. You must enable the Secret Manager API to create and manage secrets.
Accessing a secret from a function
To access a secret, the function's runtime service account must be granted the Secret Manager Secret Accessor role (roles/secretmanager.secretAccessor) on that secret. You then choose how the secret is exposed to the function:
To use a secret that lives in a different project than the function: grant the function's runtime service account access to the secret as above, then reference it by its full resource path including the project ID - projects/PROJECT_ID/secrets/SECRET_NAME.
Using a secret end to end
- Create and store the API key as a secret in Secret Manager.
- Write function code that reads the secret from a file (mounted volume) or environment variable.
- Deploy the function, providing the secret name and the access method (mounted file path or environment variable).
- Grant the function's runtime service account the Secret Accessor role on the secret.
- Call the external API from the function with the secret value (the API key).