How do I coalesce Stackdriver logs/sinks into one

Question

Setting up Stackdriver log sinks to BigQuery is straightforward.

However, I have lots of projects, and instead of each export sink going to its corresponding project, I'd like to coalesce the logs from all my projects to one dedicated project.

The configuration in the Stackdriver sink config doesn't appear to let me select a different project to send the logs to.

---

How do I select a different project/dataset?

Answer 1

You need to select the 'Custom destination' option. This will allow you to plug in a different project/dataset. The default datasets you see are always tied to whichever project you've currently got selected, so you can't select a different project/dataset.

Instead:

1. Select 'Custom Destination'
2. Add the destination using this format: bigquery.googleapis.com/projects/[PROJECT_ID]/datasets/[DATASET_ID]
3. Give the sink writer editor permissions on the said project/dataset in step 2.

See more here.

Answer 2

If all of your projects are under a Cloud Folder or under the same organization, you can also create an Aggregated Export as documented in https://cloud.google.com/logging/docs/export/aggregated_exports.

The aggregated sink lives in the folder or org, and exports all logs of children projects to the same destination.