Export database(having nested documents) from clou

2019-07-27 17:54发布

I am trying to export cloud firestore data into bigquery to do sql operations.

  1. Exported cloud firestore to cloud storage [using] (https://cloud.google.com/firestore/docs/manage-data/export-import) gcloud beta firestore export gs://htna-3695c-storage --collection-ids='users','feeds'
  2. Followed https://cloud.google.com/bigquery/docs/loading-data-cloud-firestoreto import from bigquery.

We have 2 collections: Users & Feeds in the cloud firestore. I have successfully exported feeds collection but am not able to export users collection. I am getting an error while importing data from storage to bigquery

Error: unexpected property name 'Contacts'. we have contacts field in the collection users. this contacts field is of type 'Map'.

I also tried the command line. Below is the command to export bigquery.

**bq --location=US load --source_format=DATASTORE_BACKUP myproject_Dataset.users gs://myproject-storage/2019-04-19T13:29:28_75338/all_namespaces/kind_users/all_namespaces_kind_users.export_metadata**

here also I got the same error:

'unexpected property name 'Contacts'.

I thought to add projection fields to export only specified fields some thing like below **bq --location=US load --projection_fields=[Coins,Referral,Profile] --source_format=DATASTORE_BACKUP myproject_Dataset.users gs://myproject-storage/2019-04-19T13:29:28_75338/all_namespaces/kind_users/all_namespaces_kind_users.export_metadata**

here also I got the error:

Waiting on >bqjob_r73b7ddbc9398b737_0000016a4909dd27_1 ... (0s) Current status: DONE
BigQuery error in load operation: Error processing job 'myproject:bqjob_r73b7ddbc9398b737_0000016a4909dd27_1': An internal error occurred and the request could not be completed. Error: 4550751

Can anyone please let me know how to fix these issues?

Thank you in advance.Image of firestore Db

0条回答
登录 后发表回答