CLI “bq load” - how to use non-printable character

问题:

I'm having trouble loading data into BigQuery as a single column row. I wish BigQuery offered the ability to have "no delimiter" as an option, but in the meantime I need to choose the most obscure ASCII delimiter I can find so my single column row is not split into columns.

When doing this the CLI won't allow me to input strange characters, so I need to use the API through Python or other channels.

How can I use the CLI instead with a non printable character?

Python example from BigQuery lazy data loading: DDL, DML, partitions, and half a trillion Wikipedia pageviews:

#!/bin/python
from google.cloud import bigquery
bq_client = bigquery.Client(project='fh-bigquery')
table_ref = bq_client.dataset('views').table('wikipedia_views_gcs')
table = bigquery.Table(table_ref, schema=SCHEMA)
extconfig = bigquery.ExternalConfig('CSV')
extconfig.schema = [bigquery.SchemaField('line', 'STRING')]
extconfig.options.field_delimiter = u'\u00ff'
extconfig.options.quote_character = ''

回答1:

To use a non-printable character with BQ load you can use echo in bash:

bq load \
 --source_format=CSV \
 --field_delimiter=$(echo -en "\x01") \
 --noreplace --max_bad_records=100 \
 <bq_dataset>.<bq_table> gs://<bucket_name>/<file_name>.csv