I want to upload compressed gzip of Json into Google Storage.
I have this code:
import datalab.storage as storage
import gzip
path = prefix + '/orders_newline.json.gz'
storage.Bucket('orders').item(path).write_to(gzip.compress(bytes(create_jsonlines(source)),encoding='utf8'), 'application/json')
The create_jsonlines(source)
is a function that returns Json Newline Delimited.
Running this code gives:
TypeError: string argument without an encoding
The Python docs says the format is: bytes([source[, encoding[, errors]]])
I'm not sure I understand it as there is no example of how to use it.
I tried also
bytes([(create_jsonlines(source))[,encoding='utf8']])
This gives :
SyntaxError: invalid syntax
I'm running Python 3.5
You are not using the bytes
function correctly. Check this:
>>> a = "hi"
>>> bytes(a, encoding='utf8')
b'hi'
You can try:
bytes((create_jsonlines(source)), encoding='utf8')
encoding
is the argument of the bytes
function, and you are using it outside of that function.
You are probably only one step away from the answer.
See bytesarray() and bytes for the function usage (you may need to change python version of the document).
And it says:
The optional source parameter can be used to initialize the array in a
few different ways:
- If it is a string, you must also give the encoding (and optionally, errors) parameters; bytearray() then converts the string to bytes
using str.encode().
- If it is an integer, the array will have that size and will be initialized with null bytes.
- If it is an object conforming to the buffer interface, a read-only buffer of the object will be used to initialize the bytes array.
- If it is an iterable, it must be an iterable of integers in the range 0 <= x < 256, which are used as the initial contents of the
array.
Notice that the square bracket indicates that those parameters can be omitted, it is not an array type of python language.
So you should use bytes(create_jsonlines(source), encoding='utf8')
.
When you read any python function docs as
bytes([source[, encoding[, errors]]])
square brackets represent that those parameters are optional. multiple square brackets inside another mean they are next level of option params. For example
bytes([source....
means we can call bytes as byes() itself as [source]
is optional here
bytes() -> empty bytes object
bytes(22)
Here 22 is being passed as the source
read this for more details about bytes and its params
https://docs.python.org/3/library/stdtypes.html#bytes