Encoding error using df.to_csv()

Question

I am trying to save information from Twits (screen_name, created_at and text) into a pandas DataFrame and then save DataFrame as a csv file.

I am getting an encoding error

import tweepy
from tweepy import OAuthHandler

consumer_key = 'bla'
consumer_secret = 'bla'
access_token = 'bla'
access_secret = 'bla'
auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_secret)
api = tweepy.API(auth)

import pandas as pd
import numpy as np
import datetime
import sys

encoding = sys.stdout.encoding or 'utf-8'
columns = ['Screen_Name', 'Time_Stamp', 'Tweet']
todays_date = datetime.datetime.now().date()
tweetDF = pd.DataFrame(columns=columns)

for tweet in tweepy.Cursor(api.search, q="manhattan", lang="en").items(10):
    lenDF = len(tweetDF)
    tweetDF.loc[lenDF] = [tweet.user.screen_name, tweet.created_at, tweet.text]

tweetDF.to_csv("C:/tweetDF")

Here the error:

UnicodeEncodeError                        Traceback (most recent call last)
<ipython-input-11-c0aa5e7ee620> in <module>()

---> 34 tweetDF.to_csv("C:/tweetDF")

C:\Anaconda\lib\site-packages\pandas\core\frame.pyc in to_csv(self, path_or_buf, sep, na_rep, float_format, columns, header, index, index_label, mode, encoding, quoting, quotechar, line_terminator, chunksize, tupleize_cols, date_format, doublequote, escapechar, decimal, **kwds)
   1187                                      escapechar=escapechar,
   1188                                      decimal=decimal)
-> 1189         formatter.save()
   1190 
   1191         if path_or_buf is None:

C:\Anaconda\lib\site-packages\pandas\core\format.pyc in save(self)
   1465 
   1466             else:
-> 1467                 self._save()
   1468 
   1469         finally:

C:\Anaconda\lib\site-packages\pandas\core\format.pyc in _save(self)
   1565                 break
   1566 
-> 1567             self._save_chunk(start_i, end_i)
   1568 
   1569     def _save_chunk(self, start_i, end_i):

C:\Anaconda\lib\site-packages\pandas\core\format.pyc in _save_chunk(self, start_i, end_i)
   1592                                         quoting=self.quoting)
   1593 
-> 1594         lib.write_csv_rows(self.data, ix, self.nlevels, self.cols, self.writer)
   1595 
   1596 # from collections import namedtuple

pandas\lib.pyx in pandas.lib.write_csv_rows (pandas\lib.c:17992)()

UnicodeEncodeError: 'ascii' codec can't encode character u'\u2666' in position 7: ordinal not in range(128)

I tried various encoding enhancements but was not successfull

Answer 1

I found the way to fix it and would like to share:

tweetDF.to_csv("C:/tweetDF", sep='\t', encoding = 'utf-8')