Docker not saving file on KeyboardInterrupt

2019-08-29 07:37发布

问题:

I'm storing tweets into a CSV, when I run the following code in Jupyter Notebook it'll save to tweets.csv successfully.

with open(fName, 'a') as f:
while True:
    try:
        if (max_id <= 0):
            # to the beginning of twitter time
            if (not sinceId):
                results = api.search(q=query, count=tweetCount)
            # go to last tweet we downloaded
            else:
                results = api.search(q=query, since_id=sinceId, count=tweetCount)
        # if max_id > 0 
        else:
            # results from beginning of twitter time to max_id
            if (not sinceId):
                results = api.search(q=query, max_id=str(max_id - 1), count=tweetCount)
            # results from since_id to max_id
            else:
                results = api.search(q=searchQuery, count=tweetCount,
                                     max_id=str(max_id - 1),
                                     since_id=sinceId)
        if not results:
            print("No more tweets found")
            break
        for result in results:
            tweets_DF = pd.DataFrame({"text": [x.text for x in results]}, 
               index =[x.id for x in results])
            tweets_DF.name = 'Tweets'
            tweets_DF.index.name = "ID"
            tweets_DF.to_csv(f, header=False)
        tweetCount += len(results)
        print("Downloaded {0} tweets".format(tweetCount))
        max_id = results[-1].id
    except (KeyboardInterrupt, SystemExit):
        print ("Downloaded {0} tweets, Saved to {1}".format(tweetCount, os.path.abspath(fName)))
        quit()
    except tweepy.TweepError as e:
        print("Error : " + str(e))
        break

When run in a Docker container and issued a Keyboard Interrupt it returns

Downloaded 520 tweets, Saved to /app/tweets.csv

but nothing is saved.

How do I get the script to write out in the container, also what is happening under the hood here?

Edit:

Commands Run:

docker build -t dock .
docker run dock

Here's the Dockerfile:

# Use an official Python runtime as a parent image
FROM python:3.6-slim

# Set the working directory to /app
WORKDIR /app

# Copy the current directory contents into the container at /app
ADD . /app

# Install any needed packages specified in requirements.txt
RUN pip install --trusted-host pypi.python.org -r requirements.txt

# Make port 80 available to the world outside this container
EXPOSE 80

# Define environment variable
ENV NAME World

# Run app.py when the container launches
CMD ["python", "app.py"]
    enter code here

回答1:

From your docker command , it seems that you are not mounting your local directory to your docker container. Docker containers have a different filesystem from the host and files saved in docker containers are get lost once the container is removed. To fix this, run your command like this:

docker run  -v /User/Seth/some/local/directory:/app/ dock

This commands binds your local directory that you defined before the : to the directory app inside the docker container. You should check that the local directory exists, and it should be the absolute path. As your docker container runs in /app,it will download the tweets.csv there and you should be able to see it in your /some/local/directory after it is completed. You might also see your whole codebase as you are downloading the file to the same directory with your code.

Here is the link for Docker volumes


*Just for completeness, this has nothing to do with keyboard interrupt.



标签: python docker