Pycharm output limit exceeded

Question

I wrote a python code to scrap data. But the output window over rights the previous output after a few 1000 lines. The data i am scrapping is over a million lines. I use pycharm by jetbeans. What are some IDE's or methods which support a million lines of output?

Answer 1

As I mentioned in the comments, the way you've described this seems incorrect. Throwing 1M+ lines to console doesn't do anything for you, other than printing a million lines. If you wanted to use this information later, you wouldn't be able to. Instead, you should be sending this data to a file of some kind so that you can use it later.

---

That being said, you can increase the output buffer in PyCharm via the following process:

• Click "Help"
• Click "Edit Custom Properties"
• This will either open a file or ask you to create one if the idea.properties file doesn't exist. Click "Yes" in that case.
• Add idea.cycle.buffer.size=2048 to the file

This is not completely disabling the buffer, but instead doubling it from the default 1024kb to 2048kb. Feel free to adjust as appropriate.

Restart PyCharm for these setting to be utilized.