I wrote a python code to scrap data. But the output window over rights the previous output after a few 1000 lines. The data i am scrapping is over a million lines. I use pycharm by jetbeans. What are some IDE's or methods which support a million lines of output?
可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效，请关闭广告屏蔽插件后再试):
问题:
回答1:
As I mentioned in the comments, the way you've described this seems incorrect. Throwing 1M+ lines to console doesn't do anything for you, other than printing a million lines. If you wanted to use this information later, you wouldn't be able to. Instead, you should be sending this data to a file of some kind so that you can use it later.
That being said, you can increase the output buffer in PyCharm via the following process:
- Click "Help"
- Click "Edit Custom Properties"
- This will either open a file or ask you to create one if the
idea.propertiesfile doesn't exist. Click "Yes" in that case. - Add
idea.cycle.buffer.size=2048to the file
This is not completely disabling the buffer, but instead doubling it from the default 1024kb to 2048kb. Feel free to adjust as appropriate.
Restart PyCharm for these setting to be utilized.
