Export all http requests on a specific page to txt

问题:

I use SIEGE to test my web server performance. For a more realistic test the best way to go would be to have SIEGE hit the web page (website.com/our-company) and all static assets (.css, .js, .png, .jpg). Everything that you see on the firefox / chrome debbuing tools, except of course from resources loaded from external servers (cdn.facebook, apis.google.com).

I am running several tests so it is a pain to manually collect all asset urls. Is there a tool that I can use to load a web page and export the url for everything that was loaded?

This is firefox debugging. If I could export this to txt or csv, it would be perfect.

I tried CURL on debian CLI but I am no experct. Any tool will help, it does't have to be a plugin of Firefox / Chrome.

Best regards.

回答1:

In Chrome you can export these data to a HAR file (it's JSON based) in one click. Go to "Network", right click and choose "Save as HAR with content".

回答2:

Here's a free command line application to convert HAR files to CSV. Hope it helps.

http://www.yamamoto.com.ar/blog/?p=201

EDIT: added the project to GitHub:

https://github.com/spcgh0st/HarTools

回答3:

On Windows you could use HttpWatch to do this with the free Basic Edition in IE or Firefox:

http://www.httpwatch.com/download/

The CSV export function will export the URLs and other fields to a CSV file.

** Disclaimer: This was posted by Simtec Limited the makers of HttpWatch **

回答4:

Had the same requirement of exporting HAR files from Chrome DevTools or Firebug to do load testing with siege. Additionally, I wanted to replay POST requests too.

Choose one of these solutions:

• hardy @ https://github.com/nbibler/hardy - ruby script
• har2siege @ https://gist.github.com/photopresentr/7974747 - node.js (my script)

回答5:

As you guys know, the HAR file format is a JSON file. So... I looked for a JSON to CSV converter and found this:

https://json-csv.com/

This worked for my HAR file that I got from GTmetrix.com. Enjoy!

回答6:

You can Export all Http requests from Chrome Developer console by going to the Network tab

• select one of the requests in Network Tab
• press Right Mouse Button
• from PopUp menu select Copy -> Copy all as Har (Curl/Har/etc)
• paste into file

回答7:

Nevermind.

Just fond out the very nive LiveHtttpHeaders extension for firefox.

Best regards.