I have a site I'm trying to test and although I can get a list of options in a dropdown I am not sure how select it? There is no submit button so if I select it then it will load an ajax table below.
I'm just not sure if lxml/requests can do this or how it could be done? I would appreciate it if anyone could confirm or knows the function that could do it ?
edit: My site is internal and not accessible but here is a sample site: https://www.tsx.com/listings/listing-with-us/listed-company-directory Notice if you select the box beside search it will pull out ajax results(A,B,C,etc..)
With lxml
+requests
setup, you operate on a different level, a lower level. There are no dropdowns for requests
, it is not a browser.
The general approach to your problem is to:
- open Browser Developer Tools in your browser of choice, Network tab
- select an option in the dropdown
- observe the request(s) sent to the server in the Network tab
- simulate this/these requests in your Python code
There is one more "trick" that you can use to ease the developer tools to requests transition:
- right click on the request in the list of requests in Network tab
- click "Copy as cURL"
- go to http://curl.trillworks.com/ webpage and convert the cURL command to the Python+Requests code