I'm trying to scrape prices from this site: https://www.pensketruckrental.com/quote/start.html
I can easily enter the form data, and I can activate the "Get A Quote" button and click it.
What I can't seem to do is get the form data to submit using a web scraper (I'm just doing it in VBA). When I input text using the scraper, the button remains grayed out, and even making a .click
call on the button just displays errors on the form telling you not to leave the fields blank. Apparently it only recognizes data when you use an input device?
The code for one of the required fields, pickupLocation
, is the following when I enter it manually (and thus the button works and the form can be submitted):
<input
type="text"
id="pickUpLocation"
name="pickUpLocation"
class="penskeValidateField penskeGoogleTypeAhead penskeInlineError ng-isolate-scope ng-touched ng-focused ng-dirty ng-valid-penske-err_loc_empty_sa ng-valid ng-valid-parse ng-valid-required"
aria-invalid="false"
aria-required="false"
country="rentalEntryCtrl.formItems.country"
penske-validate-field="pickuplocation"
required=""
autocompelete="off"
data-penske-placeholder="rentalEntryCtrl.activePlaceHolders.pickUpLocation"
ng-model="rentalEntryCtrl.formItems.pickupLocationSearchCriteria.address"
autocomplete="off">
And when I enter the data automatically using my scraper the tag & attributes read as follows:
<input
type="text"
id="pickUpLocation"
name="pickUpLocation"
class="penskeValidateField penskeGoogleTypeAhead penskeInlineError ng-pristine ng-isolate-scope ng-invalid ng-invalid-required placeholder ng-touched"
aria-invalid="true"
aria-required="true"
country="rentalEntryCtrl.formItems.country"
penske-validate-field="pickuplocation"
required=""
autocomplete="off"
data-penske-placeholder="rentalEntryCtrl.activePlaceHolders.pickUpLocation"
ng-model="rentalEntryCtrl.formItems.pickupLocationSearchCriteria.address"
autocompelete="off">
So of course I tried to copy the fields in the first code block into the second code block using setAttribute()
, but even though I could change the attributes, I still couldn't get the form to submit properly.
I've looked at others that have dealt with something somewhat similar with autocorrect; their solutions have involved looking at the header and responses and just using the straight XHR to loop through the autocomplete queries, but the pricing information I'm scraping comes after several pages of form submissions, so that's not an option here.
I'm stuck I think; any ideas on how to populate the form and click the button/submit via my scraper?