How do I modify web scraping code to loop through

2019-09-14 04:54发布

问题:

Here is my code. It pulls the "Item Model Number" off of Amazon detail pages. This has been written to find "Item Model Number" in the detail page bullet points and extract the number next to it.

The issue is that it sometimes will be unable to pull the "Item Model Number" from a page that clearly has an Item Model Number.

Here is the code

Sub Get_ITEM_CODE(ie As Object)
    Dim WB As Workbook
    Dim WS As Worksheet
    Dim y As String
    Dim AmUrl As String
    AmUrl = ActiveCell.Value
    ''Set WB = Workbooks.Add

    Set WS = Sheets("Extract Item COde")
    ie.Navigate AmUrl
    Application.Wait (Now + TimeValue("00:00:02"))
    Do While ie.readyState <> 4: Loop
    On Error Resume Next

    y = ie.document.getElementById("productDetails_detailBullets_sections1").innerText

    WS.Range("A1").Value = y

    SplitTextItemCode
    AddtoListItemCode

End Sub

Here is a string of HTML where the Code does its job properly:

<div id="detailBullets" class="feature" data-feature-name="detailBullets">

<div id="detailBulletsWrapper_feature_div" data-feature-name="detailBullets" data-template-name="detailBullets" class="a-section a-spacing-none feature">
    <div id="detailBullets_feature_div">

URL to webpage

Here is a string of HTML where the Code did not do its job:

<div id="detailBullets" class="feature" data-feature-name="detailBullets">

<div id="detailBulletsWrapper_feature_div" data-feature-name="detailBullets" data-template-name="detailBullets" class="a-section a-spacing-none feature">
    <div id="detailBullets_feature_div">

URL to Web Page

回答1:

Try this, it will loop through every "li" item under that element id. Then it will replace/remove the "Item model number:" text so you're left with a clean model number for whatever product your looking at.

Dim Cnt As Variant
Dim oCell As Object
Cnt = 0  
With ie.Document.body.all.Item("detailBulletsWrapper_feature_div").all
    For Each oCell In .tags("li")
        If InStr(oCell.innerText, "Item model number:") > 0 Then
          ModelNum = oCell.innerText
          ModelNum = VBA.Replace(ModelNum, "Item model number:  ", "")
          Debug.Print ModelNum
        Exit For
        End If
  Cnt = Cnt + 1
Next oCell
End With
Set oCell = Nothing
   End Sub