Merge search result pages

Top  Previous  Next

This example shows how to merge pages from a search result.

 

In that example, the page URLs contain a &start= parameter, whereas the first page doesn't have that parameter, following pages have steps of 10 (&start=10, &start=20, etc.)

 

Sub Wsw_MergePages($sMem, $nPageNumber, $sUrl, ByRef $sNewUrl, ByRef $sNewPostData, ByRef $bChangeBookmarkUrl, ByRef $sStatusMessage, ByRef $iStatusCode)

   

   Dim $nPageParamNumber, $sPageParam, $sNewPageParam

   

   ' Limit the number of merged pages

   If $nPageNumber > 3 Then

      Return

   End If

   

   ' Extract page number

   $sPageParam = GetFirstRegexMatch($sUrl, "\&start=\d+")

   If $sPageParam = "" Then

      ' first page, continue with page 2 (solved with start=10)

      $sNewPageParam = "&start=10"

      $sUrl = $sUrl + $sNewPageParam

   Else

      ' increase page number

      $nPageParamNumber = CInt(ExtractDigits($sPageParam)) + 10

      $sNewPageParam = "&start=" + CStr($nPageParamNumber)

      $sUrl = Replace($sUrl, $sPageParam, $sNewPageParam)

   End If

   

   ' check (only) via Page-Param, if the next page is already available in the page source

   If Pos($sNewPageParam, $sMem) > 0 Then

      ' the next page exists

      $sNewUrl = $sUrl

      $sStatusMessage = "Merge pages via FOLLOW-Plugin"

   End If

   

End Sub