How to cache results of WebImageSearch?

Question

Every call to WebImageSearch costs money (wolfram service credits), thus, I would like to page through results without having to redownload them.

Given the following search how would one retrieve items 6-10 (without repurchasing the first 5)?

WebImageSearch["desert plains", "Thumbnails", MaxItems -> 5]

Update

Response to @kuba's question in the comments:

Update 2

@b3m2a1's solution doesn't run for me:

vgBlock[fn_, HoldPattern[SetDelayed][lhs_, rhs_], expr_] := Module[
    {$recurse},
 Internal`InheritedBlock[{fn},
    Unprotect[fn];
 (lhs /; ! TrueQ[$recurse] := Block[{$recurse = True}, rhs]);
    DownValues[fn] = SortBy[DownValues[fn], FreeQ[$recurse]];
    expr
   ]
];
vgBlock~SetAttributes~HoldAllComplete;
Options[googleImageSearch2]=Join[Options[WebImageSearch], {"StartingFrom" -> 1}];
googleImageSearch2[params:Except[_?OptionQ].., ops:OptionsPattern[]] :=
 vgBlock[GCSFormatRequestParameters, GCSFormatRequestParameters[e___] := GCSFormatRequestParameters[e] /. ("start" -> "1") -> ("start" -> ToString[OptionValue["StartingFrom"]]),
  WebImageSearch[params,  Sequence @@ FilterRules[{Method -> "Google",  ops}, Options @ WebImageSearch]
  ]
]

When I run googleImageSearch2["sweet dee", "MaxItems" -> 4], the progress bar starts working but then fails and the symbols turn red:

Ah ok, WebImageSearch stopped working because I ran out of service credits:

Probably should have warned me!

Answer 1

So it looks like the only real way to do this is by hacking the mainframe package that underlies this. Everything seems to be clumsy and hard coded, so I had to dig a little. The following works for Google, but it is easily adaptable to Bing.

The main function of interest is GCSFormatRequestParameters which formats request parameters to Google Custom Search. When you use this it hard codes a "start"->"1". We want to specify our own starting point so we will simply hack this function to do our bidding before it passes the parameters to the API call.

First a function to implement a single Villegas-Gayley redefinition in a general manner:

vgBlock[fn_, 
   HoldPattern[SetDelayed][lhs_, rhs_], 
   expr_
   ] :=
  Module[{$recurse},
   Internal`InheritedBlock[{fn},
    Unprotect[fn];
    (lhs /; ! TrueQ[$recurse] :=
      Block[{$recurse = True}, rhs]);
    DownValues[fn] = SortBy[DownValues[fn], FreeQ[$recurse]];
    expr
    ]
   ];
vgBlock~SetAttributes~HoldAllComplete

Then we'll use this to override the old results:

Options[googleImageSearch2] =
  Join[
   Options[WebImageSearch],
   {
    "StartingFrom" -> 1
    }
   ];
googleImageSearch2[params : Except[_?OptionQ] .., 
  ops : OptionsPattern[]] :=
 vgBlock[GoogleCustomSearchFunctions`GCSFormatRequestParameters,
  GoogleCustomSearchFunctions`GCSFormatRequestParameters[e___] :=

   GoogleCustomSearchFunctions`GCSFormatRequestParameters[e] /. ("start" -> "1") -> ("start" -> 
       ToString[OptionValue["StartingFrom"]]),
  WebImageSearch[params,  
   Sequence @@ 
    FilterRules[{Method -> "Google",  ops}, Options@WebImageSearch]
   ]
  ]

And finally we can test that it works:

googleImageSearch2["doggo", "MaxItems" -> 4]

googleImageSearch2["doggo", "MaxItems" -> 4, "StartingFrom" -> 3]

One useful note when digging make sure wrap any calls to WebImageSearch in:

Block[
 {
  WebSearch`Private`GCSPaginationCalls,
  WebSearch`Private`BSPaginationCalls
  },
 doop doop
 ]

So that you can test the parameters without actually searching / downloading the results