More information about the Underscore mailing list

[_] YAPCW (Yet another price comparison website)

Tom Holder taholder at
Wed Sep 21 12:16:59 BST 2011

Agreed, worth checking their website T&C for reference to automated scraping
but judging by the requirements this isn't something that would need to be
done ever 20 seconds :) so in theory they should have a problem with it,
especially if it is driving more sales for them!


On 20 September 2011 20:07, Tim Wintle <timwintle at> wrote:

> On Tue, 2011-09-20 at 19:19 +0100, Tom Holder wrote:
> > Lots of negativity towards the maintainability of scrapers but I tend to
> > disagree due to where we are at with things.
> >
> > 1. Cloud infrastructure means providing you make your scraper portable
> and
> > make it operate from multiple sites blocking based on IP can be
> > circumvented.
> If they block you based on IP then you're doing something wrong in their
> opinion - which means they can set the lawyers on you (assuming
> appropriate terms and conditions etc). circumventing that means you're
> actively trying to sidestep a measure they've put in place, so that in
> itself would probably count as mis-using their service IMO.
> Tim
> --
> underscore_ list info/archive ->