Dosage

Penny Arcade Scraper Fix

Bug #482055 reported by Ged Walsh on 2009-10-26

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	Dosage	Fix Committed	Medium	Tristan Seligmann	Dosage 1.7.0 "Do you have anything else horrible?"

Bug Description

http://www.penny-arcade.com/comic/ (They changed the image url and previous links)

class PennyArcade(BasicScraper):
    latestUrl = 'http://www.penny-arcade.com/comic/'
    imageUrl = 'http://www.penny-arcade.com/comic/%s/'
    imageSearch = compile(r'(?<!<!--)<img src="(http://art.penny-arcade.com/photos/.+?)"')
    prevSearch = compile(r'<a href="(/comic/.+?)/">Back</a>')
    help = 'Index format: yyyy/mm/dd'

Tags:

Related branches

lp://qastaging/~dosage-dev/dosage/bunch-of-comics-2

Merged into lp://qastaging/~dosage-dev/dosage/old at revision 605

Jonathan Jacobs: Approve on 2010-01-07

Revision history for this message

Ged Walsh (bleedingheart) wrote on 2009-12-04:

Better fix.

class PennyArcade(BasicScraper):
imageUrl = 'http://alienlovespredator.com/%s'
imageSearch = compile(r'(?<!<!--)<img src="(http://art.penny-arcade.com/photos/.+?)"')
prevSearch = compile(r'<a href="(/comic/.+?)/">Back</a>')
help = 'Index format: yyyy/mm/dd'
starter = indirectStarter('http://www.penny-arcade.com/comic/', compile(r'<a href="(/comic/.+?)/">Back</a>'))
def namer(cls, imageUrl, pageUrl):
     num = pageUrl.split('/')[-3]
     ccc = pageUrl.split('/')[-2]
        ddd = pageUrl.split('/')[-1]
     return '%s-%s-%s' % (num, ccc, ddd)

This deliberately skips the latest image so it can name the strips in order.

summary:

- Penny Arcade Module Fix
+ Penny Arcade Scraper Fix

Tristan Seligmann (mithrandi) on 2010-01-03

Changed in dosage:
assignee:	Jonathan Jacobs (jjacobs) → Tristan Seligmann (mithrandi)
importance:	Unknown → Medium

Tristan Seligmann (mithrandi) on 2010-01-03

tags:

added: comic

Tristan Seligmann (mithrandi) on 2010-01-06

Changed in dosage:
milestone:	none → 1.7.0

Tristan Seligmann (mithrandi) on 2010-01-07

Changed in dosage:
status:	New → Fix Committed

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.