Tuesday, February 25, 2014

Squash Bugs While You Sleep: Automating JIRA

We've got issues. Oh boy, do we have issues. To keep Polyvore's databases filled with up-to-date information about the best and latest products you want to see and buy, we crawl hundreds of partner websites (with their permission, of course) and load feed files from dozens more. Every time one of those retailers changes their website design, our crawler configuration has to change. Every time one of those sites adds a "New Summer 2014 Collection" section, our crawler needs to know about it. And every time one of those stores has a website problem, our crawler has a problem. With hundreds of websites to crawl, we run into 5-10 errors a day, plus a bunch of warnings. Somebody should really fix all of those things...

Over the past couple of years, Polyvore's approach to issue tracking has evolved from countless hours of human labor into an indispensable tool that multiplies our attention tenfold. I'd like to walk you through our process, in the hopes that you see something that looks familiar and maybe even enlightening.

Wednesday, February 5, 2014

Instant MySQL Restores (with just this script and a ton of disk)

I don’t think that I need to sell you on the concept of backing up your data. But backup strategies and coverage vary widely from organization to organization. This blog post will detail Polyvore's approach to MySQL Backups.

We see 3 basic uses for database backups.

  • Recover from hardware failure
  • Recover from human or application error
  • Create a new replica or slave

Each of these use cases provides a different set of challenges, so we want a comprehensive policy that covers each use case.