Friday, February 27, 2015

Data and the User Experience

Post 2: Data and the User Experience
By Matt Wheeler

Last month we shared some of the technology behind Polyvore’s Style Profile and how we’re using machine learning to understand our users' individual style to recommend more personalized products and outfits.

We discovered that our unique set data (our users create over 3 million sets every month) helps improve the recommendations to create a more engaging shopping and discovery experience for our users. Let’s dive into more detail:

Without divulging too much of the secret sauce, we recently developed three independent algorithms -- or “streams,” as we call them-- that we use to generate product recommendations.
  1. Stream 1: Generates recommendations based on a user’s brand-affinity (a passion for Prada, for example).
  2. Stream 2: Generate recommendations based on collaborative filtering: items that similar users have liked.
  3. Stream 3: Leverages the talent of our awesome community of creators by recommending items frequently paired in sets.

We decided to get a deeper understanding of how users react to the three types of recommendations by launching an experiment that tests each stream. The results are pretty interesting and give us insight into ways that we can improve our user experience.

Which of the three streams do users prefer?
One of the features we make available is the ability to “like” a product. We can use these likes as a tool for measuring the quality of our recommendations. That is, if we show you 20 items and you like 10 of them then we are doing much better than if we show you 20 items and you like 5 of them.

We looked at the distribution of like rates data to understand which streams users prefered:

Figure 1 - Overall like rate distribution (median = black horizontal bar)

The horizontal black lines in the middle of the boxes are the median like rates for each recommendation type. The boxes themselves represent the range of like rates in which most users fall. It is clear that the similar-users-based recommendations are the most popular, followed by the Polyvore set-based recommendations, with the brand-based recommendations bringing up the rear. So, we should focus our energies on increasing the number of impressions from user-based recommendations, right? Well, maybe. The overall variances in the individual like rates is relatively high (the boxes cover a lot of area on the graph).

Does this mean that we have a lot of heavy “likers” and a bunch of “non-likers”? Does it mean that users actually have individual preferences for recommendation type? Or does it mean something else entirely?
To answer these question, we used each user’s likes to create their “perfect mix” of streams and compared it against the average user’s perfect mix. For instance, if we showed you 100 recommendations from each stream and in each stream you liked 10 items, then your perfect mix would be an even 33%/33%/33% mix of recommendations from each stream. If the average Polyvore user’s perfect mix was 40% similar-user-based recommendations, 35% set-based recommendations, and 25% brand-based recommendations then we would say you have a -7% relative affinity for similar-user-based recommendations, a -2% affinity for set-based recommendations, and a +8% affinity for brand-based recommendations. Plotting these affinities per user in ascending order we get the following graphs:

Figure 2 - Distribution of individual preference for each of the three streams. The x-axis is individual users, sorted by increasing stream affinity. A positive .1 means that a user’s ideal mix of streams would add an additional 10% to the average mix (e.g. go from 25% of total individual impressions to 35% of total individual impressions). Note that it is not possible for a stream to be increased in a user’s ideal mix without a decrease in at least one other stream.

In each of these graphs, a large negative value indicates a user who likes the stream much less than the average Polyvore user, while a large positive number indicates someone who likes the stream much more. We can see that, for each stream, there is a non-trivial minority of users who have a strong positive affinity. There is a similarly sized minority who have a strong negative affinity. This suggests that we could improve the individual user experience by showing users more impressions from the streams they like and fewer from those they dislike.

Testing our hypothesis with an experiment:
Another interesting finding concerns the relationship between user’s reactions to the streams. It turns out that there is a pronounced negative correlation between brand-based affinity and similar-user-based affinity:

Figure 3 - 95% prediction interval of similar user affinity regressed on brand affinity

This chart uses the same values as those in Figure 2 and shows that users who have a strong affinity for recommendations based on similar-users generally have a strong negative affinity for brand-based recommendations, and vice versa. Interestingly, this relationship is much weaker when comparing similar-user-based affinity with set-based affinity.

Figure 4 - 95% prediction interval of similar user affinity regressed on set affinity

This tells us that users who like recommendations from other users also appreciate -- or at least are not overly annoyed by -- recommendations based on sets. We see this same weak relationship when we compare brand-affinity folks to the set-based stream.

Figure 5 - 95% prediction interval of brand affinity regressed on set affinity

This is great news for us! Making recommendations based on sets is something that only Polyvore can do, and the data suggests that investing time to improve these recommendations will only complement the user experience. Our creators’ sets are a rich store of fashion data and this set-based product recommendation algorithm only scratches the surface of what’s possible. All is powered by the unique data from our creators’ sets. Stay tuned.  

Wednesday, January 7, 2015

Core of Personalization at Polyvore: Style Profile

Over the past year, our engineering team has undertaken the task of creating a more personalized experience for our users. We already have an amazing community of designers, artists, and fashion enthusiasts who come to Polyvore to get inspired around shopping. However, we felt that with a little bit of machine learning we could help users discover and shop for even more products that they may not have found on their own.
In this blog post we’ll walk through some of the ways we are using machine learning to understand our users individual style, which we call a Style Profile, to recommend more personalized products and outfits.

What is a Style Profile?

When we first started building each user’s Style Profile, we quickly realized how tricky quantifying fashion can be. It’s intangible, means different things for different people and even when most people might own the same black shirt, they might wear it in completely different ways. Luckily, Polyvore is uniquely positioned to understand personal style through our users rich interactions on Polyvore, including:

  • Global factors: occasions, trends, seasonality and other contextual information
  • Catalog data: rich and high-quality metadata of products from our retail partners
  • Product data: product likes and dislikes, collections of products, products viewed and search queries
  • Shopper behavioral data: impressions, likes, outbound clicks while they are interacting with products, sets and other curated content
  • Community data: Our global community has generated billions of data points that helps us understand the relationship between retail products. Every time a user creates a set, they are implying that those products go together and share the same style. 

From a technology standpoint, a user’s Style Profile can be represented with a vector in a high-dimensional space and the component for each dimension, indicating the strength of their preference in a particular aspect or a combination of multiple aspects in fashion. The following is a simplified representation of two users’ style profile on combinations of color, category, material and brand:

Figure 1: Style profiles

  • Style Space Definition: a high dimensional space where any point represents the style of a user or product that is subject to constraints that points with similar style should be closer to each other than those with different tastes.
  • Style Vector Definition: the coordinates in the Style Space denote the taste vector for that particular user or product.

Thursday, December 11, 2014

Meet the Team Behind Polyvore: Andrea Sandke

Get to know the Polyvore team!
As we approach the end of 2014, Andrea talks about how she analyzed Polyvore search queries to get a pulse on the community over the past year. To learn more about Andrea, check out her About Me set here. And to learn more about Polyvore’s end of year search data, check out our blog post here.

Meet the Polyvore Team: Andrea Sandke

What do you do every day?
I have a fascinating job! Every day I spend time reviewing Polyvore’s search query logs to find notable patterns in Shop search, Set search, and Editor search, and then I share insights about these trends with the rest of the Polyvore team.

Friday, November 21, 2014

Making Things

Making Things

Kurt Schaefer is our Junkyard Mega-Wars star and a Staff Engineer who works on making our iOS apps amazing. Since working here, Kurt has learned both to code in Objective-C and how to mount an effective foosball defense (so key!). Kurt is an incredibly creative craftsman of the Maker-verse who sews amazing Halloween costumes from scratch, and loves building whimsical projects in his workshop while listening to Ella Fitzgerald. Read more about his projects at his blog, Retro Tech Journal, and check out his Polyvore set!
As an amature wood-worker, machinest, welder, and tinkerer, I'm constantly learning from my hobbies and applying what I learn in my role as a Software Engineer at Polyvore. I am passionate about making things. Here's some advice on making things both at home and professionally.

Why Make Things?

It turns out that it's fun to make things
From building a custom door knocker to launching home brew rockets with your kids, building things is fun. It gives you a sense of accomplishment. Physical things have a heft and permanence. They're very accessible.

Tuesday, October 14, 2014

Meet the Engineers Behind Polyvore: Esha Pathak

Get to know the Polyvore engineering team! Here, Esha talks about how she works with big data, our tech stack and what makes up our engineering culture.

What do you do every day?

I’m on the infrastructure team, so I help maintain our backend infrastructure in our Data Center and Cloud on AWS. In addition to my day job, I love getting updates on all the latest fashion trends from our creative community. It’s pretty cool that I’m able to help build a platform that empowers people all over the world to express their style.

How does Polyvore’s scale affect your daily work?

As our Polyvore community grows, my team continues to scale our backend systems. With new features built on our data, I help create new Cassandra clusters, and scale the existing Cassandra clusters by adding more nodes to accommodate our growing data. Cassandra is our backend for storing our unique outfit data for our newly launched Style Graph, which provides users with personalized product recommendations.

Tuesday, October 7, 2014

Software Engineers Wanted....For Style Advice

"Software Engineers Wanted...For Style Advice" - not two things that you'd necessarily expect in the same sentence. But that’s what we do here at Polyvore, using machine learning and data science.

Today we’re excited to announce the launch of personalized recommendations on Polyvore for iPhone. You can download the app here!

Although the user experience of a personalized feed is simple and delightful, what goes on under the hood is quite complex. Personalization is no easy feat, but we were able to create hyper-personalized recommendations at scale that produce a 4x increase in product likes (one of our key measures of engagement).

Over the next few months, we’ll be doing a series of blog posts about how it all works. To get started, here's a quick infographic explaining the problem, our approach, and the results:

So, why is style such a hard problem?

Style is complex and nebulous. It's intangible, highly personal, and constantly evolving. Think about how you would try to go about describing your style or the kind of image you want to project. Think about all the details that need to be taken into account: the occasion (wedding vs. vacation), time of year (winter vs summer), situation (work vs. weekend), all the way through to tiny details that you love or hate (rhinestones and fringe, anyone?). It's not a straightforward problem with clear rules that you can encode.

Wednesday, August 20, 2014

The 10 Biggest Surprises of My Polyvore Internship

Hi, my name is Julia Alvarez and I am a junior at Brown University, pursuing a degree in computer science. You should know that I am not one of those programmers who has been coding since I was in diapers. In fact, I switched from psychology to international relations to English and finally to computer science.

I "discovered" computer science after stepping into the opening day skit of Brown’s legendary CS15, an "Introduction to Object-Oriented Programing" course, and was hooked. I found that the beauty of CS was that I could actually make something useful, all while sitting at my desk. I wanted to learn more and to practice this magical "making" process, so I started hunting down a summer internship.

I had a mentor who did not work at a tech company, but knew many people who did. When she told me she knew some people who worked at Polyvore, I got really excited, because I had not only used Polyvore before, but I really liked it, and really believed (and still do) they had huge potential for growth. When they called me and told me they would like to interview me, I of course freaked out and said yes right away.

It has been ten weeks since I began my internship, and I would like to share the 10 most surprising things I have learned during my time here.