Interview with Tom Szumowski, Data Scientist at URBN

Automating fashion product attribution is something that many companies are offering through AI-based vision capabilities that perform common tasks like facial recognition, object detection, and text extraction. Often, however, when more specialised computer vision tasks are required, companies have historically had to build their own machine learning models from scratch.

URBN, the portfolio of global consumer brands comprised of Urban Outfitters, Anthropologie, Free People, BHLDN, Terrain and the Vetri Family, are working to build custom vision services to automate the attribution of fashion products, e.g. dress neckline, length, pattern. The Data Science team’s mission is to: identify opportunities for data-driven products and insights that benefit the brands and support the brands with deployment. These opportunities span across many disciplines, including customer user experience, engineering, logistics, marketing, business intelligence, marketing, and more.

Tom Szumowski, Data Scientist at URBN will be joining RE•WORK at the Deep Learning Summit in London this September 21 - 22 to discuss automated fashion product attribution and present a case study in using custom vision services vs. manually developed machine learning models. We caught up with Tom to learn a bit more about his career in deep learning and find out what to expect from his presentation.

How did you begin your work in deep learning and retail? What came first?

For me, deep learning came first. I joined URBN earlier in 2018 after spending several years working in the defense industry for Lockheed Martin. I was part of a corporate research lab, investigating methods to automate radio-frequency (RF) signals analysis. I was introduced to deep learning in late 2013 by a colleague who discovered the topic at a conference. At that time, the majority of our efforts were spent “feature engineering” in order to apply classical statistics or machine learning (ML). We saw deep learning as an opportunity to relieve that effort and map directly from raw signal to classification, and so our research began in early 2014.

As industries began adopting deep learning, I became interested in applications in other domains. That is what led me to retail earlier this year. Several people were surprised to hear of my jump from defense to retail, but “under the hood” a lot of the same technical approaches and math are the same! The differences lie in domain-specific subtleties.

What motivates you to keep working in this space?

For me, it’s a combination of the data and the domain. While there has been fantastic research in AI for fashion that dates back several years, the intersection between the two is still rather fresh. In my prior position, it was often a challenge to acquire relevant data, which sometimes ends up being a showstopper. Being at URBN, we’re lucky to have several years of e-commerce data to evaluate interesting models.

URBN also has an incredible pool of talented creative professionals, including: merchants, buyers, designers, photographers, shoppers, and stylists. Each has so much to offer in terms of domain expertise. I see a huge opportunity to support these creative roles with ML through a form of human-machine-interaction (HMI). It’s an example of how new technology doesn’t necessary displace jobs, but rather augments them to improve either productivity or overall product quality.

I’m also motivated by how URBN leadership engages technologists. There is a refreshing top-down push for innovation using new technologies. For example, URBN hosts a semi-annual Hackathon where all of IT and engineering are encouraged to bring forward out-of-the-box ideas and prototype them. The first one I participated in gave me the opportunity to work alongside members of other disciplines such as the photo studio to demonstrate cool uses of ML for URBN.

How are you using AI to automate fashion product attribution?

We’ve been investigating fashion product attribution to augment existing descriptions of our products. We believe a rich description of the product enables many opportunities such as better user experience through recommendations and search, improved catalogue management, and more detailed business reporting. Typically the products are categorized at a more coarse level for other use cases (e.g. short-sleeved floral A-line dress). The fine details are surfaced through rich imagery or free-form text descriptions. A finer-grained attribution can help tease out the subtle differences between products. For example, you may have two short-sleeved floral A-Line dresses, but one is small-print and one is large-print. Without an understanding of “print scale”, this difference may be lost. That difference can be critical for business intelligence. Finer-grained attributes may be achieved through explicit labels, or can be indirectly achieved through visual embeddings as well. We experiment with both.

What challenges are you currently facing in your work, and how are you using AI to overcome these?

One of the larger challenges is that fashion cannot simply be distilled into objective numbers. So we need to take care in selecting the problems that are more suitably solved by a machine via optimization. More concretely, one significant challenge is product segmentation. Urban Outfitters, Anthropologie, and Free People all place a heavy focus on lifestyle photographs. These are photographs with models wearing the product, often part of an entire outfit, and sometimes with backdrops like desert landscapes or urban scenery. This rich, detailed imagery is fantastic to provide immersive context for the consumer. But compared to products photographed as flats, this makes it challenging for us data scientists to isolate the product from everything else. So instead of using images as-is, we require a pipeline to localize the object of interest first (via object bounding boxes or masking approaches). With that said, we’re excited to leverage these rich lifestyle photos to support potential future initiatives such as outfit styling or adding more context to the product.

How are you using AI for a positive impact?

I’m most excited for the impact in overall customer experience. In my opinion, URBN is a very customer-centric organization. Our team met with a personal stylist from Anthropologie who shared stories of interactions between her team and customers seeking style advice. Both Anthropologie and BHLDN provide customers the opportunity to interact digitally or in-person with stylists for advice on pretty much anything. The requests are diverse, with examples including: fit recommendations given body type, recommendations for a summer wedding guest dress, what other blouses go well with the skirt they bought, and more. It’s amazing to see how effective this approach is and more importantly, how thrilled the customers are with their selections afterward. If we can use ML to help those types of initiatives, they can bring tremendous value to the customer and subsequently to the business.

What are some other real-world applications of your work - what other industries could benefit?

My background was originally more in traditional statistics, optimization, and estimation theory. When it comes to deep learning, I have typically taken a more pragmatic perspective. It’s provided huge advancements in the field but it’s important to look at your raw data, transformations, model inputs, and model outputs very critically. I’d like to say all industries can benefit from deep learning but I think it’s more appropriate to say that all industries can benefit from data-driven approaches. Whether or not deep learning is the solution is very application-specific and should be left up to a quantitative comparison with simpler techniques that are carry benefits in being more interpretable and manageable.

AI is becoming applied in countless industries - what areas are you most excited to see transformed, and where do you think we’ll see the biggest impact?

AI has transformed the way many large corporations operate. I’m interested to see how that will trickle down to smaller organizations with less resources. Take small non-profit or humanitarian organizations. Given limited resources for staff, an on-site data scientist doesn’t make practical sense in many cases as other roles play a higher priority. However, they can likely benefit from even just more traditional data science products or processes to inform decision making, let alone deep learning. I’m excited to see groups emerging that are working to pair skilled data professionals and organizations with missions that would benefit from them. I’m also excited to see how resources to enable these are becoming more democratized through open access courses and accessible compute technology. In the near future, I expect data science and ML to weave into other professions, equipping a new generation of professionals with more-accessible data tools to power their discipline.

What’s next for you?

I’m excited to learn more about how these new technologies are supporting other businesses and domains, which is what steered us toward re-work. I’m also excited to learn more about this domain and discover opportunities where machine learning can support it. I’m eager to continue to sit down alongside the various creative teams at URBN to collectively identify tools and processes that enhance their everyday workflows. For me, plugging numbers into the latest, greatest neural network isn’t what drives forward progress. It’s the cross-pollination of these advanced technologies with the domain experts that transform industries.