Compatibility Modeling: Data and Knowledge Applications for Clothing Matching

by Xuemeng Song (Author), Liqiang Nie (Author), Yinglong Wang (Author), Gary Marchionini (Editor)

Nowadays, fashion has become an essential aspect of people's daily life.

As each outfit usually comprises several complementary items, such as a top, bottom, shoes, and accessories, a proper outfit largely relies on the harmonious matching of these items. Nevertheless, not everyone is good at outfit composition, especially those who have a poor fashion aesthetic. Fortunately, in recent years the number of online fashion-oriented communities, like IQON and Chictopia, as well as e-commerce sites, like Amazon and eBay, has grown. The tremendous amount of real-world data regarding people's various fashion behaviors has opened a door to automatic clothing matching.

Despite its significant value, compatibility modeling for clothing matching that assesses the compatibility score for a given set of (equal or more than two) fashion items, e.g., a blouse and a skirt, yields tough challenges: (a) the absence of comprehensive benchmark; (b) comprehensive compatibility modeling with the multi-modal feature variables is largely untapped; © how to utilize the domain knowledge to guide the machine learning; (d) how to enhance the interpretability of the compatibility modeling; and (e) how to model the user factor in the personalized compatibility modeling. These challenges have been largely unexplored to date.

In this book, we shed light on several state-of-the-art theories on compatibility modeling. In particular, to facilitate the research, we first build three large-scale benchmark datasets from different online fashion websites, including IQON and Amazon. We then introduce a general data-driven compatibility modeling scheme based on advanced neural networks. To make use of the abundant fashion domain knowledge, i.e., clothing matching rules, we next present a novel knowledge guided compatibility modeling framework. Thereafter, to enhance the model interpretability, we put forward a prototype wise interpretable compatibility modeling approach. Following that, noticing the subjective aesthetics of users, we extend the general compatibility modeling to the personalized version. Moreover, we further study the real-world problem of personalized capsule wardrobe creation, aiming to generate a minimum collection of garments that is both compatible and suitable for the user. Finally, we conclude the book and present future research directions, such as the generative compatibility modeling, virtual try-on with arbitrary poses, and clothing generation.

About the Author
Xuemeng Song received a B.E. from the University of Science and Technology of China in 2012, and a Ph.D. from the School of Computing, National University of Singapore in 2016. She is currently an assistant professor of Shandong University, Jinan, China. Her research interests include information retrieval and social network analysis. She has published several papers in top venues, such as ACM SIGIR, MM, TIP, and TOIS. In addition, she has served as a reviewer for many top conferences and journals.

Liqiang Nie is currently a professor with the School of Computer Science and Technology, Shandong University. In addition, he is the adjunct dean with the Shandong AI institute. He received his B.Eng. and Ph.D. from Xi'an Jiaotong University in 2009 and the National University of Singapore (NUS) in 2013, respectively. After his Ph.D., Dr. Nie continued his research at NUS as a research follow for three and a half years. His research interests lie primarily in multimedia computing and information retrieval. Dr. Nie has authored and coauthored more than 100 papers for SIGIR, ACM MM, TOIS, and TIP, and received more than 4400 Google Scholar citations. He is an AE of Information Science, and an area chair of ACM MM 2018/2019.

Yinglong Wang is a researcher, Ph.D. Supervisor, and Party Secretary of the Qilu University of Technology (Shandong Academy of Sciences). He was declared a Young and Middleaged Expert with outstanding contributions to Shandong Province and High-End Think Tank Expert of Shandong Province, and he enjoys special government allowances from the State Council. He serves as the vice chairman of the Shandong Science and Technology Association, the president of the Shandong Internet of Things Association, the director of the China-Australia International Health Technology Joint Laboratory, a member of the Shandong Informatization Expert Group, a member of the Shandong Informatization Expert Advisory Committee, and the deputy chairman of the Shandong Information Standardization Technical Committee. Dr. Wang's main research areas are medical artificial intelligence and high-performance computing. In recent years, he has taken charge of more than 20 national, provincial, and ministerial projects. The scientific research projects led by him won 2 first, 4 second, and 2 third prizes at the Shandong Science and Technology Progress Awards. He has published more than 60 top academic papers and owns more than 20 authorized invention patents. Moreover, he organized the compilation of three volumes of national standards.

Gary Marchionini is the Cary C. Boshamer Professor of Information Science in the School of Information and Library Science at the University of North Carolina at Chapel Hill. His Ph.D. is from Wayne State University in mathematics education with an emphasis on educational computing. His research interests are in information seeking in electronic environments, digital libraries, human-computer interaction, digital government and information technology policy. He has had grants or contracts from the National Science Foundation, the U.S. Department of Education, the Council on Library Resources, the National Library of Medicine, the Library of Congress, the Kellogg Foundation, and NASA, among others. He was the Conference Chair for the 1996 ACM Digital Library Conference and program chair for the 2002 ACM-IEEE Joint Conference on Digital Libraries. He is editor-in-chief for ACM Transactions on Information Systems and serves on the editorial boards of a dozen scholarly journals. He has published more than 150 articles, chapters, and conference papers in the information science, computer science, and education literatures. He founded the Interaction Design Laboratory at UNC-CH.