Image or text: Which one is more influential? a deep-learning approach for visual and textual data analysis in the digital economy

In a digital economy, different types of information about products communicate their quality and characteristics to prospective consumers. However, it remains unclear which type of information plays the most important role in individuals’ decision-making processes. In this study, we explore the effect that unstructured data has on and the importance of congruence between textual and visual data in consumers’ purchase decisions. We apply a deep neural network model to rank the importance of different information types and use a regression model to investigate the impact that information consistency has on sales predictions. Based on our empirical analysis, we found that both image-based and text-based information influenced consumers’ purchase decisions but that the former influenced their purchase decisions about “search goods” more and that the latter influenced their purchase decisions about “experience goods” more. Furthermore, congruence between image-and text-based information was positively associated with purchase decisions, which indicates that information congruence impacts products’ sales performance in the digital economy. In this study, we also demonstrate how to apply advanced deep-learning techniques to measure the congruence between different information types.