Finding the ImageNet of Finance | Qifu Technology Live Recording: How Does Credit Multimodal AI Set Standards?

Recently, Qifu Technology, in collaboration with researchers from Fudan University and South China University of Technology, initiated a live discussion themed “How Can Multimodal Credit AI Set Standards?” The session provided an in-depth analysis of FCMBench-V1.0, the first multimodal evaluation benchmark designed specifically for credit scenarios. This benchmark focuses on key aspects such as multimodal perception, reasoning, and decision-making, and is accompanied by open-source datasets and evaluation tools, aiming to establish a widely recognized “standard” for financial AI. The entire presentation lasted one hour, blending academic cutting-edge research with industry practice, offering professional insights and development ideas for financial institutions, research institutes, and industry practitioners. Below is a summary of the core content of this live session.

Industry Practice Perspective: FCMBench Provides a Unified Measure of Financial AI Model Capabilities

Yang Yehui, head of multimodal at Qifu Technology, first analyzed the development pain points of financial AI from an industry perspective, explaining the original intention and core design logic behind FCMBench-V1.0. He vividly compared AI to a “hoe,” a tool, while industries like finance and healthcare—high-threshold sectors—are “fertile lands” with development potential. The inherent high demands for privacy, security, and compliance in financial services mean that model capabilities cannot be validated through self-assessment alone; instead, a set of objective, unified evaluation systems must be established.

The creation of FCMBench-V1.0 aims to address the core confusion faced by financial institutions when selecting models. Yang pointed out that currently, different models claim high scores but lack a standard for comparison, and their performance often drops significantly when moving from laboratory to real-world environments. The core value of FCMBench is to serve as a “standard ruler” for measuring model capabilities, bringing different models onto the same starting line and testing their abilities under real business conditions.

Regarding the design principles of this “ruler,” Yang proposed three key tenets: fairness, scientific rigor, and practicality. Fairness prevents self-serving claims and establishes a baseline for evaluation; scientific rigor ensures that data distribution, task design, and difficulty levels are reasonable and capable of distinguishing algorithm differences; practicality is central, aiming for models that perform well on benchmarks to directly adapt to real-world scenarios.

To make evaluations more aligned with actual business needs, FCMBench simulates over ten real-world interference scenarios, such as document information verification and multi-document comparison reasoning tasks, to replicate various risk scenarios in credit operations. For example, if a user reports an annual income exceeding 500,000 yuan but has a tax rate below 10%, this obvious risk point is included as a reasoning challenge in FCMBench, testing the model’s risk identification and anti-fraud capabilities, ensuring the evaluation tasks have practical value.

Yang believes FCMBench is not just “for the sake of it” but aims to support business and industry development. It is positioned as a public resource for the financial sector, seeking to deeply integrate AI capabilities with business value through a unified standard. Additionally, FCMBench acts as a bridge between academic research and industry application for large financial models. Technologically, it will continue to expand tasks, data types, languages, and modalities to cover all credit AI scenarios; industry-wise, it will collaborate with universities to tackle technical challenges, invite banks and financial institutions to participate in co-creation, enrich real-world data and scenarios, and promote it to become an industry-recognized evaluation standard or even a group standard—serving as a practical threshold for model selection and collaboration in financial institutions.

Academic Perspective: The “ImageNet Moment” for Financial AI Is Urgently Needed

If industry focuses on “how to use the ruler,” academia is more concerned with “why the ruler is missing” and how to create a truly credible “standard.”

Professor Chen Tao from Fudan University approached the issue from the history of AI development, pinpointing the core problem: “The development of large AI models heavily depends on open-source ecosystems, but the financial sector currently lacks a universally accepted, standardized evaluation dataset and benchmark. Without a unified ‘ruler,’ companies and academia find it difficult to collaborate on research, hindering the formation of a robust development ecosystem, which fundamentally constrains the emergence of large financial models.”

He pointed to the milestone of deep learning—ImageNet. “ImageNet dataset propelled the explosion of deep learning and became a standard benchmark for image recognition. Similar evaluation standards are key to breakthroughs in AI.” Chen argued that the financial field currently lacks such a comprehensive, unified evaluation dataset, making it difficult to foster a collaborative development ecosystem. There is an urgent need to create an industry-specific “ImageNet.”

Regarding FCMBench-V1.0, Chen praised it as one of the most extensive and authoritative unified evaluation benchmarks in the domestic and international financial credit domain. Compared to other fragmented industry datasets, FCMBench-V1.0 is the first to achieve modality unification, covering core tasks such as credit and risk control, all designed with real business scenarios in mind. Its features—developed by Qifu Technology and industry pioneers—make it comprehensive and practical, representing an important step toward building a proprietary “ImageNet” for finance.

Industry-Academia-Research Integration Perspective: The Clear Advantages of Financial AI Deployment and FCMBench’s Role in Talent Development

Professor Xu Yanwu from South China University of Technology explained the current state of financial AI application, its deployment advantages, and FCMBench’s significance in industry talent cultivation.

He first clarified a common misconception: “Many people intuitively think that AI has a weak presence in finance, but that’s not accurate. AI is already deeply involved in core scenarios like insurance pricing, asset valuation, and quantitative trading. These values are just not directly visible in consumer-facing products, so it appears less obvious.”

Xu also highlighted that compared to high-threshold industries like healthcare, financial AI has significant deployment efficiency advantages—potentially dozens or even hundreds of times higher. This advantage stems from the ability to quickly verify model effectiveness through historical data backtesting and parallel testing of dual models, enabling rapid model adjustments. In contrast, changing algorithms in healthcare requires redoing extensive clinical trials, which can take three to five years, making operational costs vastly different.

For building financial datasets, Xu emphasized three core elements: value-driven, comprehensive and meticulous, and fair and inclusive. High-quality financial datasets should address real industry problems with innovative topics, be designed to meet multi-dimensional application needs, and employ evaluation methods that are fair and based on public industry values rather than private interests.

The launch of FCMBench aligns with these principles and also plays a vital role in industry talent development. Xu stated that FCMBench serves as a crucial link connecting talent cultivation with industry needs, helping to improve the talent pipeline. It provides students pursuing AI with practical industry scenarios, enhancing their employability, and offers algorithm students realistic financial application contexts, helping them quickly adapt to industry requirements. This, in turn, supports the continuous supply of high-quality talent to the financial sector and helps build a complete industry talent echelon.

In this live session, the three experts from industry practice, academic research, and industry-academia integration engaged in in-depth discussions on the construction of multimodal credit AI standards, providing the industry with a clearer understanding of the current status, challenges, and future directions of financial AI. With the ongoing operation and co-creation of FCMBench-V1.0, and increased participation from financial institutions and research institutions, the financial sector is expected to gradually develop an open-source ecosystem similar to ImageNet. This will enable deeper integration of AI technology with financial services, promote standardization and normalization of financial AI, and ultimately drive technological breakthroughs and industry implementation through mutual empowerment.