Reining in AI power

In today's rapidly evolving AI era, algorithms are no longer confined to mathematical experiments in the lab. They have become "invisible decision-makers" deeply embedded in economic and social life, determining what content people see, whether a loan is approved, whether a resume gets screened out, and even assisting with medical diagnoses, judicial proceedings and urban governance.

The screen at the inaugural Global Dialogue on AI Governance in Geneva, Switzerland, on July 6 (XINHUA)

From basic AI algorithms to artificial general intelligence and even super intelligence, algorithmic decision-making is now everywhere. It has brought unmatched efficiency and unprecedented uncertainty. However, algorithms may make mistakes or even get out of control, and when these systems fail, the fallout can be disastrous, potentially affecting tens of thousands of people. So, how can we entrust machines with part of our decision-making power while still feeling confident in the outcome?

A targeted governance framework is needed to ensure that algorithmic decision-making is transparent, fair and safe. It must be recognized that risks vary enormously across different algorithmic applications. A one-size-fits-all approach will not work. Instead, governance must be risk-based and context-specific: conducting strict supervision and adopting rigorous standards for high-risk applications, and implementing flexible and prudent oversight for low-risk scenarios. This approach calls for a set of safeguards that can address risks at different stages of the AI lifecycle.

Transparency and traceability

AI algorithms are often "black boxes," making their decisions difficult to understand or explain. Users may have no idea why they are treated in a particular way, while regulators struggle to trace the internal logic behind algorithmic operations. If we do not even know the "why," how can we expect people to trust these systems or seek redress when something goes wrong?

For these reasons, algorithmic transparency and traceability are crucial. This does not mean requiring companies to disclose their proprietary source code or confidential technologies. Rather, it means being transparent about AI-generated content and interactions with AI systems, and providing explanations or clarifications of algorithmic decisions when necessary. Going forward, for high-risk areas such as lending, recruitment and healthcare, we should refine tiered requirements for traceability. The greater the impact of a decision, the clearer the explanation should be.

It's no exaggeration to say that transparency is becoming the key to understanding, trusting and governing AI. It can help address problems such as algorithmic discrimination, biased decision-making and AI-enabled deception, while also providing researchers and regulators with a real-world window into AI systems and first-hand data for assessing and addressing AI safety risks.

Yet transparency alone cannot ensure that algorithms make decisions in line with human interests. That requires value alignment to be embedded in model design.

Algorithms have no inherent moral compass, but the objective functions we set when designing them inevitably reflect certain values. If the sole objectives are click-through rates and conversion rates, algorithms may feed users vulgar content and intensify social biases. But if fairness, safety and social good are embedded from the start, algorithms can become trustworthy assistants. This is the widely discussed issue of "value alignment": ensuring that the goals and behavior of AI systems remain consistent with human objectives, values and ethical principles.

In China, this is reflected in the people-centered governance approach and the idea of developing AI for the positive and for good. Value alignment needs to be integrated into AI training and development through engineering and standardization. Alignment requirements should be incorporated throughout training data filtering, model training and safety testing. Through alignment training, principles such as honesty, fairness and non-harm can be embedded deep within AI systems, giving models the ability to distinguish "right from wrong and good from evil," rather than patching problems after they occur.

As the capabilities of large AI models continue to advance, value alignment is increasingly becoming a crucial lever in AI safety governance. The incorporation of this concept into the AI Safety Governance Framework 2.0, released by the Standardization Administration of China's National Technical Committee 260 on Cybersecurity in September 2025, is a response to this trend.

Humans' final say

Even with such safeguards in place, risks in algorithmic systems can be difficult to detect and may accumulate over time. Safety governance cannot always be reactive; it must be proactive. China's Cybersecurity Law has introduced specific provisions on AI, requiring enhanced risk monitoring, assessment and safety oversight. This is an important step forward.

Next, we should draw on established practices in areas such as workplace safety and pharmacovigilance (identifying and preventing the adverse effects of pharmaceuticals) to develop a tiered system for reporting and sharing information on major AI incidents. Whenever an algorithmic decision causes significant harm or shows serious signs of losing control, service providers should report the incident and take timely remedial action, so that every close call becomes part of the industry's collective safety memory.

At the same time, it must be acknowledged that algorithms can evolve in a matter of days, making it difficult for external regulation to keep pace. Therefore, risk prevention must also rely on internal corporate governance: establishing ethics committees, designating safety managers, conducting internal audits and implementing risk management throughout the entire AI lifecycle. In this way, safety responsibility can become an integral part of an organization's capabilities.

But monitoring risk alone is not enough. As AI systems take on increasingly consequential tasks, human oversight must remain in place, particularly for high-risk decisions. No matter how advanced technology becomes, one bottom line must never be crossed: Decisions concerning life, liberty or major rights and interests must ultimately rest with human beings. This reflects the internationally recognized principles of "human-in-the-loop" and maintaining human control: Algorithms can assist, suggest, and even make decisions, but humans must always have the final say. Algorithms should never completely replace human judgment.

Even the most intelligent system must leave room for people to review and override their decisions. Doctors can refer to AI diagnoses, but only they can sign prescriptions. Judges can use AI to find similar cases, but only they can issue the final ruling. "Human-in-the-loop" is not about putting the brakes on technology. It's about building a safety cushion for society. It preserves the possibility of correction, accountability and human dignity.

As AI agents grow increasingly autonomous, defining the boundary between automation efficiency and human control will become a top priority in future governance. In any case, AI systems must never sideline or marginalize humans as a matter of course. Meaningful human participation, oversight and control must remain firmly in place.

And when algorithmic systems do cause harm, there must also be a way to ensure that those affected can obtain redress and that those responsible face the consequences.

AI and algorithmic systems are fundamentally market products. They should never be considered or granted legal-person status. Their developers and providers should therefore be held accountable for harm caused by defects in these products.

To that end, existing product liability systems need to be reformed in areas such as the definition of product and producer, defect, compensable damage and causation.

One advantage of holding providers accountable under product liability and other tort rules is that victims need not prove complex technical errors. They only need to demonstrate that the AI algorithm was defective and that the defect caused the harm. And companies, to avoid compensation payouts, will step up investment in safety testing and risk management. Accountability is not the enemy of innovation but the guardian of safety. Only when being unsafe becomes costly will businesses have an incentive to choose safety.

In short, if we want AI, the "thoroughbred" of our time, to run both fast and steadily, it needs robust safety governance. Only with the use of both "helmets" and "reins" can AI become a trustworthy assistant that people can rely on—one that amplifies human capabilities and ultimately serves humanity and society for the better.