Skip to content
Main Site News Console

Zhipu AI's New Model Ranks #1 Globally in Legal Capabilities

· 量子位
国内AI

The AI wave has finally reached the legal sector.

Zhipu is bringing intense LLM competition directly into the legal domain!

Just moments ago, Zhipu unveiled a brand-new open-source legal-specialized large language model—GLM-Legal.

It doesn’t just chat about statutes and regulations; it handles end-to-end legal tasks like contract review, evidence chain reconstruction, and cross-border compliance research—directly delivering production-ready, professional work products.

Benchmark results are equally impressive.

In Vals AI’s latest legal evaluation, GLM-Legal secured the #1 spot among open-weight models on both the Harvey LAB and LegalBench leaderboards, achieving a 19.2% full-task pass rate and an 86.23% accuracy score, respectively.

After pushing the envelope in daily chat, office collaboration, and coding assistance, AI is finally diving into high-barrier, precision-demanding work like legal contracts and case files.

The announcement quickly sparked widespread discussion across the community:

GLM-Legal is officially my in-house legal team from now on.

But wait—which legal jurisdiction was it trained on? Can it handle domestic legal workflows effectively?

Let’s take a closer look.

The good news is that it is deeply tailored to domestic legal systems, showing zero signs of “jurisdictional mismatch.”

Whether it’s Chinese legal practice, English legal tasks, or complex matters like cross-border compliance in Southeast Asia, it handles them with ease.

Yet benchmarks are ultimately just theoretical tests.

Real-world legal practice is riddled with risks and hidden pitfalls, where a single misstep can lead to severe consequences.

Whether it truly delivers practical utility must be tested in real-world scenarios—seeing how well it helps teams redline contract clauses or verify pre-launch privacy compliance abroad.

Let’s first examine three official demonstrations (a head-to-head matchup between GLM-5.3-Flash and GLM-Legal):

1. Reviewing Contracts and Sorting Out Post-Termination Handover Terms

When a company needs to switch software service providers, the first priority is usually clarifying the termination and transition terms in the contract:

Can all code, data, and documentation be fully retrieved? Can the new vendor ensure a seamless handover?

These potential risks are often buried in fine-print clauses.

In response, GLM-Legal recommended reducing the termination notice period from 60 days to 30 days and adding a transition support clause for up to 90 days, giving the team ample time to manage code, documentation, and knowledge transfer.

Additionally, taking into account the contract’s existing IP/work product ownership provisions, it explicitly required the return of all work products (including incomplete or unbilled deliverables).

Compared to GLM-5.3-Flash, GLM-Legal’s advantage is striking:

While the former offered only high-level, generic advice, GLM-Legal specified concrete timelines, the scope of handover, and other critical details, tightly aligning proposed amendments with existing contractual terms.

The entire transition and handover mechanism upon termination was structured cleanly and methodically.

What about when faced with a chaotic pile of case evidence? How does GLM-Legal perform?

2. Analyzing Evidence and Connecting Dispersed Clues

This addresses a very real practical pain point:

When a project dispute arises, how can we sift through massive volumes of reports, working documents, and chat records to identify contradictions and loopholes in the opposing party’s claims?

Take the official case study as an example: an external report claimed project completion at 60%, whereas actual internal logs recorded only around 41%.

Testing showed that both GLM-5.3-Flash and GLM-Legal identified this discrepancy;

however, the difference lies in execution: GLM-5.3-Flash stopped at providing recommendations, whereas GLM-Legal not only offered advice but also linked the progress report to earlier risk warnings and internal correspondence, directing the user toward specific verification actions.

Only by establishing a comprehensive chain of evidence can a team quickly trace back to primary documents, verify facts item-by-item, and determine which data points are credible and usable to support future claims.

3. Cross-Border Enterprise Compliance: Turning Regulatory Mandates into an Actionable Checklist

When enterprises expand website operations into overseas markets such as Singapore, Malaysia, or Indonesia, they face compounding data privacy compliance challenges across multiple touchpoints—including user tracking, email marketing, and cross-border cloud storage.

In such scenarios, teams urgently need a specialized assistant to assemble an essential pre-launch checklist.

During testing, GLM-Legal broke down complex, interwoven regulatory requirements into clear, actionable steps—such as publishing localized privacy notices and logging unsubscribe requests into a suppression list—with remarkable clarity.

Operational efficiency improved substantially as a result.

Looking back in light of this practical performance, it becomes much easier to grasp why Harvey LAB’s 19.2% full-pass rate is such a demanding—and meaningful—milestone.

A 19.2% Full-Pass Rate: How Difficult Is This Benchmark?

The Harvey LAB evaluation benchmark is notorious for its rigor.

It employs an “all-conditions-met” scoring mechanism, requiring the model to satisfy every single grading criterion in a task simultaneously to pass; missing even one criterion results in a failure.

Its task design directly mirrors the workflow of top-tier law firms: an AI agent is provided with business requirements and an entire packet of raw client materials, and must directly output partner-ready, compliant legal deliverables.

In its inaugural version alone, Harvey LAB encompasses over 1,200 simulated law firm practice tasks spanning 24 core practice areas, including M&A, intellectual property, and banking & finance.

This means that from ingesting and understanding documents, to tool orchestration, to structuring outputs according to industry standards, the model cannot falter at any step of the pipeline to score well.

Beyond the benchmark scores, the comparison of testing costs is equally telling.

With comparable final scores, GLM-Legal was roughly 30 minutes slower than Gemini 4 Argon, but Gemini’s cost per test was approximately 70x that of GLM-Legal:

These test findings offer a key takeaway for the industry: when selecting a domain-specific legal assistant, organizations must evaluate capability, inference cost, and latency holistically.

So, how exactly was GLM-Legal trained?

The answer lies in Zhipu’s tailored model progression roadmap.

Not Just Studying Law: Putting AI Through a “Law Firm Internship”

Zhipu’s training roadmap for GLM-Legal can be summarized as:

First, send it to “law school” to build foundational knowledge; then, place it in a “simulated law firm” for hands-on practice.

Step 1: Using GLM-5.3-Flash as the base model, conduct intensive mid-training on dense legal-domain corpora, delving deep into legal concepts, statutory logic, and case analysis frameworks to cement professional legal foundations.

Step 2: Subject the model to reinforcement learning across tens of thousands of legal tasks, repeatedly honing skills such as document analysis, logical reasoning, procedural workflows, and deliverable drafting.

Simultaneously, the team continuously refined the Agent execution framework—optimizing document retrieval and structuring logic, tool-calling interfaces, and feedback loops—to ensure deep integration with real-world workflows.

This training methodology closely mirrors the professional development trajectory of human lawyers:

From entering law school to pore over classic treatises and build doctrinal understanding, to interning at law firms during upper-level studies, and finally climbing the ranks from junior associate to senior attorney through years of practical experience.

What AI is replicating today is the quintessential growth path of legal professionals.

Objectively speaking, GLM-Legal is still in the entry-level stage of its legal career and is not yet on par with senior partners.

This “legal specialist” still requires extensive validation in real-world practice across multiple dimensions. A 19.2% full-pass rate also indicates that under strict, multi-criteria evaluations, there is plenty of room for improvement on complex tasks.

This aligns directly with Zhipu’s current rollout strategy:

Officially inviting law firms, legal tech companies, in-house legal departments, and solo practitioners to join the GLM-Legal Beta test, continuously fine-tuning and polishing the model through active practitioner participation and feedback.

At the same time, Zhipu plans to integrate these legal capabilities into its AutoClaw platform to accelerate adoption in production environments.

This sends a clear industry signal:

Zhipu is collaborating with seasoned legal professionals to co-iterate GLM-Legal, with the ultimate goal of transforming it into an indispensable productivity tool for practicing lawyers.

Whether Zhipu can fully realize this industry vision will be closely watched by the market.

One More Thing

Zhipu’s move into vertical legal models is by no means an impromptu pivot.

As early as 2023, Zhipu partnered with PowerLaw, a legal tech company specializing in intelligent contract solutions, to co-release PowerLawGLM—a 100-billion-parameter legal domain model based on ChatGLM-130B, widely adopted for core workflows such as legal advisory, contract drafting, and contract review.

Since then, Zhipu, as an existing shareholder, has continued to back PowerLaw, participating in strategic funding rounds and a nearly 80-million-RMB Pre-B round.

Zhipu is far from the only player adapting general-purpose foundation models for legal specialization.

In August of this year, legal AI unicorn Harvey teamed up with Fireworks to build Tenet, a proprietary legal model post-trained on top of the Kimi K3 base model.

According to Harvey, this targeted post-training enabled the model to double the task completion volume of the base model on LAB holdout tasks while keeping cost structures relatively stable.

These pioneering explorations point to a clear evolutionary path for domain-specific models:

Selecting a high-quality general foundation model, followed by continuous deep training grounded in high-value domain data, vertical task scenarios, and real-world working environments, is the proven blueprint for achieving a quantum leap in specialized model capability.

Moving forward, whoever can teach the model its “core professional curriculum” more thoroughly, build “practical training grounds” closer to real-world workflows, and equip it with the hard skills needed to solve genuine business challenges will ultimately earn deep market and user adoption in vertical domains.

References:
[1] https://x.com/Zai_org/status/2109678174468534511?s=20
[2] https://z.ai/blog/glm-legal
[3] https://x.com/ValsAI/status/2109678921017168061
[4] https://www.harvey.ai/blog/introducing-harveys-legal-agent-benchmark?utm_source=chatgpt.com
[5] https://media.qimingpian.cn/2023/06/29/bd6b454d1bf2400827f682a6c5e7eacf.html?utm_source=chatgpt.com