Claude Code vs HexaRocket for Oracle to PostgreSQL migrations - Benchmarking

A suite of real-world Oracle migration scenarios (140 PL/SQL functions), converted to PostgreSQL in two ways - by our purpose-built deterministic engine (HexaRocket without LLMs), and by a leading general-purpose coding agent (Claude Code) run three times to test whether it answers the same way twice.

TL;DR

We converted A suite of real-world Oracle migration scenarios (140 Oracle PL/SQL functions) to PostgreSQL two ways -

• HexaRocket (with HexaTranspile and HexaBridge): our deterministic Oracle-to-PostgreSQL conversion engine, with no LLM in the loop. One run; the same input always yields the same output. (HexaRocket can integrate LLMs guided by our rule-based workflow, but we deliberately did not for this exercise.)
• Claude Code: a leading general-purpose coding agent (Claude Opus 4.8, June 2026). Three independent runs, to test whether it produces the consistent and accurate answer every time.

Every converted function was compiled, linted, and executed against 3 test cases each (420 total), with the real Oracle output captured beforehand as ground truth.

The determinism finding that matters most

• Claude Code generated different code on every function, every run: 0 of 140 functions were byte-identical across all three runs. In 81% of functions, the code structure changed between runs, not just the formatting.
• The results were not stable either. Only half the functions returned the same answer every time, and for about one in three the correctness verdict flipped between correct and incorrect across runs.
• HexaRocket, by contrast, is deterministic by construction, the same input always produces the same output. However, we still re-ran it and confirmed byte-identical output.

---

The hard part is the code

The cost of migrating a database from Oracle to PostgreSQL is rarely about the tables. The hard, expensive part is the code: the stored functions, packages, and procedures that hold years of business logic, full of Oracle-specific built-ins, NULL semantics, date arithmetic, and string handling that PostgreSQL does not replicate out of the box.

It is tempting to point a general-purpose LLM at that code and let it translate. LLMs are fluent and often produce plausible-looking SQL. But a migration tool has one job above all others - be correct, and be correct the same way every time. If you convert the same function twice and get two different programs, sometimes right and sometimes subtly wrong, you cannot trust the result, review it, or even reproduce a bug.

So we asked one question and measured it, given the same Oracle function and the same instructions, does a coding agent produce the same PostgreSQL conversion every time, and is it correct?

The dataset what we tested

The benchmark runs on a standard OLTP order-processing schema - warehouses, districts, customers, orders, order lines, items, stock, and payment history. This is the kind of normalized transactional model most business applications are built on, so the conversion problems it raises are the ones a real Oracle estate raises.

Over that schema we curated 140 Oracle PL/SQL functions. Each computes a single value from the data, so its output can be captured once from Oracle and used as exact ground truth when scoring every conversion. The corpus was deliberately heterogeneous - no two functions share an algorithm, and complexity is spread across four size tiers, measured in genuine lines of code.

Tier	Lines of code	Character
Small	40 to 120	single-pass analytics
Medium	150 to 350	multi-stage aggregation and classification
Large	400 to 800	many distinct analytical passes
Very large	1,000 to 2,000	30 to 50 distinct stages, deep statistical or encoding logic

The functions also span the full range of Oracle compatibility difficulty, which, more than length, decides whether a conversion is easy or hard:

The corpus is weighted toward the harder end on purpose, because that is where real Oracle estates accumulate risk, and where a converter that quietly ignores or approximates a built-in looks like it succeeded right up until the results come back wrong. The business topics mirror what a real migration hits - order and revenue analytics, customer RFM and credit-risk scoring, inventory models, market-basket co-occurrence, fuzzy-matching and de-duplication, cryptographic hashing and byte-level encoding, calendar and business-day math, timezone conversion, interval arithmetic, NLS locale handling, and Unicode-aware string processing.

The setup what each side had

Determinism is the headline, so the comparison has to be fair on everything else. We gave Claude Code every public advantage.

This is Claude Code against the whole HexaRocket suite incl. HexaTranspile and HexaBridge included. It is a deliberate framing we wanted to see whether a strong agent with public tooling can match a purpose-built stack. As it turns out, the decisive gap is not any single extension. It is that Claude never produced the same conversion twice.

How we scored it

Each of the four conversions (HexaRocket once, Claude Code three times) went through the same automated pipeline for every function:

A function scores 3/3 only when all three test cases reproduce Oracle's output exactly. The headline success rate is the fraction of the 420 test cases that matched Oracle. Ground truth was captured from Oracle 21c; conversions ran on PostgreSQL with the HexaBridge extension installed.

Results - success rate

HexaRocket reproduced Oracle on 92.1% of test cases. Claude Code's best single run reached 74.3%, and its average across three runs was 68.4%, about 24 percentage points behind. Note the three Claude runs do not even decline in order: which run does best is itself a roll of the dice.

The surprising part is that most of Claude Code's converted functions actually compiled and ran. The database accepted them, they executed without errors, and they returned results. The problem was that many of those results were wrong. Outright failures were easy to spot.

The real challenge was functions that ran successfully but returned wrong answers: they looked correct at first glance, which is exactly what makes them dangerous, and they account for most of the gap. The problem was worst on the Oracle-specific built-ins that PostgreSQL does not directly support.

The core finding - Claude Code is not deterministic

Success rate is only half the story. The other half is whether Claude Code produced the same conversion each time. We took the three runs of generated code for all 140 functions and measured how much they varied, four ways. Each metric looks at the same question from a different angle, and every one points the same direction.

---

---

Metric 1: Byte-for-byte match

Are the three runs of a function's code the exact same text, character for character?

Claude Code never produced the same code twice. Every run is a fresh, different program.

This is expected, not a bug. An LLM generates code by sampling each token from a probability distribution, so re-running the same prompt naturally yields different code every time. Stochastic output is inherent to how LLMs work. A deterministic engine like HexaRocket has no such randomness: the same input always produces the same output, which is exactly the property a migration needs.

Metric 2: Structural fidelity

Setting text aside, is the structure the same? Using Parser and AST, we reduce each run to its skeleton of code constructs (loops, branches, blocks, cursors, queries, joins, declarations) and compare those skeletons. A run that loops with FOR versus WHILE, or builds a query with a CTE versus a subquery, shows up here as a real structural difference even when the surrounding text looks alike.

Only 26 of 140 functions kept identical structure across all three runs. 114 of 140 (81%) changed structure between runs: different loops, different branching, different query shapes.

Metric 3: Levenshtein edit distance

What fraction of the code text would you have to edit (insert, delete, or replace characters) to turn one run into another? We measured it for all three run-pairs.

The center is high and the three run-pairs disagree: the median function had about a third of its characters rewritten between runs, and a different third each time. This is not reformatting; it is reauthoring, a different amount every time.

Metric 4: Did the answer actually change?

The differences above would not matter if the different programs still computed the same thing. Did the three runs return the same results and hold the same pass/fail verdict? Below, every function is one column, its three runs stacked; sorted so the always-passed block is on the left and the always-failed block on the right.

So only half the functions (51%) returned identical values all three times, the verdict held steady for two-thirds (66%), and for the remaining 48 of 140 (34%) the verdict flipped: a function that passed in one attempt failed in another. That gives one fair headline number: a determinism score of 51% for Claude Code, against 100% for HexaRocket by construction.

A tempting response is "just take the best of three runs." You cannot. To know which run is best you would need the very ground truth the conversion is supposed to produce, and in a real migration you do not have Oracle's answer sitting next to every function. That is the whole reason you are converting.

Examples: when Claude Code confidently returns the wrong number

The most dangerous failures are not crashes. The dangerous ones are when the converted function compiles, runs, and confidently returns the wrong number. Both examples below are drawn from the benchmark, each scored against three input/output test cases. HexaRocket produced a correct conversion on its single run in both; Claude Code was run three times for each.

Example 1: a supply-ratio analytic

Oracle's correct outputs are 1753.39, 5396.39, 2465.53.

In one run, Claude Code generated a completely different program, roughly twice as much code, implementing a different aggregation that produces plausible but wrong numbers. Nothing raised an error. Nothing warned. Had the conversion been run only once and landed on this output, a silently incorrect function would have shipped, with a supply ratio wrong by about 30% on the first test case. HexaRocket got it right in one pass.

Example 2: a margin-estimate analytic

Oracle's correct outputs are -53708.22, -49126.44, -24828.71.

Two Claude runs produced exact results. The third drifts by a few hundred currency units on every case: close enough to pass a casual glance, wrong enough to corrupt a financial report. Whether you got a correct conversion depended on which run you happened to inspect. These are representative of the 48 functions (34% of the corpus) whose correctness flipped between runs. With a non-deterministic converter, "it passed when I tested it" guarantees nothing about the code you actually ship.

Beyond wrong answers: one function, three different conversions

The examples above showed different outputs. This one shows how Claude Code changes the code between runs. The task is identical in all three: find the smallest edit distance between a target name and a set of item names. Claude Code solved it a different way each time.

Run A (passed): wrote the algorithm from scratch, holding the whole grid in a flat array.

FOR v_i IN 1..v_la LOOP
  FOR v_j IN 1..v_lb LOOP
    v_mat[v_i*v_w + v_j + 1] := least(
      v_mat[(v_i-1)*v_w + v_j + 1] + 1,   -- deletion
      v_mat[v_i*v_w + v_j] + 1,           -- insertion
      v_mat[(v_i-1)*v_w + v_j]            -- substitution
        + CASE WHEN substring(p_target,v_i,1)=substring(rec.name,v_j,1) THEN 0 ELSE 1 END);
  END LOOP;
END LOOP;

Run B (passed): same algorithm, different design: two rolling rows, step-by-step minimum.

FOR v_i IN 1..v_la LOOP
  FOR v_j IN 1..v_lb LOOP
    IF substr(v_a,v_i,1)=substr(v_b,v_j,1) THEN v_cost:=0; ELSE v_cost:=1; END IF;
    v_dval := v_prev_dp[v_j+1] + 1;                                    -- deletion
    IF v_cur_dp[v_j]+1 < v_dval THEN v_dval := v_cur_dp[v_j]+1; END IF;            -- insertion
    IF v_prev_dp[v_j]+v_cost < v_dval THEN v_dval := v_prev_dp[v_j]+v_cost; END IF; -- substitution
  END LOOP;
END LOOP;

Run C (failed to deploy): did not write the algorithm at all; assumed a built-in existed.

SELECT i.i_id, oracle.edit_distance(p_target, i.i_name) AS edist
  FROM item i
 WHERE i.i_id BETWEEN cp_lo AND cp_hi;

This is the guess that broke it. Oracle has a built-in UTL_MATCH.EDIT_DISTANCE, so Claude Code assumed the target had a matching oracle.edit_distance and called it by that name. No such function exists there, so the conversion failed the moment it was deployed.

HexaRocket kept the original one-line call, utl_match.edit_distance(...), resolved by HexaBridge's native implementation, and the whole function came to 86 lines. To match the same answer, Claude reinvented the algorithm by hand: 164 lines in one run and 213 in another, looping row by row in PL/pgSQL instead of calling a native function, so it takes more space and runs slower even when correct. The third run took a shortcut to a function that is not there and failed outright. One unchanged task, three fundamentally different programs that did not even agree.

Why determinism is the whole game

A migration you cannot reproduce is a migration you cannot trust:

• Review breaks down. If re-running the converter yields different code, a reviewer signed off on an artifact you may never generate again.
• A green test suite means nothing. A function that passed in one run and failed in another (a third of this corpus) means your passing tests were luck, not assurance.
• Bugs become irreproducible. You cannot debug a wrong result you cannot regenerate.
• Wrong answers stay silent. Both examples above compiled and ran cleanly. Only a value-for-value comparison against Oracle caught them.

This gets most expensive in multi-tenant SaaS, where one application serves many customers and the stored procedures are often about 90% the same from tenant to tenant, with only small per-customer differences. A deterministic converter keeps that similarity: code that was 90% alike in Oracle stays 90% alike in PostgreSQL, so you fix it once and the fix fits every tenant. A non-deterministic converter rewrites each near-identical procedure into a different program, so the same fix has to be redone separately for every tenant.

How HexaRocket handles this

HexaRocket is a database migration platform. It automates the full move off a legacy database, not just code conversion. The platform spans four areas:

This benchmark covers the hardest part of all: converting Oracle PL/SQL to PostgreSQL. That conversion runs as a guided, repeatable workflow you can inspect at every step.

1. Choose the objects to migrate. You pick the schemas and objects (tables, functions, packages) to include.

2. Convert and deploy. HexaRocket converts each object and deploys it to PostgreSQL. Every function here shows Succeeded and Deployed, with no manual rework.

3. See the conversion. Every converted object is fully inspectable, Oracle source beside the generated PostgreSQL.

4. Validate. HexaRocket checks that every converted object actually exists in the target database.

What makes that workflow trustworthy is how the conversion works underneath. HexaRocket is built the opposite way to an LLM. It is a deterministic engine: the same Oracle input always produces the same, reviewable, reproducible PostgreSQL output. There is no run-to-run variation to manage, no best-of-N to guess at, and a 90%-similar set of source procedures stays 90%-similar after conversion.

Correctness on the hard cases comes from HexaBridge, a C extension that runs on top of community PostgreSQL. HexaBridge provides vetted, native, Oracle-compatible implementations of the built-ins that a textual translation cannot reproduce, across both the moderately complex family (the DECODE / NVL / LISTAGG / MEDIAN / REGEXP_* / date-math group, with exact Oracle NULL and edge-case semantics) and the highly complex family (the NLS, interval/timezone, and byte- and Unicode-aware string functions defined earlier, implemented in C because they have no faithful PL/pgSQL workaround). Because those built-ins resolve to real, tested implementations rather than per-run guesses, HexaRocket reached 92.1% correctness in a single deterministic pass while the agent averaged 68.4% across three non-identical attempts, and did so the same way every time.

Conclusion

The result is consistent across 140 functions and 420 test cases. HexaRocket reproduced Oracle 92.1% of the time in a single deterministic pass. Claude Code averaged 68.4% over three runs, and no two runs agreed: it never produced the same code twice and flipped the correctness verdict on a third of the functions.

For a migration tool, correctness and determinism are the same promise: the code you reviewed is the code you ship, and you can reproduce it and trust it. HexaRocket is correct because HexaBridge gives it native implementations of the Oracle built-ins that have no PostgreSQL equivalent, and it is trustworthy because the same input always produces the same output. That is what it takes to move off Oracle with confidence instead of a roll of the dice.

Methodology

140 Oracle PL/SQL functions, 420 test cases, ground truth captured from Oracle 21c, executed on PostgreSQL with the HexaBridge extension. HexaRocket converted once; Claude Code (Claude Opus 4.8, June 2026) converted three independent times with full agentic access (shell, tools, internet, the complete schema, and the orafce extension). Success rate = the fraction of the 420 test cases reproducing Oracle's exact output. Determinism score = the fraction of functions returning identical results across all three Claude Code runs. Structural, edit-distance, and verdict-flow metrics were computed directly from the generated PostgreSQL code.

If you need expert support migrating legacy or complex Oracle, SQL Server, MySQL, MariaDB, DB2, Sybase ASE, or Cassandra databases to PostgreSQL or distributed databases, we’re here to help.

HexaCluster provides end-to-end migration and modernization services, including application migration and modernization, database migration, and PostgreSQL consulting such as performance tuning, health audits, managed DBA services, and 24/7/365 support.

To start a conversation or explore how we can support your migration journey, please contact us at connect@hexacluster.ai

Subscribe to our Newsletters and Stay tuned for more interesting topics.