The key insight, as the paper puts it, is that -

The product structure of one of the prototypical qLDPC codes, HGP codes, naturally matches the product structure of crossed AOD optical hardware, enabling its hardware-efficient implementation

I wrote some Python code to reproduce a few of the paper’s results, and this article explains it. The code of my work can be found at the GitHub repository.

The paper’s methods section provides the formula for computing the hypergraph product of two classical codes:

To implement this, I wrote the hypergraph function:

def hypergraph(H1, H2):
    """Hypergraph product of two classical codes. Returns (Hx, Hz)."""
    H1 = H1.copy().astype(np.uint8) % 2
    H2 = H2.copy().astype(np.uint8) % 2
    r1, n1 = H1.shape
    r2, n2 = H2.shape
    def I(k):
        return np.eye(k, dtype=np.uint8)
    Hx = np.hstack([np.kron(np.transpose(H1), I(r2)),
                    np.kron(I(n1), H2)])
    Hz = np.hstack([np.kron(I(r1), np.transpose(H2)),
                    np.kron(H1, I(n2))])
    return Hx, Hz

But which classical codes do we feed as inputs to this function? The paper then tells us exactly which classical codes we are HGP-ing together:

So we generate a random (3, 4)-regular Tanner graph and look at the corresponding parity check matrix. To do this, I defined the function regular_seed:

def regular_seed(n_bits, wc=3, wr=4, seed=0):
    """Random (wc, wr)-regular parity check matrix"""
    assert (n_bits * wc) % wr == 0
    r = n_bits * wc // wr
    bit_stubs   = np.repeat(np.arange(n_bits), wc)   # [0,0,0,1,1,1,2,2,2,...]
    check_stubs = np.repeat(np.arange(r), wr)        # [0,0,0,0,1,1,1,1,2,2,2,2,...]
    rng = np.random.default_rng(seed)
    while True:
        rng.shuffle(bit_stubs)
        H = np.zeros((r, n_bits), dtype=np.uint8)
        for i in range(n_bits * wc):
            H[check_stubs[i], bit_stubs[i]] += 1
        if np.all(H <= 1):
            break
    assert np.all(np.sum(H, axis=0) == wc) # assert the column sums are all wc
    assert np.all(np.sum(H, axis=1) == wr) # assert the row sums are all wr
    return H

But the condition “(3, 4)-regular Tanner graph” specifies an entire ensemble of classical codes. The regular_seed function just produced a random matrix from the ensemble. What if we want to produce a particular matrix from the ensemble that also satisfies some good properties? Indeed, the paper suggests the following criteria:

1. Filter the ensemble of (3, 4)-regular Tanner graphs to only consider Tanner graphs with girth ≥ 6. (This property is useful in decoding error syndromes.)

2. Within the set of (3, 4)-regular Tanner graphs with girth ≥ 6, prioritize those with the largest distance.

3. Within the set of (3, 4)-regular Tanner graphs with girth ≥ 6 and maximum distance, prioritize those with the largest spectral gap.

The standard way to do this — the way I tried doing it at first — is to first define the following functions:

def best_seed(n_bits, trials=100):
    """Sample many seeds, keep the one with the largest exact distance."""
    best_H, best_d = None, -1
    for s in range(trials):
        H = regular_seed(n_bits, wc=3, wr=4, seed=s)
        d = exact_distance(H)
        if d > best_d:
            best_H, best_d = H, d
    return best_H, best_d

def has_four_cycle(H):
    """True iff two checks share two or more bits. Equivalent to girth < 6."""
    G = H.astype(np.int64) @ H.astype(np.int64).T
    np.fill_diagonal(G, 0)
    return bool((G >= 2).any())

def spectral_gap(H):
    """sigma_1 - sigma_2 of H"""
    s = np.linalg.svd(H.astype(float), compute_uv=False)
    return float(s[0] - s[1])

And then run a flow chart algorithm like this:

Unfortunately, this doesn’t work so well in practice, because the proportion of (3, 4)-regular Tanner graphs that have girth ≥ 6 is quite small, so one would have to wait a long time to generate enough graphs. In particular, this is related to the Poisson distribution, which is beyond the scope of this article. But the key result is that

One could, of course, just brute force search for a long time. But I used a different approach instead — constructing graphs with girth ≥ 6 from the get-go, using the following progressive edge growth function:

def regular_seed_girth6(n_bits, rng, wc=3, wr=4, attempts=200):
    """A random (wc,wr)-regular seed whose Tanner graph has girth >= 6.

    girth >= 6 iff no two bits share more than one check.  We enforce that
    edge by edge while building: walk the bits in random order and, 
    for each of its wc edges, connect it to
    the least-loaded check that would not close a 4-cycle."""

    r = n_bits * wc // wr
    for _ in range(attempts):
        H       = np.zeros((r, n_bits), dtype=np.uint8)
        rowd    = [0] * r                          # bits placed in each check
        co      = [set() for _ in range(n_bits)]   # bits sharing a check with b
        members = [[] for _ in range(r)]
        order   = list(range(n_bits))
        rng.shuffle(order)
        ok = True

        for b in order:
            for _edge in range(wc):
                cand = [c for c in range(r)
                        if rowd[c] < wr                     # check not yet full
                        and H[c, b] == 0                    # no repeated edge
                        and not (co[b] & set(members[c]))]  # no 4-cycle
                if not cand:
                    ok = False
                    break
                lo = min(rowd[c] for c in cand)             # least-loaded check
                cand = [c for c in cand if rowd[c] == lo]   # random tie-break
                c = cand[rng.randrange(len(cand))]

                H[c, b] = 1
                for x in members[c]:
                    co[x].add(b)
                    co[b].add(x)
                members[c].append(b)
                rowd[c] += 1
            if not ok:
                break

        if ok and all(d == wr for d in rowd):
            return H
    raise RuntimeError(f"no girth-6 seed at n_bits={n_bits} in {attempts} attempts")

By construction, this function always gives us (3, 4)-regular Tanner graphs with girth ≥ 6. Excellent!

Now, I run a search function that uses regular_seed_girth6 to generate many samples of such graphs, and then optimize for distance and spectral gap to give the final seed that we desired.

def search(n_bits, draws=1000, seed=0):
    """Returns (best_H, record).
    record holds (d, gap, k1, rank) for every accepted draw"""

    rng = random.Random(seed)
    record, best_H, best_key = [], None, None

    for _ in range(draws):
        H = regular_seed_girth6(n_bits, rng)   # girth >= 6 by construction
        assert not has_four_cycle(H)           # cheap independent confirmation

        k1  = nullspace(H).shape[0]
        d   = exact_distance(H)
        gap = spectral_gap(H)
        record.append({"d": d, "gap": gap, "k1": k1, "rank": n_bits - k1})

        key = (d, gap)                         # distance first, gap as tie-break
        if best_key is None or key > best_key:
            best_key, best_H = key, H

    return best_H, record

Finally, we have good classical seeds that we can take the HGP of. Here, I take the best classical matrix returned by the search function and take the HGP with itself.

Now, the only question remains, what size n of bits should we take? To answer this, I look at the codes that the paper uses. Figure 3 a of the paper gives the codes:

Reading from the figure, we see that the codes used by the paper are [[225, 9, 4]], [[625, 25, 6]], [[1225, 49, 8]], [[2500, 100, 12]], [[5625, 225, 16]], and [[10000, 400, 18]].

Now, we must reverse-engineer the classical seed that would give us an HGP code with parameters [[225, 9, 4]], and we will later repeat this for the other codes too.

Let H_seed be a classical (3, 4)-regular parity check matrix that we will use as a ‘seed’ in the hypergraph product. Suppose H_seed acts on n_seed bits, and assume H has full rank. We would like to know, how many checks r_seed does H_seed have? How many physical qubits n_qubits does the HGP have? How many logical qubits k_qubits does the HGP have? What is the rate k_qubits / n_qubits of the HGP code?

We can count the checks r_seed two different ways using the (3, 4)-regular property: 3n or 4r. Setting these equal, we get

The number of message bits of the classical code H_seed is then

Recalling that the hypergraph product places qubits on check × check and bit × bit positions, we get that the physical qubit count of the HGP is

and the logical qubit count of the HGP is

So rate of the HGP code is

Observe that the rate is independent of n — this is exactly the “constant overhead” in the paper’s title.

But the paper says that the rate of the HGP code is lower-bounded by 0.04, why is that?

If H_seed has full rank, then the number of checks (i.e., number of rows) r_seed = rank(H_seed), and the rate of the HGP code is exactly 0.04. But in the general case,

Now, back to the task of reverse-engineering, we want n_qubits = 225 and k_qubits = 9, so

Repeating similarly for the other codes, we find that

Once I got these n_seed values, I ran my search function to reproduce the codes used by the paper. Here are my results:

I am not sure why I got a higher distance than the paper in the cases of n_seed being 12, 20, and 28. In all likelihood, there is some error in my code that I am trying to find out. In the case n_seed = 60, I got a lower distance than the paper, presumably because I searched only 1000 samples.

Next, I also cross-checked Table 1 of the paper, which tells us their calculation of the overhead savings in using HGP or LP codes over the surface code.

We will focus on the row about HGP codes. From the above results, we already know that 25 logical qubits ←→ n_seed = 20 and 400 logical qubits ←→ n_seed = 80. So let us try to understand where the corresponding physical qubit counts for those HGP codes (1,235 and 19,600 respectively) come from.

We already know that a (3, 4)-regular seed on n_seed bits gives an HGP code with 25/16 n²_seed data qubits. Syndrome extraction needs one ancilla atom per stabilizer generator, so how many ancilla qubits do we have?

Well, recalling that HGP places stabilizers on check × bit and bit × check positions, the number of stabilizer generators (i.e., the number of ancilla qubits) is

So the total number of physical qubits is data qubits + ancilla qubits, which gives us

And indeed, plugging in n_seed = 20 gives total qubits = 1,225 (a disagreement with the 1,235 figure in the table) and plugging in n_seed = 80 gives total qubits = 19,600.

To conclude this article, here are some questions and observations I had while working on this project -

1. It is not clear to me from reading the paper whether they took HGP of a good seed code with itself, or HGP of two different good seed codes.

2. It is not clear to me from reading the paper whether they first generated entire sets of (3, 4)-regular Tanner graphs and then rejected those with girth ≤ 4, or they too used a progressive edge growth algorithm to deliberately construct only those (3, 4)-regular Tanner graphs that already have girth ≥ 6.

3. In Table 1, I had a disagreement with the paper for total qubits for an HGP code at 25 logical qubits (paper says 1,235 v/s my result of 1,225). Also, 80 and 180 are not k² for any integer k, so perhaps they interpolated the fit to a continuous code size? Or maybe they used HGP of two different seed codes (perhaps with something like k₁ = 8 and k₂ = 10)

4. While distance and spectral gap had a maximisation constraint, girth only had a threshold. We were not trying to maximise girth, instead we were allowing everything above threshold girth ( ≥ 6). To understand this better, I will look into how exactly decoders rely on the no-4-cycle property.

5. In the flow chart for seed generation, we did not optimise for low-rank (say, non-full rank) even though low rank improves the code rate. What if we include a rank-minimisation criteria in the seed generation algorithm?

6. Why exactly do we maximise spectral gap?

7. It would be interesting to see what fraction of criteria-satisfying seeds of a given size come out full rank.

Subscribe now

Let’s start by understanding linear codes — a key notion in classical error correction — and use it to set the stage for quantum error correction.

The usual presentation of linear codes begins with the 3-bit repetition code, then considers a general [n, k, d] code and its rate k/n, and so on, all from the engineer’s perspective of building practical codes. But as any math major at UChicago would ask, “that’s all well and good in practice... but how does it work in theory?”

In this post, I want to present linear codes from the abstract, pure math perspective of linear algebra.

Consider the space of n-bit strings, denoted {0, 1}ⁿ. Notice that {0, 1} = F₂, so the space of n-bit strings is exactly {0, 1}ⁿ = F₂ⁿ.

Just as ℝⁿ is a vector space over the field ℝ, we might guess that F₂ⁿ is a vector space over F₂. Before we can evaluate this guess, we must define an addition operator on the space of n-bit strings.2

So, how can we add n-bit strings? Here are a few ideas one might think of:

1. Convert the n-bit strings into decimal numbers, add them in decimal, and convert back to binary.

   For example, if I want to add the 2-bit strings 10 and 11, I look at their decimal representations, 2 and 3 respectively, add them in decimal to get 2 + 3 = 5, then convert 5 back into binary to get 101.

   But clearly, this doesn’t work, because we added 2-bit strings (10 and 11) and got a 3-bit string (101), so the space is not closed under addition!

2. To resolve this, one can do the same thing — convert to decimal, add in decimal and convert back — but this time do the decimal addition modulo n.

   For example, if I want to add the 3-bit strings 101 and 110, I look at their decimal representations, 5 and 6 respectively, add them in decimal to get 5 + 6 = 11, take modulo 3 to get 2, then convert 2 back into binary to get 010. So under this definition of the addition operator, we get 101 + 110 = 010.

   The space is now closed under addition, but intuitively, this feels strange. To see why, think about what possible answers we can get when we add modulo n: we can only get the remainders 0, 1, …, n-1. So, adding any 3-bit strings can never give an output greater than 010 (i.e., 2). Isn’t this strange?

   More rigorously, the addition map is not surjective onto {0, 1}ⁿ, which means that the space under this addition operator has no identity element. (Exercise: think about how failure of surjectivity implies lack of identity element. Suppose there was an identity element e, and try adding it to the 3-bit string 111 — what happens?)

3. To fix this, we repeat the same procedure, but we add modulo 2ⁿ instead of n. (Exercise: think about why this fixes the problem of the identity element!)

   Alas, we are not yet done. Think about what we originally set out to do: we wanted to construct a vector space over the field F₂. Observe that on F₂, 1 + 1 = 0, so any vector v added to itself must give:

   But with our modulo 2ⁿ definition, 011 + 011 = 110 ≠ 0.

   Let us think back to the analogy with ℝⁿ that we began with. How does ℝⁿ avoid this problem? Well, addition on ℝⁿ is coordinate-wise, i.e., (u + v)_j depends only on u_j and v_j.

   If we generalised such a notion of coordinate-wise addition to our space, we naturally get addition of n-bit strings defined as a bitwise modulo 2 addition (i.e., a bitwise XOR).

Finally, we have an addition operator under which {0, 1}ⁿ is a genuine vector space over F₂. (Exercise: convince yourself that with the addition operator defined as bitwise XOR, this satisfies all the vector space axioms).

Whew! All that work was just to define an addition operator?! Let’s take a breather.

In pure mathematics, one often encounters the idea of a space — metric space, vector space, topological space, etc. — and one wonders what this means. After all, this is not the same space as one thinks of in the astronomical sense of the word.

An abstract space in mathematics is simply a set endowed with some structure. For example, a metric space is a set equipped with a notion of distance between points. A topological space is a set equipped with a collection of open subsets. A vector space is a set associated with a field, equipped with addition and scalar multiplication operators.

Given two spaces X and Y of the same kind, it is a natural question to ponder “which functions f: X → Y preserve the structure of the space?” For example, if X and Y are both metric spaces, then it is natural to study the functions that preserve distance. In the general case, given a space, we would like to consider the functions that preserve the essential structure of the space. The general name for such a function is homomorphism.

(Exercise: Considering the structure of vector spaces — namely, the operations they are equipped with — think about and write down the conditions required for a function to be a vector space homomorphism. Concretely, what properties would a function between two vector spaces f: V → W need to satisfy, in order to preserve the essential structure of a vector space?)

Spoiler: these requirements are exactly the definition of a linear transformation between vector spaces. In other words, a vector space homomorphism is precisely a linear transformation. This explains why we care so much about linear transformations: they preserve the essential structure of vector spaces.

Back to our study of {0, 1}ⁿ.

Note that this space has an additional structure: we can think of the “weight” of an n-bit string vector as the number of 1’s in it. For example, the weight of 10010 is 2, and the weight of 10111 is 4. We call this the Hamming weight of the vector.

And observe that our choice of the bitwise XOR as addition operator naturally gives a metric: given two n-bit string vectors, simply add them to get a new vector, and look at the Hamming weight of that vector to get the distance between the original two vectors:

So we actually have not just a vector space, but also a metric space! We call this the n-dimensional Hamming space. (Exercise: check that this is a valid metric: it is symmetric, non-negative, d(x, y) = 0 iff x = y, and satisfies the triangle inequality).

Going forward, we can think about Hamming space both as a vector space and as a metric space.

Now, what does one do when handed vector spaces? One thinks about maps between them, of course! So let us consider k-dimensional Hamming space H_k and n-dimensional Hamming space H_n and a map T: H_k → H_n between them. (For now, this can be any arbitrary map — we will impose constraints on it soon).

Our linear algebra intuition already tells us that imposing the constraint that T be a linear transformation might somehow make it interesting, though we don’t know exactly how just yet. Hold onto this intuition; it will come up soon!

Now, finally, let us turn to the application of all this theory.3

I posit the following for your consideration: if we wish to reliably send a k-bit message to a friend, across a noisy channel that may introduce errors in our message, then it is necessary to transmit more than k bits. (Exercise: think about why we need strictly more bits — what would happen if we sent exactly k bits and an error occurred?)

Let n and k be integers representing the number of transmitted bits and the number of bits in the message, respectively. Clearly, then, n > k.

Let us revisit the map T: H_k → H_n. We may think of this map as encoding our k-bit message as an n-bit string.

For example, if k = 3 and n = 5 and if our message is 101, then T might, for instance, map this to T(101) = 10111. Or some other choice of the map might give us, say, T(101) = 01101. For now, T is arbitrary and all that matters is that it is converting a k-bit message into an encoded n-bit codeword.

Let us think about what extra properties we desire T should have. For one thing, it would be strange if two different messages got encoded into the same codeword — e.g., if T(101) = T(111) — for how would our friend decode the codeword to extract our intended message? So we require T to be injective.

What is the cardinality of the image of T? Well, the domain has 2^k elements and T is injective, so |Image(T)| = 2^k. So the image of T is a 2^k-element subset of the 2ⁿ-element set {0, 1}ⁿ. By the way, we refer to Image(T) by C, the set of codewords.

Now, just for fun, how about we force T to be linear and see what happens?4

If T: H_k → H_n is a linear transformation, then the image of T is a linear subspace of the codomain H_n. When this is the case, we call the code a linear code.

What does this mean? It means that adding any two codewords always produces another codeword. Peculiar! How could this possibly be of use to us? Stow this question and we’ll return to it.

In what scenario is an error correctable? Suppose, with k = 3 and n = 5, that my message is 101 and the corresponding codeword — the thing that I transmit through the channel — is T(101) = 11101. If no errors occur, then we are done, because T is injective so the recipient can simply invert to get T ^-1(11101) = 101.

If errors do occur, however, then the recipient will receive something different from 11101; for example, they may receive 11000. Therefore, note that the recipient flags an error when the received string is not a codeword.

In this case, we need to supply a decoding mechanism: a way to interpret the received, faulty n-bit string to extract the original k-bit message. Even without specifying the particular decoding mechanism, it is easy to imagine a case in which it fails: if so many errors occur that the received, faulty n-bit string is a codeword, then the situation is hopeless. For example, if 11000 is itself a codeword, then how can the recipient possibly know that the true message is not T ^-1(11000)?

Observe that there are two ways in which the recipient can receive a codeword: either no errors occurred, or so many errors occurred that another codeword was received.

So the operative question is: how many errors must occur to turn one codeword into another? But the codewords need not be evenly spaced apart, so what matters is, in the set of all 2^k codewords, what is the minimum distance over all pairs of distinct codewords? We call this the distance d of the code, because it quantifies the code’s robustness: the larger the distance d, the more errors that can be detected.5

How does one compute the distance of a code? A priori, one would need to compute (2^k choose 2)-many pairwise Hamming distances and find the minimum. But recall that for a linear T, the set of codewords is closed under addition. So we may simply find the minimum Hamming weight over non-zero codewords and we’re done!

For a linear code, the code distance is equal to the minimum, over all non-zero codewords, of the Hamming weights of the codewords.

This is the first indication of our vindication — we already see that forcing T to be a linear transformation, in line our with linear-algebraic intuition, made it much easier for us to compute d.6

We said earlier that the larger the code distance d, the more errors that can be detected. It is natural to ask, then, how exactly does the code distance d relate to the number of errors that can be corrected, call it t — what equation relates them?

Firstly, we must distinguish the number of detectable errors from the number of correctable errors — a priori, they may or may not be the same.

We defined the code distance d as the minimum number of bits that need to be flipped to go from one codeword to another. Clearly, this means that if d errors occur, then our code cannot detect them. But if fewer than d errors occur, then no matter what the original codeword was, it cannot have been distorted into another codeword — so it now sits somewhere in Hamming space, between codewords.

Recall that the recipient flags an error whenever the received string is not a codeword. This tells us that if fewer than d errors occur, then we know for sure that our code can detect them — so upto d-1 are guaranteed detectable.

What about correctable errors? In order to correct an error, we must construct a decoder function D: H_n → C that reads the received n-bit string and outputs the codeword that was transmitted.

We say that D corrects t errors if D(c + e) = c for every n-bit-error-string e that has Hamming weight ≤ t. This is a condition that an arbitrary function may or may not satisfy; what we are interested in is whether there exists a particular decoder function D that satisfies it. (Exercise: prove that such a D exists if and only if d ≥ 2t + 1).

So, detection works with guarantee up to d-1 errors, correction only up to (d-1)/2 errors. To detect, the error need only be visible; to correct, it must be unambiguous, and ambiguity sets in at the midpoint between codewords.

Having used an abstract path to getting here, let us reflect on the intuition it provides us.

Our linear algebra intuition told us that we should pick T to be a vector space homomorphism, namely a linear transformation, and this worked out well, giving us the concept of linear codes.

Seeing as Hamming space is also a metric space, we may very well have the idea of making T a metric space homomorphism too, namely a distance-preserving map, also called an isometry.

But in fact, an isometric T is the worst possible encoder, for it means that the code has d = 1 (exercise: why?), meaning that no errors can be detected, let alone corrected! The very nature of a useful code is that any two distinct k-bit messages — no matter how far apart in H_k — get mapped to codewords that are Hamming distance ≥ d apart in H_n — namely, the fact that T is not an isometry!

Clearly, we have an asymmetry that begs to be resolved: why are vector space homomorphisms ‘good’ but metric space homomorphisms ‘bad’? The answer lies in revisiting the notion of homomorphism as a map that preserves essential structure. But the question is, what is essential?

In this case, the message — the k-bit string vector — is essential. The notion of Hamming distance as a metric is used only as an evaluation criteria to gauge how good the code is — how many errors it can correct — and is not essential.

Put differently, a structure can come up as a constraint defining which maps are admissible, or as an objective that measures which admissible maps are good. Here, the vector space structure plays the first role, and the metric space structure plays the second.

To recapitulate,

• Bitwise XOR is the natural addition operator that makes {0, 1}ⁿ a vector space over F₂

• A linear code is one where the encoding operator T: H_k → H_n is a linear transformation

• T: H_k → H_n is a vector space homomorphism but not a metric space homomorphism (even though Hamming space is both, a vector space and a metric space).

We are only just scratching the surface of error correction theory (pun intended), and future posts will expand on further topics.

Feedback and reading recommendations are invited at malhar.manek@gmail.com

Some resources I found useful and would recommend:

1. This video on Hamming codes by 3blue1brown

2. This blog article on classical error correction by Arthur Pesah