reed-solomon | reliable Reed-Solomon erasure

miscorrection for error-only decoding comes from computing the wrong error locations

Yes. For example, consider an RS code with 6 parities, which can correct 3 errors. Assume that 4 errors have occurred, and that a 3 error correction attempt created an additional 3 errors, for a total of 7 errors. It will produce a valid codeword, but the wrong codeword.

There are situations where the probability of miscorrection can be lowered. If the message is a shortened message, say 64 bytes of data and 6 parities for a total of 70 bytes, then if a 3 error case produces an invalid location, miscorrection can be avoided. In this case, the odds of 3 random locations being valid is (70/255)^3 ~= .02 (2%).

Another way to avoid miscorrection is to not use all of the parities for correction. With 6 parities, the correction could be limited to 2 errors, leaving 2 parities for detection purposes. Or use 7 parities, for 3 error correction, with 1 parity used for detection.

Follow up based on comments:

First note that there are 3 decoders that can be used for BCH view Reed Solomon: PGZ (Peterson Gorenstein Zierler) matrix decoder, BKM (Berlekamp Massey) discrepancy decoder , and Sugiyama's extended Euclid decoder. PGZ has greater time complexity O((n-k)^3) than BKM or Euclid, so most implementations use BKM or Euclid. You can read a bit more about these decoders here:

https://en.wikipedia.org/wiki/Reed%E2%80%93Solomon_error_correction

Getting back to 6 parities, 4 errors. All valid RS(n+6, n) codewords differ from each other by at least 7 elements. If 4 of the elements of a message are in error, there may be a valid codeword that differs by only 3 more elements from that message with 4 error elements, and in this case, all 3 decoders will find that the message differs from a valid codeword by 3 elements, and "correct" those 3 elements to produce a valid codeword, but in this case the wrong valid codeword, which will differ by 7 elements from the original codeword. With 5 elements in error, a 2 or 3 error miscorrection could occur, and with 6 or more elements in error, a 1 or 2 or 3 error miscorrection could occur.

Invalid location - consider a RS code based on GF(2^8), which allows for a message size up to 255 bytes. Valid locations for a 255 byte message are 0 to 254. If the message size is less than 255 bytes, for example 64 data + 6 parity = 70 bytes, then locations 0 to 69 are valid, while locations 70 to 254 are invalid. In what would otherwise be a case of miscorrection, if a calculated location is out of range, then the decoder has detected an uncorrectable message, rather than miscorrect it. Assume a garbled message and that the decoder generates 3 random locations in the range 0 to 254, the probability of all 3 being in the range 0 to 69 is (70/255)^3.

Another case where miscorrection is avoided is when the number of distinct roots of the error locator polynomial does not match the degree of the polynomial. Consider a 3 error case with generated error locator polynomial x^3 + a x^2 + b x + c. If there are more than 3 errors in the message, then the generated polynomial may have less than 3 distinct roots, such as a double root, or zero roots, or ... , in which case miscorrection is avoided and the message is detected as being uncorrectable.

The encoding matrix is a 6 x 4 Vandermonde matrix using the evaluation points {0 1 2 3 4 5} modified so that the upper 4 x 4 portion of the matrix is the identity matrix. To create the matrix, a 6 x 4 Vandermonde matrix is generated (where matrix[r][c] = pow(r,c) ), then multiplied with the inverse of the upper 4 x 4 portion of the matrix to produce the encoding matrix. This is the equivalent of "systematic encoding" with Reed Solomon's "original view" as mentioned in the Wikipedia article you linked to above, which is different than Reed Solomon's "BCH view", which links 1. and 2. refer to. The Wikipedia's example systematic encoding matrix is a transposed version of the encoding matrix used in the question.

https://en.wikipedia.org/wiki/Vandermonde_matrix

https://en.wikipedia.org/wiki/Reed%E2%80%93Solomon_error_correction#Systematic_encoding_procedure:_The_message_as_an_initial_sequence_of_values

The code to generate the encoding matrix is near the bottom of this github source file:

https://github.com/Backblaze/JavaReedSolomon/blob/master/src/main/java/com/backblaze/erasure/ReedSolomon.java

An important aspect is that Reed-Solomon (RS) codes are cyclic: the set of codewords is stable by cyclic shift.

A consequence is that no particular part of a code word is more protected or less protected.

A RS code has a error correction capability equal to t = (n-k)/2, where n is the code length (generally expressed in bytes) and k is the information part length.

If the total number of errors (in both parts) is less than t, the RS decoder will be able to correct the errors (more precisely, the t erroneous bytes in the general case). If it is higher, the errors cannot be corrected (but could be detected, another story).

The emplacement of the errors, either in the information part or the added part, has no influence on the error correction capability.

EDIT: the rule t = (n-k)/2 that I mentioned is valid for Reed-Solomon codes. This rule is not generally correct for BCH codes: t <= (n-k)/2. However, with respect to your question, this does not change the answer: these families of code have a given capacity correction, corresponding to the minimum distance between codewords, the decoders can then correct t errors, whatever the position of the errors in the codeword

You could use a shortened code word such as (5, 19, 6, 7) 31.5% correction ratio, 95 bits, 48 bars. One advantage of a shortened code word is reduced chance of mis-correction if it is allowed to correct the maximum of 6 errors. If any of the 6 error locations is outside of the range of valid locations, that is an indication of that there are more than 6 errors. The probability of mis-correction is about (19/31)^6 = 5.3%.

Posting the answer :

First, I made a mistake in the modulo-2 algebra used to expand the polynomial. Non-modulo expanded form is :

what is the ideal value of the generator_polynomial_index

There probably isn't an "ideal" value.

I had to look at the github code to determine that the generator field index is the log of the first consecutive root of the generator polynomial.

https://github.com/ArashPartow/schifra/blob/master/schifra_sequential_root_generator_polynomial_creator.hpp

Typically the index is 0 (first consecutive root == 1) or 1 (first consecutive root == Alpha (the field primitive)). Choosing index = 1 is used for a "narrow sense" code. It slightly simplifies the Forney Algorithm. Link to wiki article, where "c" represents the log of the first consecutive root (it lists the roots as a^c, a^(c+1), ...):

https://en.wikipedia.org/wiki/Forney_algorithm

Why use a narrow sense code:

https://math.stackexchange.com/questions/2174159/why-should-a-reed-solomon-code-be-a-narrow-sense-bch-code

For hardware, the number of unique coefficients can be reduced by using a self-reciprocal generator polynomial, where the first consecutive root is chosen so that the generator polynomial is of the form: 1 x^n + a x^(n-1) + b x^(n-2) + ... + b x^2 + a x + 1. For 32 roots in GF(2^16), the first consecutive root is alpha^((65536-32)/2) = alpha^32752, and the last consecutive root would be alpha^32783. Note this is only possible with a binary field GF(2^n), and not possible for non-binary fields such as GF(929) (929 is a prime number). The question shows a range for index that doesn't include 32752; if 32752 doesn't work with this library, it's due to some limitation in the library, and not with Reed Solomon error correction algorithms.

Other than these 3 cases, index = 0, 1, or self-reciprocal generator polynomial, I'm not aware of any reason to choose a different index. It's unlikely that the choice of index has any effect on trying to decode beyond the normal limits.

The maximum number of errors and erasures which can be rectified should obey the following inequality: 2*num_errors + num_erasures < fec_length

That should be

With two parity symbols, the syndromes will only be unique for single symbol errors, which is why they can be used to correct single symbol errors. In the case of two symbol errors, the syndromes will be non-zero (one of them may be zero, but not both), but not always unique, for all combinations of two error locations and two error values (which is why two symbol errors can't be corrected if there are only two parity symbols).

With two parity symbols, the Hamming distance is 3 symbols. Every valid (zero syndromes == exact multiple of the generator polynomial) codeword differs by at least 3 symbols from every other valid code word, so no 2 symbol error case will appear to be a valid (zero syndrome) codeword.

It is possible for a 3 or more error case combination of error locations and values to produce syndromes == 0. The simplest example of this is to take a valid codeword (zero error message), and xor the 3 symbol generator polynomial anywhere within the message, which will be another valid codeword (an exact multiple of the generator polynomial).

In addition, there is a maximum length codeword. For BCH type Reed Solomon code, which is what you are using, for GF(2^n), it's (2^n)-1 symbols. If a message contains 2^n or more symbols (including the parity symbols), then a two error case with the same error value at message[i] and message[i + 2^n - 1] will produce syndromes of zero. For the original view type Reed Solomon code, the maximum length codeword is 2^n (one more symbol than BCH type), but it's rarely used since decoding operates on the entire message, while BCH decoding operates on syndromes.

Update - I forgot to mention that with two parity symbols, attempting to perform a one error correction on a two error message may end up causing a third error, which will result in a valid codeword (syndromes will be zero), but one that differs from the original codeword in three locations.

The probability of this happening is reduced if the codeword is shortened, as any calculated location that is not within range of the shortened codeword would be a detected error. If there are n symbols (including the parity symbols), then the probability of of a single error mis-correction based on the calculated location being within range is about n/255.

In this case, with a codeword size of 4 bytes, I found 3060 out of a possible 390150 two error cases that would create a valid code word by doing a single error correction that ends up creating a third error.

The ReedSolomon package's github page clearly indicates that you cannot encode arrays that are larger than k (223 in your case). This means that you have to split your image first before encoding it. You can split it into chunks of 223 and then work on the encoded chunks:

You can do a web search for "QR Code ISO", to find a pdf version of the document. I found one here:

https://www.swisseduc.ch/informatik/theoretische_informatik/qr_codes/docs/qr_standard.pdf

There are multiple strengths of error correction in the standard, and to avoid mis-correction, in some cases, some of the "parity" bytes are only used for error detection, and not for error correction. This is shown in table 13 in the pdf file linked to above. The ones marked with a "b" are cases where some of the parity bytes are used only for error detection. For example, the very first entry in table 13 shows (26,19,2)b, which means 26 total bytes, 19 data bytes, and 2 byte correction, which means of the 26-19 = 7 parity bytes, 4 are used for correction (each corrected byte requires 2 parity bytes unless hardware can flag "erasures"), and 3 are used for detection only.

If the error correction calculates an invalid location (one that is "outside" the range of valid locations), that will be flagged as a detected error. If the number of unique calculated locations is less than than the number of assumed errors used to calculate those locations (duplicate or non-existing root) that will be flagged as a detected error. For higher levels of error correction, the odds of all the calculated locations being valid for bad data is so small that none of the parity bytes are used for error detection only. These cases don't have the "b" in their table 13 entries.

The choices made for the various levels of error correction result in a very small chance of a bad result, but it's always possible.

are there other levels of integrity checks (checksums etc.) that would prevent that?

A QR-Code reader could flags bytes where any of the bits were not clearly 0 or 1 (like a shade of grey on a black / white code) as potential "erasures", which would lower the odds of a bad result. I don't know if this is done.

When generating a QR-Code, a mask is chosen to even out the ratio of light and dark areas in a code, and after correction, if there is evidence that the wrong mask was chosen, that could be flagged as a detected error, but I'm not sure if the "best" mask is always chosen when a code is printed, so I don't know if a check for "best" mask is used.