I wonder if the Lynx encryption shares anything with the 3DO encryption. Both systems were partly designed by RJ Mical.
I asked about the Lynx encryption on AtariAge, and got the following answer (Thanks to Karri):
It is standard RSA encryption. The formula is:
PLAINTEXT = (ENCRYPTED ^ 3) % PUBLIC_KEY
The public key is
static unsigned char N[MAX_m] = {
0x35, 0xB5, 0xA3, 0x94, 0x28, 0x06, 0xD8, 0xA2,
0x26, 0x95, 0xD7, 0x71, 0xB2, 0x3C, 0xFD, 0x56,
0x1C, 0x4A, 0x19, 0xB6, 0xA3, 0xB0, 0x26, 0x00,
0x36, 0x5A, 0x30, 0x6E, 0x3C, 0x4D, 0x63, 0x38,
0x1B, 0xD4, 0x1C, 0x13, 0x64, 0x89, 0x36, 0x4C,
0xF2, 0xBA, 0x2A, 0x58, 0xF4, 0xFE, 0xE1, 0xFD,
0xAC, 0x7E, 0x79
};
You decrypt the data in 51 byte chunks.
So basically you take 51 bytes and treat it as a very large number and raise it to the power of 3. Then divide it by the 51 byte very large public key and the modulo of this division is the plain text (51 bytes).
After that you take the next 51 bytes and repeat this procedure until the whole binary has been decrypted.
The private key for doing the encryption is not known in plain text. It was encrypted by another RSA key which means that it existed only for a brief time in the Amiga RAM during the encryption process.
I did find RJ's site (
http://www.mical.org/), I wonder if he would be willing to share any information on the encryption.