What encryption is used on this algo?

packetkim · **Joined:** 20 Jul 2009 02:25 **Posts:** 3

Hello, I'm a begginner on encryption algorithms and I have reversed only simple algorithms alone.
But this one is too big (look the attachment) and I would die before I could reach it's end.
It encrypts a given string based on 256 dword keys passed to this function.

/*
void EncryptString(u_long*keys, int src_len, char*src, char*dest);
keys : points to an array of 256 dwords
src_len : size of src
src : points to a null terminated string to be encrypted
dest : where to save the encrypted string
*/

I used the Luigi's signsearch tool and it showed me many algorithms used by this program:

Code:

004ae5f0 165  AES Rijndael S / ARIA S1 [..256]
004ae6f0 166  AES Rijndael Si / ARIA X1 [..256]
004ae7f0 167  Rijndael Te0 (0xc66363a5U) [32.le.1024]
004aebf0 169  Rijndael Te1 (0xa5c66363U) [32.le.1024]
004aeff0 171  Rijndael Te2 (0x63a5c663U) [32.le.1024]
004af3f0 173  Rijndael Te3 (0x6363a5c6U) [32.le.1024]
004af7f0 176  Rijndael Td0 (0x51f4a750U) [32.le.1024]
004afbf0 178  Rijndael Td1 (0x5051f4a7U) [32.le.1024]
004afff0 180  Rijndael Td2 (0xa75051f4U) [32.le.1024]
004b03f0 182  Rijndael Td3 (0xf4a75051U) [32.le.1024]
004acd78 190  Blowfish bfp table [32.le.72]
004acdc0 192  Blowfish ks0 table [32.le.1024]
004ad1c0 194  Blowfish ks1 table [32.le.1024]
004ad5c0 196  Blowfish ks2 table [32.le.1024]
004ad9c0 198  Blowfish ks3 table [32.le.1024]
004acd98 325  Haval hash pass2 [32.le.128&]
009835ca 550  Windows CryptAcquireContext [..21]
00983630 551  Windows CryptCreateHash [..16]
0098361e 552  Windows CryptImportKey [..15]
00983642 553  Windows CryptHashData [..14]
0098366a 555  Windows CryptDeriveKey [..15]
0098367c 556  Windows CryptDecrypt [..13]
00470c92 568  classical random incrementer 0x343FD 0x269EC3 [32.le.8&]
004acd78 1299 Haval init [32.le.32&]
004ace18 1301 Haval mc3 [32.le.128]
004ace98 1303 Haval mc4 [32.le.128]
004acf18 1305 Haval mc5 [32.le.128]
004b44fc 1386 seed_SS0 [32.le.1024]
004b48fc 1388 seed_SS1 [32.le.1024]
004b4cfc 1390 seed_SS2 [32.le.1024]
004b50fc 1392 seed_SS3 [32.le.1024]
004acef8 1451 HAVAL1_DS [32.le.32]
004ace78 1453 HAVAL2_DS [32.le.32]
004acdc0 1561 Blowfish_s_init [32.le.4096]
004b17f7 1636 Noekeon Nessie round [..17]

How can I know what encryption algorithm is being used by the attached disassembled code by knowing the above information?

Paul R.

aluigi · **Posted:** 20 Jul 2009 11:55

at a first quick look it doesn't seem an algorithm on which I have already worked (aes, blowfish and so on) but I could be wrong because in some cases the different implementations and binary-optimizations make them hard to recognize them on the fly except if there are the s-boxes and constants identified by signsrch and visible in the pasted code.

for example the *des algorithms are easily identified by the 33333333, 55555555 and other similar constants and the length of the code (looong) while blowfish is visible by the usage of values like [regs+400], [regs+800], [regs+c00] and so on.

in this case the algorithm doesn't look complex (only a bit long probably because the original code uses many #define as common in these algorithms) and no constants are used and I guess that the buffer you identified as key is the result of an init function with the real key, so find it first (it could contain constants/s-boxes to identify).

anyway I highly suggest you to collect all the files available here:

http://www.theargon.com/achilles/cryptography/ or http://www.mirrorservice.org/sites/ftp. ... lgorithms/ (the original site http://www.mirrors.wiretapped.net/secur ... lgorithms/ seems dead)
http://ftp.icm.edu.pl/packages/replay/applied-crypto/ (or other mirrors)
and obviously openssl

packetkim · **Joined:** 20 Jul 2009 02:25 **Posts:** 3

I know how the 256 dword keys are created. First, the program creates a 128-bit key randomly (each byte is created by a calc with the C rand() function). Second, the program creates the 256 Dword keys based only on the 128-bit key. And third, the 256 dword keys are used by the EncryptString() function. I'm going to check those files you have suggested me.
Thanks Luigi for the tip.

aluigi · **Posted:** 20 Jul 2009 23:39

uhmmm the only time I saw rand() used in an encryption algorithm was for a recent password recovery I wrote (don't worry, that one was simple so not like yours).
anyway it's an excellent starting point, at least now you know exactly what to search:

Code:

grep -rs "rand()" crypto openssl*

remember only to remove from the list the rand() used in the test cases of the algorithms which are not part of them

Luigi Auriemma

What encryption is used on this algo?