First, love your research, helped me get over a part with the udp. (don't know much about sendto/recvfrom)
Well I noticed in
http://aluigi.altervista.org/papers/ventrcon.zip
in ventrilo3_handshake.c
You have...
Code:
tmp[0] = key >> 24;
tmp[1] = key >> 16;
tmp[2] = key >> 8;
tmp[3] = key;
for(i = 16; i < len; i++) {
data[i] += tmp[i & 3];
}
Well when I was looking through it I saw...
Code:
00451AB0 /$ 33C0 XOR EAX,EAX ; Encrypt sendto
00451AB2 |. 8A4424 04 MOV AL,BYTE PTR SS:[ESP+4]
00451AB6 |. 56 PUSH ESI
00451AB7 |. 83E0 0F AND EAX,0F
00451ABA |. 0FAF4424 0C IMUL EAX,DWORD PTR SS:[ESP+C]
00451ABF |. 8BF0 MOV ESI,EAX
00451AC1 |. B8 10000000 MOV EAX,10
00451AC6 |. 66:3941 0A CMP WORD PTR DS:[ECX+A],AX
00451ACA |. 76 27 JBE SHORT Ventrilo.00451AF3
00451ACC |. 53 PUSH EBX
00451ACD |. 8D49 00 LEA ECX,DWORD PTR DS:[ECX]
00451AD0 |> 3D 00020000 /CMP EAX,200
00451AD5 |. 7D 1B |JGE SHORT Ventrilo.00451AF2
00451AD7 |. 8A1C08 |MOV BL,BYTE PTR DS:[EAX+ECX]
00451ADA |. 8BD6 |MOV EDX,ESI
00451ADC |. 83E2 03 |AND EDX,3
00451ADF |. 8A5414 0C |MOV DL,BYTE PTR SS:[ESP+EDX+C]
00451AE3 |. 02DA |ADD BL,DL
00451AE5 |. 881C08 |MOV BYTE PTR DS:[EAX+ECX],BL
00451AE8 |. 0FB751 0A |MOVZX EDX,WORD PTR DS:[ECX+A]
00451AEC |. 46 |INC ESI
00451AED |. 40 |INC EAX
00451AEE |. 3BC2 |CMP EAX,EDX
00451AF0 |.^ 7C DE \JL SHORT Ventrilo.00451AD0
00451AF2 |> 5B POP EBX ; 001850F0
00451AF3 |> 5E POP ESI ; 001850F0
00451AF4 \. C2 0800 RETN 8
Which if you look at the start. There is a small number passed which takes the first byte of tmp, ands with F and multiplies with the passed number and that is where tmp starts. Heres some code to understand what I am saying.
idx being the idx of the ventrilo3_auth (might be a bit different).
Code:
tmp[0] = key >> 24;
tmp[1] = key >> 16;
tmp[2] = key >> 8;
tmp[3] = key;
idx = (tmp[0] & 0xF) * idx;
for(i = 16; i < len; i++) {
data[i] += tmp[(idx + i) & 3];
}
Anyways... Don't think it makes a difference, just wanted to point it out.
