Request Wordlist cleaner

JD · **Joined:** 08 Jun 2008 07:17 **Posts:** 92

If you are willing can someone build a program as I really am no good in C,C++ much

Here's some Detail it's simple.

App name: Wordlist Cleaner
what it does: Cleans wordlist form a specified number of chars, remove duplicate words

Here's an example

Wordlist_cleaner.exe
Usage Wordlist_Cleaner.exe {options} {text file} {output file after cleaned}

-R # Removes all characters under chosen number {#}
-D Removes Duplicate words
-C Counts how many words their is (Does not write to output text file)
-S Show cleaned stats (Does not write to output text file)

And here's a example of it working

Wordlist_Cleaner.exe -R 8 -D -S -C Input.txt cleaned.txt

This would remove all words under 8 characters, and duplicate and the resulting output file would be cleaned.txt and in the terminal is shows 50000 words are in cleaned.txt and 3245 where removed due to under 8 characters long/ 400 removed due to duplicate.

This is just a suggestion after all, but I would appreciate it it Luigi, someone can build me a program like this :], I've been trying in VB6 but it limits textbox to about 65k characters and 200k characters :[

aluigi · **Posted:** 11 Jul 2009 20:50

for the duplicate stuff there is the good old unix sort (http://gnuwin32.sourceforge.net/downlin ... in-zip.php):
sort -c list.txt > newlist.txt

while for the length I guess there is some other unix tool for doing it, maybe to insert in pipe with the sort's output

JD · **Joined:** 08 Jun 2008 07:17 **Posts:** 92

Well I've taken the task up for myself.

But I need some help

here's my current code

Code:

#include <iostream>
#include <string>
#include <fstream>
using namespace std;
int main (void)
{    
int lesschar1;
int maxchar1;
string wordlist1;
    
      cout << "Wordlist Cleaner" << endl;
      cout  << endl;
      cout << "Remove words with LESS then how many characters?: ";
      cin >> lesschar1;
      cout << endl;
      cout << "Remove words with MORE then how many characters?: ";
      cin >> maxchar1;
      cout << endl; 
      cout << "Drop a Wordlist here to begin cleaning: ";
      cin >> wordlist1; 
      cout << endl; 
      
  string line;   
  ifstream words; 

  words.open (wordlist1.c_str());
  if (words.is_open());
  {
    while (! words.eof() )
    {
     getline (words,line);
     cout << line << endl;
    }
    words.close();
  }
  
      
      system("PAUSE");
      return 0;
}

All it does so far is display a whole text file for me, but I'm trying to use strlen to get the length of "line" but it's not working.

Code:

    {
     getline (words,line);
     cout << line << endl;
    }

Right now the line prints out what ever is in my text file but I can't make it get a length of a word from it, I need some help :]

aluigi · **Posted:** 13 Jul 2009 23:34

I don't know c++ (and I hate it too), sorry

anyway you can ever implement one by hand:

Code:

int mystrlen(char *line) {
    int     i;
    for(i = 0; line[i]; i++);
    return(i);
}

Leben · **Joined:** 29 May 2009 05:42 **Posts:** 4

what's up..
i made a batch file that essentially counts the lines of a text file.
so if you have a word list that is like mine, with one word per line.. it should count every word (the default txt file it counts is wordlist1.txt)

http://www.filedropper.com/linecounter

lataz

Sethioz · **Posted:** 15 Jul 2009 15:54

use Raptor3, it can do a lot more, remove dubs, combine wordlists, split wordlists, sort by lenght, add specific string at end of each word..etc.

Luigi Auriemma

Request Wordlist cleaner