Get Even More Visitors To Your Blog, Upgrade To A Business Listing >>

SOLVED: The probability distribution of two words in a file using java 8

m.kabiri:

I need the number of lines that contain two words. For this purpose I have written the following code: The input file contains 1000 lines and about 4,000 words, and it takes about 4 hours. Is there a library in Java that can do it faster? Can I implement this code using Appache Lucene or Stanford Core NLP to achieve less run time?


ArrayList reviews = new ArrayList();
ArrayList terms = new ArrayList();
Map pij = new HashMap();

BufferedReader br = null;
FileReader fr = null;
try
{
fr = new FileReader("src/reviews-preprocessing.txt");
br = new BufferedReader(fr);
String line;
while ((line= br.readLine()) != null)
{
for(String term : line.split(" "))
{
if(!terms.contains(term))
terms.add(term);
}
reviews.add(line);
}
}
catch (IOException e) { e.printStackTrace();}
finally
{
try
{
if (br != null)
br.close();
if (fr != null)
fr.close();
}
catch (IOException ex) { ex.printStackTrace();}
}
long Count = reviews.size();
for(String term_i : terms)
{
for(String term_j : terms)
{
if(!term_i.equals(term_j))
{
double p = (double) reviews.parallelStream().filter(s -> s.contains(term_i) && s.contains(term_j)).count();
String key = String.format("%s_%s", term_i,term_j);
pij.put(key, p/Count);
}
}
}



Posted in S.E.F
via StackOverflow & StackExchange Atomic Web Robots
This Question have been answered
HERE


This post first appeared on Stack Solved, please read the originial post: here

Share the post

SOLVED: The probability distribution of two words in a file using java 8

×

Subscribe to Stack Solved

Get updates delivered right to your inbox!

Thank you for your subscription

×