我从互联网上复制了一个简单的网络爬虫,然后开始在测试类中运行该应用程序。每次我尝试运行该应用程序时,我都会得到“线程中的异常”主“java.lang.NoClassDefFoundError: org/j的/J的”错误。我首先在Libary中导入了一个外部罐子,因为我需要它来处理超文本传输协议的事情。
错误消息:
Exception in thread "main" java.lang.NoClassDefFoundError: org/jsoup/Jsoup
at com.copiedcrawler.SpiderLeg.crawl(SpiderLeg.java:35)
at com.copiedcrawler.Spider.search(Spider.java:40)
at com.copiedcrawler.SpiderTest.main(SpiderTest.java:9)
Caused by: java.lang.ClassNotFoundException: org.jsoup.Jsoup
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:602)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
... 3 more
蜘蛛类
package com.copiedcrawler;
import java.util.HashSet;
import java.util.LinkedList;
import java.util.List;
import java.util.Set;
public class Spider
{
private static final int MAX_PAGES_TO_SEARCH = 10;
private Set<String> pagesVisited = new HashSet<String>();
private List<String> pagesToVisit = new LinkedList<String>();
public void search(String url, String searchWord)
{
while(this.pagesVisited.size() < MAX_PAGES_TO_SEARCH)
{
String currentUrl;
SpiderLeg leg = new SpiderLeg();
if(this.pagesToVisit.isEmpty())
{
currentUrl = url;
this.pagesVisited.add(url);
}
else
{
currentUrl = this.nextUrl();
}
leg.crawl(currentUrl); // Lots of stuff happening here. Look at the crawl method in
// SpiderLeg
boolean success = leg.searchForWord(searchWord);
if(success)
{
System.out.println(String.format("**Success** Word %s found at %s", searchWord, currentUrl));
break;
}
this.pagesToVisit.addAll(leg.getLinks());
}
System.out.println("\n**Done** Visited " + this.pagesVisited.size() + " web page(s)");
}
/**
* Returns the next URL to visit (in the order that they were found). We also do a check to make
* sure this method doesn't return a URL that has already been visited.
*
* @return
*/
private String nextUrl()
{
String nextUrl;
do
{
nextUrl = this.pagesToVisit.remove(0);
} while(this.pagesVisited.contains(nextUrl));
this.pagesVisited.add(nextUrl);
return nextUrl;
}
}
蜘蛛腿类
package com.copiedcrawler;
import java.io.IOException;
import java.util.LinkedList;
import java.util.List;
import org.jsoup.Connection;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
public class SpiderLeg
{
// We'll use a fake USER_AGENT so the web server thinks the robot is a normal web browser.
private static final String USER_AGENT =
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.112 Safari/535.1";
private List<String> links = new LinkedList<String>();
private Document htmlDocument;
/**
* This performs all the work. It makes an HTTP request, checks the response, and then gathers
* up all the links on the page. Perform a searchForWord after the successful crawl
*
* @param url
* - The URL to visit
* @return whether or not the crawl was successful
*/
public boolean crawl(String url)
{
try
{
Connection connection = Jsoup.connect(url).userAgent(USER_AGENT);
Document htmlDocument = connection.get();
this.htmlDocument = htmlDocument;
if(connection.response().statusCode() == 200) // 200 is the HTTP OK status code
// indicating that everything is great.
{
System.out.println("\n**Visiting** Received web page at " + url);
}
if(!connection.response().contentType().contains("text/html"))
{
System.out.println("**Failure** Retrieved something other than HTML");
return false;
}
Elements linksOnPage = htmlDocument.select("a[href]");
System.out.println("Found (" + linksOnPage.size() + ") links");
for(Element link : linksOnPage)
{
this.links.add(link.absUrl("href"));
}
return true;
}
catch(IOException ioe)
{
// We were not successful in our HTTP request
return false;
}
}
/**
* Performs a search on the body of on the HTML document that is retrieved. This method should
* only be called after a successful crawl.
*
* @param searchWord
* - The word or string to look for
* @return whether or not the word was found
*/
public boolean searchForWord(String searchWord)
{
// Defensive coding. This method should only be used after a successful crawl.
if(this.htmlDocument == null)
{
System.out.println("ERROR! Call crawl() before performing analysis on the document");
return false;
}
System.out.println("Searching for the word " + searchWord + "...");
String bodyText = this.htmlDocument.body().text();
return bodyText.toLowerCase().contains(searchWord.toLowerCase());
}
public List<String> getLinks()
{
return this.links;
}
}
SpiderTest类
包装com.copiedcrawler;
public class SpiderTest {
public static void main(String[] args) {
// TODO Auto-generated method stub
Spider s1 = new Spider();
s1.search("https://www.w3schools.com/html/", "html");
}
}
基于堆栈跟踪,您正在从命令行运行java程序,并且您忘记将j汤添加到类路径中。尝试运行
java -cp类:libs/jsoup . jar com . copied crawler . spider test
其中类是编译的程序,libs是包含库的文件夹。
问题内容: 我使用ant脚本构建了一个可执行jar,我使用的唯一外部jar是joda-time 2.0。由于我没有收到任何编译错误,因此该蚂蚁构建脚本“似乎”可以正常工作,如果我要从lib目录中删除jode-time 2.0.jar,则构建确实会按预期失败。无论如何,构建jar之后,当我尝试运行时会出现此错误: java -jar myapp.jar 线程“主”中的异常java.lang.NoCl
问题内容: 我正在尝试使用Eclipse在Maven项目中作为Java应用程序运行,并且遇到以下运行时错误。错误如下所示。 线程“主”中的异常java.lang.NoClassDefFoundError:com.wang.testMaven.App.main(App.java:16)上的org / apache / http / ConnectionReuseStrategy造成原因:java.l
问题内容: 我尝试通过Solrj查询Solr 这是我的代码 } 这给了我一个例外 PS。我的个人有以下罐子: * apache-commons-logging, httpclient -4.2, httpcore 4.2.1,slf4j-api 1.6.2, apache-solr-solrj -1.4.0 *** 问题答案: 您需要在类路径上使用commons-codec.jar。
问题内容: 我正在尝试执行以下代码: 但这给了我这个错误: 我已将SLF4J添加到引用的库中,但不知道为什么这不起作用。 在WIN 7上运行,在Eclipse和NetBeans中尝试过 任何想法? 问题答案: 下载SLF4J并将slf4j-api-1.7.2.jar文件放到您的类路径中。
包com.copiedcrawler;
在<code>HTML</code>页面中,我想选择<code>javascript</code>变量的值 下面是页面的片段: 我的目标是使用< code>jsoup从该页面读取变量< code>key的值。< br >可以使用< code>jsoup吗?如果是,那么怎么做?