jsoup code works in Java but not in Android- Nullp

2019-08-21 06:07发布

站内文章 / Java
36 0 0

问题:

I'm trying to write a weather scraping android app using jsoup. I have code working in Java, but even the simplest form I can write in android throws an error. Here's part of my code:

private class Retrieve_Weather extends AsyncTask<Void, Void, String> {
    private final String TAG = "Retrieve_Weather";

    protected String doInBackground(Void... params) {

        String weatherline= "oTHErly";
        Log.i(TAG, weatherline);
        Document doc = null;
        try {
            Log.i(TAG, "trying document...");
            doc = Jsoup.connect("http://m.wund.com/cgi-bin/findweather/getForecast?brand=mobile&query=02139").get();
            Log.i(TAG, "after document");
            weatherline = "newline";
            }
        catch (IOException except) {
            Log.i(TAG, "Cannot Connect", except);   
            except.printStackTrace();
            weatherline = "Can't Connect";
            Log.i(TAG, "End stacktrace");

        }

        Log.i(TAG, weatherline);
        return weatherline;
    }

    @Override
    protected void onPreExecute() {
        Log.i(TAG, "onPreExecute");
    }

    @Override
    protected void onProgressUpdate(Void... values) {
        Log.i(TAG, "onProgressUpdate");
    }
    @Override
    protected void onPostExecute(String result){
        //super.onPostExecute(result);
        Log.i(TAG, "onPostExecute");
    }


}

I've played around with my code and logcat enough to figure out that my error is a nullpointerexception thrown when I run .get(). I just can't figure out why or how to fix it. Here's my stacktrace:

07-09 03:18:13.901: I/Retrieve_Weather(1754): trying document...
07-09 03:18:15.900: D/dalvikvm(1754): GC_CONCURRENT freed 80K, 7% free 2711K/2912K, paused 83ms+20ms, total 244ms
07-09 03:18:15.910: D/dalvikvm(1754): WAIT_FOR_CONCURRENT_GC blocked 157ms
07-09 03:18:16.351: D/dalvikvm(1754): GC_CONCURRENT freed 297K, 15% free 2841K/3340K, paused 81ms+124ms, total 329ms
07-09 03:18:16.740: W/dalvikvm(1754): Exception Ljava/lang/NullPointerException; thrown while initializing Lorg/jsoup/nodes/Entities;
07-09 03:18:16.740: W/dalvikvm(1754): Exception Ljava/lang/ExceptionInInitializerError; thrown while initializing Lorg/jsoup/nodes/Entities$EscapeMode;
07-09 03:18:16.810: W/System.err(1754): java.util.concurrent.ExecutionException: java.lang.ExceptionInInitializerError
07-09 03:18:16.830: W/System.err(1754):     at java.util.concurrent.FutureTask.report(FutureTask.java:94)
07-09 03:18:16.830: W/System.err(1754):     at java.util.concurrent.FutureTask.get(FutureTask.java:160)
07-09 03:18:16.860: W/System.err(1754):     at android.os.AsyncTask.get(AsyncTask.java:482)
07-09 03:18:16.880: W/System.err(1754):     at com.example.weatherresponse.Weather.onCreate(Weather.java:35)
07-09 03:18:16.880: W/System.err(1754):     at android.app.Activity.performCreate(Activity.java:5104)
07-09 03:18:16.911: W/System.err(1754):     at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1080)
07-09 03:18:16.911: W/System.err(1754):     at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2144)
07-09 03:18:16.960: W/System.err(1754):     at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2230)
07-09 03:18:16.970: W/System.err(1754):     at android.app.ActivityThread.access$600(ActivityThread.java:141)
07-09 03:18:16.970: W/System.err(1754):     at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1234)
07-09 03:18:16.980: W/System.err(1754):     at android.os.Handler.dispatchMessage(Handler.java:99)
07-09 03:18:16.990: W/System.err(1754):     at android.os.Looper.loop(Looper.java:137)
07-09 03:18:16.990: W/System.err(1754):     at android.app.ActivityThread.main(ActivityThread.java:5041)
07-09 03:18:16.990: W/System.err(1754):     at java.lang.reflect.Method.invokeNative(Native Method)
07-09 03:18:17.010: W/System.err(1754):     at java.lang.reflect.Method.invoke(Method.java:511)
07-09 03:18:17.061: W/System.err(1754):     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:793)
07-09 03:18:17.061: W/System.err(1754):     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:560)
07-09 03:18:17.061: W/System.err(1754):     at dalvik.system.NativeStart.main(Native Method)
07-09 03:18:17.090: W/System.err(1754): Caused by: java.lang.ExceptionInInitializerError
07-09 03:18:17.090: W/System.err(1754):     at org.jsoup.nodes.Document$OutputSettings.<init>(Document.java:210)
07-09 03:18:17.150: W/dalvikvm(1754): threadid=11: thread exiting with uncaught exception (group=0x40a71930)
07-09 03:18:17.150: W/System.err(1754):     at org.jsoup.nodes.Document.<init>(Document.java:18)
07-09 03:18:17.160: W/System.err(1754):     at org.jsoup.parser.TreeBuilder.initialiseParse(TreeBuilder.java:27)
07-09 03:18:17.160: W/System.err(1754):     at org.jsoup.parser.TreeBuilder.parse(TreeBuilder.java:40)
07-09 03:18:17.160: W/System.err(1754):     at org.jsoup.parser.HtmlTreeBuilder.parse(HtmlTreeBuilder.java:37)
07-09 03:18:17.170: W/System.err(1754):     at org.jsoup.parser.Parser.parseInput(Parser.java:30)
07-09 03:18:17.170: W/System.err(1754):     at org.jsoup.helper.DataUtil.parseByteData(DataUtil.java:102)
07-09 03:18:17.170: W/System.err(1754):     at org.jsoup.helper.HttpConnection$Response.parse(HttpConnection.java:498)
07-09 03:18:17.215: W/System.err(1754):     at org.jsoup.helper.HttpConnection.get(HttpConnection.java:154)
07-09 03:18:17.215: W/System.err(1754):     at com.example.weatherresponse.Weather$Retrieve_Weather.doInBackground(Weather.java:85)
07-09 03:18:17.220: W/System.err(1754):     at com.example.weatherresponse.Weather$Retrieve_Weather.doInBackground(Weather.java:1)
07-09 03:18:17.220: W/System.err(1754):     at android.os.AsyncTask$2.call(AsyncTask.java:287)
07-09 03:18:17.230: W/System.err(1754):     at java.util.concurrent.FutureTask.run(FutureTask.java:234)
07-09 03:18:17.230: W/System.err(1754):     at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:230)
07-09 03:18:17.240: W/System.err(1754):     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1080)
07-09 03:18:17.240: W/System.err(1754):     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:573)
07-09 03:18:17.240: W/System.err(1754):     at java.lang.Thread.run(Thread.java:856)
07-09 03:18:17.260: W/System.err(1754): Caused by: java.lang.ExceptionInInitializerError
07-09 03:18:17.290: W/System.err(1754):     at org.jsoup.nodes.Entities$EscapeMode.<clinit>(Entities.java:20)
07-09 03:18:17.290: W/System.err(1754):     ... 17 more
07-09 03:18:17.290: W/System.err(1754): Caused by: java.lang.NullPointerException: in == null
07-09 03:18:17.290: W/System.err(1754):     at java.util.Properties.load(Properties.java:246)
07-09 03:18:17.290: W/System.err(1754):     at org.jsoup.nodes.Entities.loadEntities(Entities.java:151)
07-09 03:18:17.290: W/System.err(1754):     at org.jsoup.nodes.Entities.<clinit>(Entities.java:135)
07-09 03:18:17.290: W/System.err(1754):     ... 18 more
07-09 03:18:17.383: I/System.out(1754): ExecExceptjava.lang.ExceptionInInitializerError
07-09 03:18:19.570: E/AndroidRuntime(1754): FATAL EXCEPTION: AsyncTask #1
07-09 03:18:19.570: E/AndroidRuntime(1754): java.lang.RuntimeException: An error occured while executing doInBackground()
07-09 03:18:19.570: E/AndroidRuntime(1754):     at android.os.AsyncTask$3.done(AsyncTask.java:299)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:352)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at java.util.concurrent.FutureTask.setException(FutureTask.java:219)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at java.util.concurrent.FutureTask.run(FutureTask.java:239)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:230)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1080)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:573)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at java.lang.Thread.run(Thread.java:856)
07-09 03:18:19.570: E/AndroidRuntime(1754): Caused by: java.lang.ExceptionInInitializerError
07-09 03:18:19.570: E/AndroidRuntime(1754):     at org.jsoup.nodes.Document$OutputSettings.<init>(Document.java:210)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at org.jsoup.nodes.Document.<init>(Document.java:18)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at org.jsoup.parser.TreeBuilder.initialiseParse(TreeBuilder.java:27)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at org.jsoup.parser.TreeBuilder.parse(TreeBuilder.java:40)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at org.jsoup.parser.HtmlTreeBuilder.parse(HtmlTreeBuilder.java:37)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at org.jsoup.parser.Parser.parseInput(Parser.java:30)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at org.jsoup.helper.DataUtil.parseByteData(DataUtil.java:102)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at org.jsoup.helper.HttpConnection$Response.parse(HttpConnection.java:498)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at org.jsoup.helper.HttpConnection.get(HttpConnection.java:154)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at com.example.weatherresponse.Weather$Retrieve_Weather.doInBackground(Weather.java:85)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at com.example.weatherresponse.Weather$Retrieve_Weather.doInBackground(Weather.java:1)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at android.os.AsyncTask$2.call(AsyncTask.java:287)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at java.util.concurrent.FutureTask.run(FutureTask.java:234)
07-09 03:18:19.570: E/AndroidRuntime(1754):     ... 4 more
07-09 03:18:19.570: E/AndroidRuntime(1754): Caused by: java.lang.ExceptionInInitializerError
07-09 03:18:19.570: E/AndroidRuntime(1754):     at org.jsoup.nodes.Entities$EscapeMode.<clinit>(Entities.java:20)
07-09 03:18:19.570: E/AndroidRuntime(1754):     ... 17 more
07-09 03:18:19.570: E/AndroidRuntime(1754): Caused by: java.lang.NullPointerException: in == null
07-09 03:18:19.570: E/AndroidRuntime(1754):     at java.util.Properties.load(Properties.java:246)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at org.jsoup.nodes.Entities.loadEntities(Entities.java:151)
07-09 03:18:19.570: E/AndroidRuntime(1754):     at org.jsoup.nodes.Entities.<clinit>(Entities.java:135)
07-09 03:18:19.570: E/AndroidRuntime(1754):     ... 18 more

My code runs correctly except for this single, basic line. I've tested my code on both an emulator and an android phone, and both had internet access. I've tried multiple websites, and the mobile versions of those websites, but none have worked. I have internet permissions in my manifests and my library files in the correct place. I've looked around a lot on this site and others for solutions to my problem. I've modified my code and permissions due to what I've found in related posts, but I have yet to find an answered post that works for me. I have, however, found a few unanswered posts that ask the exact same question. The only answer I found that I haven't tried is to this question Error when releasing APK with AsyncTask and Jsoup: I can't figure out which file to change this in to test if it solve my issue. I'm fairly new to programming- any help would be greatly appreciated. Thanks!

回答1:

Add these lines to proguard-project.txt

-keep public enum * {
    public static **[] values();
    public static ** valueOf(java.lang.String);
}


回答2:

My code works now! I ended up moving my .jar file into lib instead of using it as a referenced external jar, as recommended in Android-SDK r17 ruins working projects. I also added a user agent and reworked my AsyncTask so that Jsoup.connect(url) is in onPreExecute while leaving .get() in doInBackground.

    private class Retrieve_Weather extends AsyncTask<Void, Void, String> {
    private final String TAG = "Retrieve_Weather";
    Connection con = null;

    protected Void doInBackground(String... params) {

        //Log.i(TAG, weatherline);
        Document doc = null;
        if (con !=null){
            try {
                Log.i(TAG, "trying document...");
                doc = con.get();
                Log.i(TAG, "after document");
                Element currentWeather = doc.getElementById("curCond");
                currentweather = currentWeather.text();
                Log.i(TAG, currentWeather.text());
                }
            catch (IOException except) {
                Log.i(TAG, "Cannot Connect", except);   
                except.printStackTrace();
                currentweather = "Can't Connect";
                Log.i(TAG, "End stacktrace");

            }

        }
        else{
            currentweather = "Connection returned null";
        }

        //Log.i(TAG, weatherline);
        return currentweather;

    }

    @Override
    protected void onPreExecute() {
        //Log.i(TAG, "onPreExecute");
        Log.i(TAG, "trying to connect...");
        con = Jsoup.connect("http://www.wunderground.com/cgi-bin/findweather/getForecast?query=zmw:02142.1.99999").userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0");
        Log.i(TAG, "CONNECTED!!!");

        }




    @Override
    protected void onProgressUpdate(Void... values) {
        Log.i(TAG, "onProgressUpdate");
    }

    @Override
    protected void onPostExecute(Void result){
        Log.i(TAG, "onPostExecute");

    }


}


回答3:

looking at the source code for org.jsoup.nodes.Entties https://github.com/jhy/jsoup/blob/master/src/main/java/org/jsoup/nodes/Entities.java

looks like its trying to load "entities-base.properties" from class path and unable to find it. Can you make sure these files are "entities-full.properties" available and in the classpath.

just a hint for future debugging always look for deepest exception, in your case 

07-09 03:18:17.290: W/System.err(1754): Caused by: java.lang.NullPointerException: in == null
07-09 03:18:17.290: W/System.err(1754):     at java.util.Properties.load(Properties.java:246)
07-09 03:18:17.290: W/System.err(1754):     at org.jsoup.nodes.Entities.loadEntities(Entities.java:151)
07-09 03:18:17.290: W/System.err(1754):     at org.jsoup.nodes.Entities.<clinit>(Entities.java:135)

The last line pointed me to where to look for. hope this helps



回答4:

After some research, I believe this is actually an Android bug. It seems that Android caches the META-INF/MANIFEST.MF file, but does not invalidate the cache when reinstalling the app. As a result, the system doesn't see the new .property files, which causes getResourceAsStream to return null.

Solution: Reboot the device.



标签： java android nullpointerexception web-scraping jsoup
举报

收藏的人(0)

Ta的文章 更多文章
登录 后发表评论
0条评论
还没有人评论过~