DirectMapping Calls Failing

[PeteSL]

Just tried jna-4.2.0.jar on a RaspberryPi V2 running Raspbian Linux and JDK 8 build 51 and received the following in 2 different programs using 2 completely different libraries:

java.lang.NullPointerException
        at com.sun.jna.Native.toNative(Native.java:1736)

The specific methods being called are

native public NativeSize write(int fd, byte[] buffer, NativeSize count);
public static native NativeSize sendto(int __fd, Pointer __buf, NativeSize __n, int __flags, sockaddr __addr, int __addr_len);

NativeSize is a size_t encapsulation:

public class NativeSize extends IntegerType
{
    /**
     *
     */
    private static final long serialVersionUID = 2398288011955445078L;
    /**
     * Size of a size_t integer, in bytes.
     */
    public static int SIZE = Native.SIZE_T_SIZE;//Platform.is64Bit() ? 8 : 4;

    /**
     * Create a zero-valued Size.
     */
    public NativeSize()
    {
        this(0);
    }

    /**
     * Create a Size with the given value.
     */
    public NativeSize(long value)
    {
        super(SIZE, value);
    }
}

Switched back to 4.1.0 and working perfectly.

[PeteSL]

The JNA code in question is the return statement below:

    // Called from native code
    private static Object toNative(ToNativeConverter cvt, Object o) {
        // NOTE: technically should be either CallbackResultContext or
        // FunctionParameterContext
        return cvt.toNative(o, new ToNativeContext());
    }

Because the NullPointerException is being thrown at that line and not in either the toNative() call or ToNativeContext() contructor, I would say cvt is null. Since this is "called from native code", that is where to start, I would guess. The RPi 2 is a quad core ARMv7 device and the Raspbian (Debian Linux derivative) is 3.18.11-v7+.

I did clear the temp files from 4.1.0 before trying 4.2.0 and again before switching back to 4.1.0.

[twall]

Try adding -Djna.nosys=true to your VM args. You might be picking up a system-installed JNA.

[twall]

The origin of the NullPointerException (Native.toNative(ToNativeConverter cvt, Object o)) is not used for NativeMapped types, but rather for TypeMapper-mapped types. Is there a TypeMapper in play for your library?

[PeteSL]

There is no system-installed JNA. As indicated, the only thing "unique" that I see in the library functions are the use of the NativeSize class which was originally generated by JNAerator for one of the applications and I used in the other application. I see your commit that references this issue. Is there something I should do with that or are you testing? As I say, all working with 4.1.0 with no changes, but 4.2.0 breaks. I am using 4.2.0 on a Windows machine but that API doesn't use the NativeSize class (doesn't use size_t).

[PeteSL]

-Djna.nosys=true had no effect. I also reduced the NativeSize class to simply the 2 constructors (no SIZE or serialVersionUID since not used elsewhere) and renamed it NativeSize_t for clarity. I will be interested in your test results.

[PeteSL]

I looked inside the 4.2.0 jar vs. the 4.1.0 jar and the libjnidispatch.so for linux-arm (I am assuming that is what is being loaded) is timestamped 2014-11-06 in 4.2.0 and 2013-06-04 in 4.1.0 so there is definitely a difference in native libraries.

[twall]

Yes, linux-arm would be what’s loaded on a Raspberry Pi. It’s quite possible the linux-arm binary didn’t get updated in the release.

So you’re seeing this on linux-arm and Windows (x86 or x86-64?)?

On Sep 19, 2015, at 10:21 AM, PeteSL notifications@github.com wrote:

I looked inside the 4.2.0 jar vs. the 4.1.0 jar and the libjnidispatch.so for linux-arm (I am assuming that is what is being loaded) is timestamped 2014-11-06 in 4.2.0 and 2013-06-04 in 4.1.0 so there is definitely a difference in native libraries.

I am using 4.2.0 on a Windows machine but that API doesn't use the NativeSize class (doesn't use size_t).

[PeteSL]

I am seeing this with linux-arm, NOT Windows but NativeSize_t is not used in the Windows APIs that I am using. To recap, 4.1.0 linux-arm works with no error, 4.2.0 linux-arm fails with above error. Focus is linux-arm in 4.2.0 (and possibly other implementations?) apparently not handling an IntegerType subclass as a parameter (parameter since it is toNative, not fromNative which would be for a return value).

[twall]

I've committed a clean build of libjnidispatch.so (within lib/native/linux-arm.jar) for linux-arm on branch 507-linux-arm-NPE. If you get a chance, see if that resolves the error. Also trying to repro on local Raspberry Pi setup.

[PeteSL]

I am going to be a bit dumb here but should I just download the new libjnidispatch.so and replace it in my jna-4.2.0.jar? I understand a jar is just a zip, just want to make sure I don't break anything.

[PeteSL]

I will run 2 tests with the 4.2.0 jar. First will be with the libjnidispatch.so you just compiled. Second will be with the libjnidispatch.so from the 4.1.0 jar. I will run both even if the new compile works without error and report back on my results.

[PeteSL]

Placed the 4.1.0 libjnidispatch.so in the 4.2.0 jar, same issue. Placed the new libjnidipatch.so in the 4.2.0 jar, working without error.

[PeteSL]

Just an observation, most likely way off base but..., because the original 2014 library failed and the 4.1.0 library failed at the same place but the newly compiled library did not fail, maybe something changed in the calls to the JNI dispatch library in 4.2.0 which precludes using older libraries with 4.2.0. Just an observation, nothing more...

[twall]

Ok, thanks for checking. There were no API changes at the JNI level, but apparently the actual API usage is sufficiently different to cause an issue, although it does seem odd given that the error trigger is coming from the native level in the older library.

On Sep 19, 2015, at 10:16 PM, PeteSL notifications@github.com wrote:

Just an observation, most likely way off base but..., because the original 2014 library failed and the 4.1.0 library failed at the same place but the newly compiled library did not fail, maybe something changed in the calls to the JNI dispatch library in 4.2.0 which precludes using older libraries with 4.2.0. Just an observation, nothing more...

—
Reply to this email directly or view it on GitHub.

[PeteSL]

The reason I mentioned it was because if this is true for the linux-arm libjnidispatch.so, could it be true of other dispatch libraries? Glad you were able to isolate the issue.

[twall]

I was able to reproduce the NPE using the released versions ("old" JNI library and "new" Java code). The IllegalArgumentExceptions raised due to WString errors are expected, since the older JNI stub does not support that type. That was a bug fixed in this latest release.

...........E..................E........E.
Time: 2.148
There were 3 errors:
1) (WString IllegalArgument exception)
2) testNativeLongArgument(com.sun.jna.DirectArgumentsMarshalTest)java.lang.NullPointerException
    at com.sun.jna.Native.toNative(Native.java:1736)
    at com.sun.jna.DirectArgumentsMarshalTest$DirectTestLibrary.returnLongArgument(Native Method)
    at com.sun.jna.ArgumentsMarshalTest.testNativeLongArgument(ArgumentsMarshalTest.java:242)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at com.sun.jna.DirectArgumentsMarshalTest.main(DirectArgumentsMarshalTest.java:139)
3) testNativeMappedArgument(com.sun.jna.DirectArgumentsMarshalTest)java.lang.NullPointerException
    at com.sun.jna.Native.toNative(Native.java:1736)
    at com.sun.jna.DirectArgumentsMarshalTest$DirectNativeMappedLibrary.returnInt32Argument(Native Method)
    at com.sun.jna.ArgumentsMarshalTest.testNativeMappedArgument(ArgumentsMarshalTest.java:297)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at com.sun.jna.DirectArgumentsMarshalTest.main(DirectArgumentsMarshalTest.java:139)

FAILURES!!!
Tests run: 38,  Failures: 0,  Errors: 3

twall@raspberrypi7 ~/jna $ java -cp build/classes:build/test-classes:lib/junit.jar -Djna.library.path=build/native-linux-arm com.sun.jna.DirectReturnTypesTest
.....E..E..E.........
Time: 1.44
There were 3 errors:
1) testInvokeNativeLong(com.sun.jna.DirectReturnTypesTest)java.lang.NullPointerException
    at com.sun.jna.Native.fromNative(Native.java:1741)
    at com.sun.jna.DirectReturnTypesTest$DirectTestLibrary.returnLongZero(Native Method)
    at com.sun.jna.ReturnTypesTest.testInvokeNativeLong(ReturnTypesTest.java:169)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at com.sun.jna.DirectReturnTypesTest.main(DirectReturnTypesTest.java:95)
2) (WString IllegalArgumentException)
3) testInvokeNativeMapped(com.sun.jna.DirectReturnTypesTest)java.lang.NullPointerException
    at com.sun.jna.Native.fromNative(Native.java:1741)
    at com.sun.jna.DirectReturnTypesTest$DirectNativeMappedLibrary.returnInt32Magic(Native Method)
    at com.sun.jna.ReturnTypesTest.testInvokeNativeMapped(ReturnTypesTest.java:228)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at com.sun.jna.DirectReturnTypesTest.main(DirectReturnTypesTest.java:95)

FAILURES!!!
Tests run: 18,  Failures: 0,  Errors: 3

[twall]

While the root cause is that the linux-arm native library was not updated in this release, the newer Java code was still expected to be backwards-compatible with the older JNI code. Obviously this is not entirely the case.

Thanks again for tracking this down.

[PeteSL]

Glad you were able to reproduce it. I will leave this open and let you close it when you determine it is resolved to your satisfaction since it may affect other native builds.