FarrellF.com

One of the most common uses for the Java Native Interface (JNI) is to allow Java code to interact with C libraries. In this guide I will show how to write a small Java library that uses JNI to read data from a USB device. I will use the "D2XX" API from FTDI to communicate with a USB device that implements their "Synchronous 245 FIFO" protocol. D2XX is a relatively simple C library, so it makes for a nice introduction to JNI.

Readers not familiar with D2XX might want to skim through my previous blog post: http://www.farrellf.com/projects/software/2020-04-18_FTDI_Sync_245_FIFO_Tutorial__D2XX_with_Visual_Studio_2019/.

Some Useful Links

https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/jniTOC.html
Official JNI specification. This explains how the API works and why it was designed the way it is.

https://www3.ntu.edu.sg/home/ehchua/programming/java/JavaNativeInterface.html
An excellent tutorial on JNI.

Install the IDE, C Compiler and Related Tools

While Visual Studio might be the most popular IDE for Windows, I will be using Eclipse and GCC instead. I'd like to make my library cross-platform in the future, and that will be easier to do with GCC. There are a variety of ways to install GCC and other common UNIX utilities (make, etc.) on Windows. I've settled on using MSYS2. For the IDE, I use Eclipse with the JDT (Java Developer Toolkit) CDT (C/C++ Developer Toolkit) and TM Terminal plug-ins.

1. Install Java 8.
   https://adoptopenjdk.net/ The default options in the installer are fine: Next > Accept > Next > Next > Install > Finish
2. Install Eclpse for Java Developers, CDT and TM Terminal.
   https://www.eclipse.org/downloads/packages/installer Select "Eclipse IDE for Java Developers" > Install > Accept Now > Launch In Eclipse: Check "Use this as the default and do not ask again" > Launch Help > Check for Updates Help > Eclipse Marketplace > Search for "CDT" > Install > Confirm > Accept > Finish > Restart Now Help > Eclipse Marketplace > Search for "TM Terminal" > Install > Confirm > Yes > Accept > Finish > Restart Now File > Exit
3. Install MSYS2 and add it to the PATH environment variable.
   https://www.msys2.org/ The default options in the installer are fine: Next > Next > Next > Finish In MSYS: $ pacman -Syu (and close the window when prompted to) Start > MSYS2 64bit > MSYS2 MSYS $ pacman -Syu $ pacman -S base-devel mingw-w64-x86_64-toolchain Start > "path" > Edit the System Environment Variables > Environment Variables > Path > Edit New > "C:\msys64\mingw64\bin" > New > "C:\msys64\usr\bin" > OK > OK > OK

Test Everything with a Hello World

1. Open Eclipse and Create a New C Project.
   File > New > Project > C/C++ > C Project > Next > Project Name = "HelloWorld", Toolchain = "MinGW GCC" > Next > Finish > No File > New > Other > C/C++ > Source File > Next > Source File = "main.c" > Finish
2. Write the hello world code.
   #include <stdio.h> int main(int argc, char** argv) { printf("hello world.\r\n"); return 0; }
3. Verify that it compiles and runs without error.
   Project > Build All Run > Run

Create the Java Project

1. If the C toolchain works, we can proceed to write the Java portion of the project.
   File > New > Java Project > Project Name = "EasyD2XX" > Finish File > New > Class > Package = "com.example.easyd2xx", Name = "EasyD2XX" > Finish
2. Write the Java portion of the JNI code.
   package com.example.easyd2xx; import java.nio.ByteBuffer; import java.util.List; public class EasyD2XX { public String name; public String chipName; public String serialNumber; public int location; private long handle; private EasyD2XX(String name, String chipName, String serialNumber, int location) { this.name = name; this.chipName = chipName; this.serialNumber = serialNumber; this.location = location; } /** * Automatically load the EasyD2XX.dll file before any methods of this class are called. */ static { try { System.loadLibrary("EasyD2XX"); } catch(UnsatisfiedLinkError e) { // do nothing, to allow graceful degradation if the FTDI driver or the EasyD2XX.dll file could not be found. // native function calls will now throw an UnsatasfiedLinkError when they are called. } } /** * Gets a list of devices. * * @return A list of the attached FTDI D2XX devices. */ public static native List<EasyD2XX> getDevices(); /** * Opens and configures the device for Synchronous 245 FIFO mode. * * @param readTimeoutMilliseconds Maximum amount of time to wait when reading. * @param writeTimeoutMilliseconds Maximum amount of time to wait when writing. * @throws Exception If the device could not be opened, or * if the device does not support the Synchronous 245 FIFO mode. */ public native void openAsFifo(int readTimeoutMilliseconds, int writeTimeoutMilliseconds) throws Exception; /** * Reads a series of bytes from the device into a byte[]. * * @param byteCount How many bytes to read. * @return The received bytes, as a byte[]. * @throws Exception If the read timed out, or * if the device is no longer available. */ public native byte[] read(int byteCount) throws Exception; /** * Reads a series of bytes from the device into a ByteBuffer. * * @param buffer Location to store the read bytes. * @param byteCount How many bytes to read. * @throws Exception If the read times out, or * if the device is no longer available. */ public native void read(ByteBuffer buffer, int byteCount) throws Exception; /** * Closes the device. * * @throws Exception If the device was not already open. */ public native void close() throws Exception; }
3. Create a test class that can be used as a demo and as verification of proper functionality.
   File > New > Class > Name = Test > Finish
4. Write the test code.
   package com.example.easyd2xx; import java.nio.ByteBuffer; import java.util.List; import java.util.Scanner; public class Test { /** * A simple test for the EasyD2XX class. * * @param args Not currently used. */ public static void main(String[] args) { try { // get a list of devices List<EasyD2XX> devices = EasyD2XX.getDevices(); if(devices.isEmpty()) { System.out.println("No devices were detected. Exiting."); return; } System.out.println("Select a device to read from:"); System.out.println(); for(int i = 0; i < devices.size(); i++) { System.out.println("Device " + i + ":"); EasyD2XX device = devices.get(i); System.out.println("Name: " + device.name); System.out.println("Chip: " + device.chipName); System.out.println("SN: " + device.serialNumber); System.out.println("Location: " + device.location); System.out.println(); } // let the user pick a device Scanner stdin = new Scanner(System.in); int deviceIndex = stdin.nextInt(); stdin.close(); // connect EasyD2XX device = devices.get(deviceIndex); device.openAsFifo(1000, 1000); // read 1GB into a byte[] long start = System.currentTimeMillis(); @SuppressWarnings("unused") byte[] oneGbArray = device.read(1073741824); long stop = System.currentTimeMillis(); System.out.println("Read 1GB into a byte[] in " + (stop - start) + "ms."); // also read 1GB into a ByteBuffer start = System.currentTimeMillis(); ByteBuffer buffer = ByteBuffer.allocateDirect(1073741824); device.read(buffer, 1073741824); stop = System.currentTimeMillis(); System.out.println("Read 1GB into a ByteBuffer in " + (stop - start) + "ms."); // disconnect device.close(); System.out.println("Done. Exiting."); } catch(Exception | UnsatisfiedLinkError e) { System.out.println(e.getMessage() + " Exiting."); e.printStackTrace(); } } }

At this point the Java code is done and will compile just fine, but at run time you'll get a UnsatisfiedLinkError when any of the "native" methods are called. That is because the native (JNI) code needs to be written and compiled into a .dll file.

Add the C Portion of JNI Code to the Project

1. Add C support to the Java project.
   File > New > Other > C/C++ > Convert to a C/C++ Project (Adds C/C++ Nature) > Next > check the project, choose "C Project", choose "Makefile Project" and "MinGW GCC" > Finish > No
2. Add a Makefile.
   File > New > File > choose the project root directory, Filename = "makefile" > Finish
3. Write the makefile with three targets. The first target, "make header" will be used to generate the JNI stubs. The next two targets are just helpers that print out signatures for your Java code ("make signatures") and also for any other Java class ("make sig"). Those two targets are optional, I just include them so I don't have to rememeber the commands to type in.
   # "make header" to generate the .h file header: mkdir -p jni javac -h jni src/com/example/easyd2xx/EasyD2xx.java rm src/com/example/easyd2xx/EasyD2xx.class # "make sig" to ask the user for a class name, then print the field and method signatures for that class sig: @bash -c 'read -p "Fully-qualified class name (example: java.util.List) ? " CLASSNAME && javap -s $$CLASSNAME'; # "make signatures" to print the field and method signatures for the EasyD2XX class signatures: javac src/com/farrellf/d2xx/EasyD2xx.java -d bin javap -s -p bin/com/farrellf/d2xx/EasyD2xx.class
4. Show the "Build Targets" tab in Eclipse, then add targets for "header" and "signatures".
   Window > Show View > Other > Make > Build Targets > Open Select the project folder New Build Target > Target Name = "header" > OK New Build Target > Target Name = "signatures" > OK
5. Run the "make header" target to generature the .h file.
   Double-click the "header" target to run it.

There will now be a "jni" subfolder in the project, containing a header file with stubs for each native method.

Unfortunately "make sig" can not be run from the Build Targets panel because Eclipse does not connect stdin when running it. Instead, you can run "make sig" from a terminal. I use the "TM Terminal" plug-in for Eclipse. With that plug-in, just select the project folder, then Ctrl-Alt-T to open a terminal in that folder.

If you open the com_example_easyd2xx_EasyD2XX.h file in Eclipse, you'll get lots of warnings and errors, because Eclipse doesn't know where to find the JNI headers (the "#include <jni.h>" line.)

1. Add the JNI header folders to the Preprocessor Include Path.
   Project > Properties > C/C++ General > Preprocessor Include Paths, Macros etc. > Entires > GNU C > CDT User Setting Entries > Add Select "File System Path" Path = C:\Program Files\AdoptOpenJDK\jdk-8.0.252.09-hotspot\include Check "treat as built-in" Check "contains system headers" OK Repeat that again, but for path C:\Program Files\AdoptOpenJDK\jdk-8.0.252.09-hotspot\include\win32 Apply and Close
2. Copy the D2XX .h and .lib files into the "jni" folder.
   Copy "ftd2xx.h" and "amd64\ftd2xx.lib" from the FTDI ZIP file into the "jni" folder Then in Eclipse: right-click the jni folder > Refresh
3. Duplicate the com_example_easyd2xx_EasyD2XX.h file, rename the copy to .c, and write the C portion of the JNI code.
   #include "com_example_easyd2xx_EasyD2XX.h" #include "ftd2xx.h" #include <string.h> JNIEXPORT jobject JNICALL Java_com_example_easyd2xx_EasyD2XX_getDevices(JNIEnv* env, jclass thisClass) { // create an empty ArrayList object jclass listClass = (*env)->FindClass(env, "java/util/ArrayList"); if(listClass == NULL) return NULL; jmethodID listConstructor = (*env)->GetMethodID(env, listClass, "<init>", "()V"); if(listConstructor == NULL) return NULL; jobject list = (*env)->NewObject(env, listClass, listConstructor); if(list == NULL) return NULL; // get a handle for "list.add(object)" jmethodID listAddMethodHandle = (*env)->GetMethodID(env, listClass, "add", "(Ljava/lang/Object;)Z"); if(listAddMethodHandle == NULL) return NULL; // get a handle for "new EasyD2XX(name, chipName, serialNumber, location)" jmethodID easyD2xxConstructor = (*env)->GetMethodID(env, thisClass, "<init>", "(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;I)V"); if(easyD2xxConstructor == NULL) return NULL; // get the number of devices and build (but not read) the devices info list DWORD deviceCount = 0; if(FT_CreateDeviceInfoList(&deviceCount) != FT_OK) printf("Unable to get the FTDI devices count.\r\n"); // get the devices info list FT_DEVICE_LIST_INFO_NODE *devices = (FT_DEVICE_LIST_INFO_NODE*) malloc(sizeof(FT_DEVICE_LIST_INFO_NODE) * deviceCount); if(FT_GetDeviceInfoList(devices, &deviceCount) != FT_OK) printf("Unable to get the device info list.\r\n"); // for each FTDI device, create an EasyD2XX object and add it to the list for(DWORD i = 0; i < deviceCount; i++) { jstring name = (*env)->NewStringUTF(env, devices[i].Description); jstring chipName = devices[i].Type == FT_DEVICE_BM ? (*env)->NewStringUTF(env, "FT232BM") : devices[i].Type == FT_DEVICE_AM ? (*env)->NewStringUTF(env, "FT232AM") : devices[i].Type == FT_DEVICE_100AX ? (*env)->NewStringUTF(env, "100AX") : devices[i].Type == FT_DEVICE_UNKNOWN ? (*env)->NewStringUTF(env, "[Unknown Device]") : devices[i].Type == FT_DEVICE_2232C ? (*env)->NewStringUTF(env, "FT2232C") : devices[i].Type == FT_DEVICE_232R ? (*env)->NewStringUTF(env, "FT232R") : devices[i].Type == FT_DEVICE_2232H ? (*env)->NewStringUTF(env, "FT2232H") : devices[i].Type == FT_DEVICE_4232H ? (*env)->NewStringUTF(env, "FT4232H") : devices[i].Type == FT_DEVICE_232H ? (*env)->NewStringUTF(env, "FT232H") : devices[i].Type == FT_DEVICE_X_SERIES ? (*env)->NewStringUTF(env, "X Series") : devices[i].Type == FT_DEVICE_4222H_0 ? (*env)->NewStringUTF(env, "FT4222H, 0") : devices[i].Type == FT_DEVICE_4222H_1_2 ? (*env)->NewStringUTF(env, "FT4222H, 1-2") : devices[i].Type == FT_DEVICE_4222H_3 ? (*env)->NewStringUTF(env, "FT4222H, 3") : devices[i].Type == FT_DEVICE_4222_PROG ? (*env)->NewStringUTF(env, "FT4222, Prog") : devices[i].Type == FT_DEVICE_900 ? (*env)->NewStringUTF(env, "FT900 Series") : devices[i].Type == FT_DEVICE_930 ? (*env)->NewStringUTF(env, "FT930 Series") : devices[i].Type == FT_DEVICE_UMFTPD3A ? (*env)->NewStringUTF(env, "UMFTPD3A") : (*env)->NewStringUTF(env, "[Unknown Device]"); jstring serialNumber = (*env)->NewStringUTF(env, devices[i].SerialNumber); jint location = devices[i].LocId; jobject newEasyD2XXobject = (*env)->NewObject(env, thisClass, easyD2xxConstructor, name, chipName, serialNumber, location); if(newEasyD2XXobject == NULL) { free(devices); return NULL; } // list.add(newObject) (*env)->CallBooleanMethod(env, list, listAddMethodHandle, newEasyD2XXobject); if((*env)->ExceptionCheck(env)) { free(devices); return NULL; } } // done free(devices); return list; } JNIEXPORT void JNICALL Java_com_example_easyd2xx_EasyD2XX_openAsFifo(JNIEnv* env, jobject this, jint readTimeoutMilliseconds, jint writeTimeoutMilliseconds) { // get a handle for this class jclass thisClass = (*env)->GetObjectClass(env, this); if(thisClass == NULL) return; // get a handle for the Exception class, in case we need to throw an Exception jclass exception = (*env)->FindClass(env, "java/lang/Exception"); if(exception == NULL) return; // check if the device is a FT2232H or FT232H, because only those devices support FIFO mode jfieldID chipNameHandle = (*env)->GetFieldID(env, thisClass, "chipName", "Ljava/lang/String;"); if(chipNameHandle == NULL) return; jstring chipName = (*env)->GetObjectField(env, this, chipNameHandle); if(chipName == NULL) return; const char* chipNameCstring = (*env)->GetStringUTFChars(env, chipName, NULL); if(strcmp(chipNameCstring, "FT2232H") != 0 && strcmp(chipNameCstring, "FT232H") != 0) { (*env)->ReleaseStringUTFChars(env, chipName, chipNameCstring); (*env)->ThrowNew(env, exception, "Device does not support Synchronous 245 FIFO mode."); return; } (*env)->ReleaseStringUTFChars(env, chipName, chipNameCstring); FT_HANDLE ftdiHandle = 0; // open by location if possible (not possible on linux) jfieldID locationHandle = (*env)->GetFieldID(env, thisClass, "location", "I"); if(locationHandle == NULL) return; jint location = (*env)->GetIntField(env, this, locationHandle); if(FT_OpenEx((void*)(uintptr_t)location, FT_OPEN_BY_LOCATION, &ftdiHandle) != FT_OK) { // open by name if open by location failed jfieldID nameHandle = (*env)->GetFieldID(env, thisClass, "name", "Ljava/lang/String;"); if(nameHandle == NULL) return; jstring name = (*env)->GetObjectField(env, this, nameHandle); if(name == NULL) return; const char* nameCstring = (*env)->GetStringUTFChars(env, name, NULL); if(FT_OpenEx((void*)nameCstring, FT_OPEN_BY_DESCRIPTION, &ftdiHandle) != FT_OK) { (*env)->ReleaseStringUTFChars(env, name, nameCstring); (*env)->ThrowNew(env, exception, "Unable to open the device."); return; } (*env)->ReleaseStringUTFChars(env, name, nameCstring); } // configure the device if(FT_SetBitMode(ftdiHandle, 0xFF, 0x40) != FT_OK || // sync 245 FIFO mode FT_SetLatencyTimer(ftdiHandle, 2) != FT_OK || // minimum latency FT_SetUSBParameters(ftdiHandle, 65536, 65536) != FT_OK || // 64K buffers FT_SetFlowControl(ftdiHandle, FT_FLOW_RTS_CTS, 0, 0) != FT_OK || // flow control FT_Purge(ftdiHandle, FT_PURGE_RX | FT_PURGE_TX) != FT_OK || // flush FIFOs FT_SetTimeouts(ftdiHandle, readTimeoutMilliseconds, writeTimeoutMilliseconds) != FT_OK) { // timeouts // failure (*env)->ThrowNew(env, exception, "Unable to configure the device."); return; } else { // success jfieldID ftdiHandleHandle = (*env)->GetFieldID(env, thisClass, "handle", "J"); (*env)->SetLongField(env, this, ftdiHandleHandle, (uintptr_t) ftdiHandle); } } JNIEXPORT jbyteArray JNICALL Java_com_example_easyd2xx_EasyD2XX_read__I(JNIEnv* env, jobject this, jint byteCount) { // get a handle for this class jclass thisClass = (*env)->GetObjectClass(env, this); if(thisClass == NULL) return NULL; // get a handle for the Exception class, in case we need to throw an Exception jclass exception = (*env)->FindClass(env, "java/lang/Exception"); if(exception == NULL) return NULL; // get the value of "this.handle" jfieldID ftdiHandleHandle = (*env)->GetFieldID(env, thisClass, "handle", "J"); if(ftdiHandleHandle == NULL) return NULL; FT_HANDLE ftdiHandle = (FT_HANDLE) (uintptr_t) (*env)->GetLongField(env, this, ftdiHandleHandle); // create a new byte[] jbyteArray array = (*env)->NewByteArray(env, byteCount); if(array == NULL) return NULL; jbyte* buffer = (*env)->GetByteArrayElements(env, array, NULL); // read into the byte[] jint bytesRead = 0; while(byteCount > 0) { jint amount = (byteCount < 65536) ? byteCount : 65536; DWORD readAmount = 0; if(FT_Read(ftdiHandle, &buffer[bytesRead], amount, &readAmount) != FT_OK) { (*env)->ReleaseByteArrayElements(env, array, buffer, 0); (*env)->ThrowNew(env, exception, "Unable to read from the device."); return array; } bytesRead += readAmount; byteCount -= readAmount; } // done (*env)->ReleaseByteArrayElements(env, array, buffer, 0); return array; } JNIEXPORT void JNICALL Java_com_example_easyd2xx_EasyD2XX_read__Ljava_nio_ByteBuffer_2I(JNIEnv* env, jobject this, jobject buffer, jint byteCount) { // get a handle for this class jclass thisClass = (*env)->GetObjectClass(env, this); if(thisClass == NULL) return; // get a handle for the Exception class, in case we need to throw an Exception jclass exception = (*env)->FindClass(env, "java/lang/Exception"); if(exception == NULL) return; // get the value of "this.handle" jfieldID ftdiHandleHandle = (*env)->GetFieldID(env, thisClass, "handle", "J"); if(ftdiHandleHandle == NULL) return; FT_HANDLE ftdiHandle = (FT_HANDLE) (uintptr_t) (*env)->GetLongField(env, this, ftdiHandleHandle); // get the buffer and ensure it is big enough void* bufferPtr = (*env)->GetDirectBufferAddress(env, buffer); if(bufferPtr == NULL) return; jlong bufferSize = (*env)->GetDirectBufferCapacity(env, buffer); if(bufferSize < byteCount) { (*env)->ThrowNew(env, exception, "The buffer does not have enough space."); return; } // read into the ByteBuffer jint bytesRead = 0; while(byteCount > 0) { jint amount = (byteCount < 65536) ? byteCount : 65536; DWORD readAmount = 0; if(FT_Read(ftdiHandle, &((char*)bufferPtr)[bytesRead], amount, &readAmount) != FT_OK) { (*env)->ThrowNew(env, exception, "Unable to read from the device."); return; } bytesRead += readAmount; byteCount -= readAmount; } } JNIEXPORT void JNICALL Java_com_example_easyd2xx_EasyD2XX_close(JNIEnv* env, jobject this) { // get a handle for this class jclass thisClass = (*env)->GetObjectClass(env, this); if(thisClass == NULL) return; // get a handle for the Exception class, in case we need to throw an Exception jclass exception = (*env)->FindClass(env, "java/lang/Exception"); if(exception == NULL) return; // get the value of "this.handle" jfieldID ftdiHandleHandle = (*env)->GetFieldID(env, thisClass, "handle", "J"); if(ftdiHandleHandle == NULL) return; // close the device FT_HANDLE ftdiHandle = (FT_HANDLE) (uintptr_t) (*env)->GetLongField(env, this, ftdiHandleHandle); if(FT_Close(ftdiHandle) != FT_OK) { (*env)->ThrowNew(env, exception, "Unable to close the device."); return; } }
4. Add a "dll" target to the makefile. It will be used to compile the JNI portion of the project into a .dll file. You can also add an "all" target, so that "make" with automatically call the "dll" target if no target is specified.
   dll: gcc jni/com_example_easyd2xx_EasyD2XX.c jni/ftd2xx.lib -I"C:/Program Files/AdoptOpenJDK/jdk-8.0.242.08-hotspot/include" -I"C:/Program Files/AdoptOpenJDK/jdk-8.0.242.08-hotspot/include/win32" -shared -o EasyD2XX.dll file EasyD2XX.dll nm EasyD2XX.dll | grep Java ldd EasyD2XX.dll all: dll
5. In the "Build Targets" tab, add another build target like before, but for "dll". Double-click the target to run it.

Keep in mind that you can compile this without the D2XX drivers installed, and there will be no error messages. But if you run it, Java will not be able to find the D2XX DLL at run time, and you will get an UnsatisfiedLinkError even if the EasyD2XX.dll file is fine. That is part of why I call "ldd" while compiling the DLL. If ldd prints out some lines with "???" then you will have problems at run time. For example, before installing the D2XX driver, I get this:

After installing the D2XX driver, I get this:

You can now run the Java code with Run > Run, and it should print out a list of attached FTDI devices. That's all there is to a basic JNI project. You could merge the code into an existing project, or export this project as a Jar file so it can be used as a library for other projects.

Unlike regular Java code, JNI code is obviously platform-dependant. In other words, you have to compile a .dll/.so/.dylib file for each operating system and each architecture that you wish to support. This guide only covered 64-bit Windows, but in my next post I will cover other platforms and show how to bundle everything into a single Jar file.

Finally, a small note for anyone familiar with the D2XX library: FTDI provides their library in both static and dynamic forms. I used the dynamic form because the static version for Windows will not link properly when using GCC. It seems like FTDI only intends for it to be used with the Visual C++ compiler.

YouTube Video