How to properly pass an asset FileDescriptor to FFmpeg using JNI in Android

android, android-ndk, ffmpeg, java-native-interface, metadata

Solution

Java

AssetFileDescriptor afd = getContext().getAssets().openFd("test.mp3");
setDataSource(afd.getFileDescriptor(), afd.getStartOffset(), fd.getLength());

C

void ***_setDataSource(JNIEnv *env, jobject thiz, 
    jobject fileDescriptor, jlong offset, jlong length)
{
    int fd = jniGetFDFromFileDescriptor(env, fileDescriptor);

    char path[20];
    sprintf(path, "pipe:%d", fd);

    State *state = av_mallocz(sizeof(State));
    state->pFormatCtx =  avformat_alloc_context();
    state->pFormatCtx->skip_initial_bytes = offset;
    state->pFormatCtx->iformat = av_find_input_format("mp3");

and now we can continue as usual:

if (avformat_open_input(&state->pFormatCtx, path, NULL, &options) != 0) {
    printf("Metadata could not be retrieved\n");
    *ps = NULL;
    return FAILURE;
}
...

Even better, use `<android/asset_manager.h>`, like this:

Java

setDataSource(getContext().getAssets(), "test.mp3");

C

#include <android/asset_manager_jni.h>

void ***_setDataSource(JNIEnv *env, jobject thiz, 
    jobject assetManager, jstring assetName)
{
    AAssetManager* assetManager = AAssetManager_fromJava(env, assetManager);
    const char *szAssetName = (*env)->GetStringUTFChars(env, assetName, NULL);
    AAsset* asset = AAssetManager_open(assetManager, szAssetName, AASSET_MODE_RANDOM);
    (*env)->ReleaseStringUTFChars(env, assetName, szAssetName);
    off_t offset, length;
    int fd = AAsset_openFileDescriptor(asset, &offset, &length);
    AAsset_close(asset);

Disclaimer: error checking was omitted for brevity, but resources are released correctly, except for fd. You must `close(fd)` when finished.

Post Scriptum: note that some media formats, e.g. mp4 need seekable protocol, and `pipe:` cannot help. In such case, you may try `sprintf(path, "/proc/self/fd/%d", fd);`, or use the custom `saf:` protocol.

Problem

I'm trying to retrieve metadata in Android using FFmpeg, JNI and a Java FileDescriptor and it isn't' working. I know FFmpeg supports the pipe protocol so I'm trying to emmulate: "`cat test.mp3 | ffmpeg i pipe:0`" programmatically. I use the following code to get a FileDescriptor from an asset bundled with the Android application: ``` FileDescriptor fd = getContext().getAssets().openFd("test.mp3").getFileDescriptor(); setDataSource(fd, 0, 0x7ffffffffffffffL); // native function, shown below ``` Then, in my native (In C++) code I get the FileDescriptor by calling: ``` static void wseemann_media_FFmpegMediaMetadataRetriever_setDataSource(JNIEnv *env, jobject thiz, jobject fileDescriptor, jlong offset, jlong length) { //... int fd = jniGetFDFromFileDescriptor(env, fileDescriptor); // function contents show below //... } // function contents static int jniGetFDFromFileDescriptor(JNIEnv * env, jobject fileDescriptor) { jint fd = -1; jclass fdClass = env->FindClass("java/io/FileDescriptor"); if (fdClass != NULL) { jfieldID fdClassDescriptorFieldID = env->GetFieldID(fdClass, "descriptor", "I"); if (fdClassDescriptorFieldID != NULL && fileDescriptor != NULL) { fd = env->GetIntField(fileDescriptor, fdClassDescriptorFieldID); } } return fd; } ``` I then pass the file descriptor pipe # (In C) to FFmpeg: ``` char path[256] = ""; FILE *file = fdopen(fd, "rb"); if (file && (fseek(file, offset, SEEK_SET) == 0)) { char str[20]; sprintf(str, "pipe:%d", fd); strcat(path, str); } State *state = av_mallocz(sizeof(State)); state->pFormatCtx = NULL; if (avformat_open_input(&state->pFormatCtx, path, NULL, &options) != 0) { // Note: path is in the format "pipe:<the FD #>" printf("Metadata could not be retrieved\n"); *ps = NULL; return FAILURE; } if (avformat_find_stream_info(state->pFormatCtx, NULL) < 0) { printf("Metadata could not be retrieved\n"); avformat_close_input(&state->pFormatCtx); *ps = NULL; return FAILURE; } // Find the first audio and video stream for (i = 0; i < state->pFormatCtx->nb_streams; i++) { if (state->pFormatCtx->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO && video_index < 0) { video_index = i; } if (state->pFormatCtx->streams[i]->codec->codec_type == AVMEDIA_TYPE_AUDIO && audio_index < 0) { audio_index = i; } set_codec(state->pFormatCtx, i); } if (audio_index >= 0) { stream_component_open(state, audio_index); } if (video_index >= 0) { stream_component_open(state, video_index); } printf("Found metadata\n"); AVDictionaryEntry *tag = NULL; while ((tag = av_dict_get(state->pFormatCtx->metadata, "", tag, AV_DICT_IGNORE_SUFFIX))) { printf("Key %s: \n", tag->key); printf("Value %s: \n", tag->value); } *ps = state; return SUCCESS; ``` My issue is `avformat_open_input` doesn't fail but it also doesn't let me retrieve any metadata or frames, The same code works if I use a regular file URI (e.g file://sdcard/test.mp3) as the path. What am I doing wrong? Thanks in advance. Note: if you would like to look at all of the code I'm trying to solve the issue in order to provide this functionality for my library: FFmpegMediaMetadataRetriever.

Original source