Replace The Camera Stream In A Wikitude Architectview With An External Video Stream

I'm working on an Android mobile app oriented to the real time augmented visualization of a drone's camera view (specifically I'm working on a DJI Phantom 3 Professional with relat

Solution 1:

You could implement your own Wikitude Input Plugin in combination with dji's Video Stream Decoding. So you get the raw video data from the dji sdk and pass it to the wikitude sdk. A sample code for the Input Plugin can be found in the wikitude sample app with the sample name "Custom Camera".

Solution 2:

Alex answer is correct and it works. Here are some more details: The Wikitude Input Plugin can be used to supply your own camera input and your can also apply your own augmentations and your own rendering. See The Wikitude Input Plugin documentation.

According to the Wikitude samples, the Custom Camera Plugin can be used as a starting point. You develop your own Camera Extension and within the constructor, you do something like this:

    arVideoView = newSurfaceViewRenderer(activity);
    arVideoView.setId(R.id.local_surface_rendeer);
    LinearLayout.LayoutParamsparam1=newLinearLayout.LayoutParams(
            FrameLayout.LayoutParams.MATCH_PARENT,
            FrameLayout.LayoutParams.MATCH_PARENT,
            1.0f
    );
    arVideoView.setLayoutParams(param1);
    arVideoView.setKeepScreenOn(true);
    architectView.addView(arVideoView);

In this case I added a WebRTC camera view to the Architect View.

Then I implemented a callback for the incoming frames.

// Implementation detail: bridge the VideoRenderer.Callbacks interface to the// VideoStreamsView implementation.publicclassVideoCallbacksimplementsVideoRenderer.Callbacks {
    @OverridepublicvoidrenderFrame(final VideoRenderer.I420Frame i420Frame) {
        Log.v(TAG, "renderFrame ..."+i420Frame);

.. convert the frames to nv21 Log.v(TAG, "make byte array"); byte[] b = getNV21(0,0, bmpRotate.getWidth(), bmpRotate.getHeight(), bmpRotate);

// and pass the frame to the plugin for further processing notifyNewCameraFrameN21(b);

                gotFrame = false;
            }
        });
    }
}

Hope this helps. Happy coding! Phil