Mediapipe: I am use mediapipe with camera 2.0,but get trouble.

[a798347923]

Hello everyone !
I am use mediapipe with camera 2.0,but get trouble.I create a custom Camera2TextureView add it in xml and get theSurfaceTexture.But when setSurfaceTextureAndAttachToGLContext get the trouble.
The total message is:
03/11 23:30:36: Launching 'app' on samsung SM-G9730. $ adb shell am start -n "com.example.myapplication/com.example.myapplication.MainActivity" -a android.intent.action.MAIN -c android.intent.category.LAUNCHER Connected to process 29180 on device 'samsung-sm_g9730-R28M22J0DDB'. Capturing and displaying logcat messages from application. This behavior can be disabled in the "Logcat output" section of the "Debugger" settings page. D/Camera2Initializer: CameraX initializing with Camera2 ... I/CameraManagerGlobal: Connecting to camera service D/VendorTagDescriptor: addVendorDescriptor: vendor tag id 3854507339 added I/CameraManagerGlobal: Camera 0 facing CAMERA_FACING_BACK state now CAMERA_STATE_CLOSED for client com.example.myapplication API Level 2 I/CameraManagerGlobal: Camera 1 facing CAMERA_FACING_FRONT state now CAMERA_STATE_CLOSED for client com.example.myapplication API Level 2 Camera 2 facing CAMERA_FACING_BACK state now CAMERA_STATE_CLOSED for client android.system API Level 2 Camera 20 facing CAMERA_FACING_BACK state now CAMERA_STATE_CLOSED for client android.system API Level 2 Camera 21 facing CAMERA_FACING_BACK state now CAMERA_STATE_CLOSED for client android.system API Level 2 I/CameraManagerGlobal: Camera 23 facing CAMERA_FACING_BACK state now CAMERA_STATE_CLOSED for client android.system API Level 2 Camera 3 facing CAMERA_FACING_FRONT state now CAMERA_STATE_CLOSED for client com.example.myapplication API Level 2 I/CameraManagerGlobal: Camera 40 facing CAMERA_FACING_BACK state now CAMERA_STATE_CLOSED for client android.system API Level 2 I/CameraManagerGlobal: Camera 41 facing CAMERA_FACING_FRONT state now CAMERA_STATE_CLOSED for client android.system API Level 2 Camera 50 facing CAMERA_FACING_BACK state now CAMERA_STATE_CLOSED for client android.system API Level 2 I/CameraManagerGlobal: Camera 52 facing CAMERA_FACING_BACK state now CAMERA_STATE_CLOSED for client android.system API Level 2 Camera 91 facing CAMERA_FACING_FRONT state now CAMERA_STATE_CLOSED for client com.samsung.android.bio.face.service API Level 2 D/CameraRepository: Added camera: 0 D/UseCaseAttachState: Active and online use case: [] for camera: 0 I/CameraManagerGlobal: postSingleUpdate device: camera id 0 status STATUS_PRESENT postSingleUpdate device: camera id 1 status STATUS_PRESENT postSingleUpdate device: camera id 2 status STATUS_PRESENT I/CameraManagerGlobal: postSingleUpdate device: camera id 3 status STATUS_PRESENT D/CameraRepository: Added camera: 1 I/CameraManagerGlobal: postSingleUpdate device: camera id 0 status STATUS_PRESENT postSingleUpdate device: camera id 1 status STATUS_PRESENT D/UseCaseAttachState: Active and online use case: [] for camera: 1 I/CameraManagerGlobal: postSingleUpdate device: camera id 2 status STATUS_PRESENT postSingleUpdate device: camera id 3 status STATUS_PRESENT D/CameraRepository: Added camera: 2 I/CameraManagerGlobal: postSingleUpdate device: camera id 0 status STATUS_PRESENT postSingleUpdate device: camera id 1 status STATUS_PRESENT postSingleUpdate device: camera id 2 status STATUS_PRESENT postSingleUpdate device: camera id 3 status STATUS_PRESENT D/CameraRepository: Added camera: 3 D/UseCaseAttachState: Active and online use case: [] for camera: 2 I/CameraManagerGlobal: postSingleUpdate device: camera id 0 status STATUS_PRESENT I/CameraManagerGlobal: postSingleUpdate device: camera id 1 status STATUS_PRESENT D/UseCaseAttachState: Active and online use case: [] for camera: 3 I/CameraManagerGlobal: postSingleUpdate device: camera id 2 status STATUS_PRESENT I/CameraManagerGlobal: postSingleUpdate device: camera id 3 status STATUS_PRESENT I/MultiWindowDecorSupport: updateCaptionType >> DecorView@c84c675[], isFloating: false, isApplication: true, hasWindowDecorCaption: false, hasWindowControllerCallback: true D/MultiWindowDecorSupport: setCaptionType = 0, DecorView = DecorView@c84c675[] W/e.myapplicatio: Accessing hidden method Landroid/view/View;->computeFitSystemWindows(Landroid/graphics/Rect;Landroid/graphics/Rect;)Z (greylist, reflection, allowed) Accessing hidden method Landroid/view/ViewGroup;->makeOptionalFitsSystemWindows()V (greylist, reflection, allowed) I/native: I0311 23:30:44.833956 29180 asset_manager_util.cc:43] Created global reference to asset manager. I/AdrenoGLES: QUALCOMM build : def403d, I0e419467bc Build Date : 11/06/19 OpenGL ES Shader Compiler Version: EV031.27.05.02 Local Branch : Remote Branch : refs/tags/AU_LINUX_ANDROID_LA.UM.8.1.R1.10.00.00.529.103 Remote Branch : NONE Reconstruct Branch : NOTHING Build Config : S P 8.0.11 AArch64 I/AdrenoGLES: PFP: 0x016ee187, ME: 0x00000000 E/libEGL: validate_display:244 error 3008 (EGL_BAD_DISPLAY) I/native: I0311 23:30:44.854032 29180 gl_context_egl.cc:158] Successfully initialized EGL. Major : 1 Minor: 5 I/native: I0311 23:30:44.855998 29363 gl_context.cc:233] GL version: 3.2 (OpenGL ES 3.2 [email protected] (GIT@def403d, I0e419467bc, 1573061038) (Date:11/06/19)) D/PermissionHelper: checkAndRequestCameraPermissions D/GlThread: Starting GL thread ExternalTextureConverter D/ViewRootImpl@b6055ba[MainActivity]: setView = DecorView@c84c675[MainActivity] TM=true MM=false D/GlThread: Stopping GL thread ExternalTextureConverter D/SurfaceView: onWindowVisibilityChanged(0) false android.view.SurfaceView{dc5d9d G.E...... ......I. 0,0-0,0} of ViewRootImpl@b6055ba[MainActivity] D/ViewRootImpl@b6055ba[MainActivity]: Relayout returned: old=[0,0][1080,2280] new=[0,0][1080,2280] req=(1080,2280)0 dur=9 res=0x7 s={true 507075100672} ch=true D/OpenGLRenderer: createReliableSurface : 0x761421ba80, 0x7610080000 D/OpenGLRenderer: SurfaceChanged : 0x0 -> 0x0 V/CameraTextureView: onSurfaceTextureAvailable. width: 1080, height: 2021 D/OpenGLRenderer: SurfaceChanged : 0x0 -> 0x761430c000 W/Gralloc3: mapper 3.x is not supported D/ViewRootImpl@b6055ba[MainActivity]: Relayout returned: old=[0,0][1080,2280] new=[0,0][1080,2280] req=(1080,2280)0 dur=23 res=0x1 s={true 507075100672} ch=false D/ViewRootImpl@b6055ba[MainActivity]: MSG_RESIZED_REPORT: frame=[0,0][1080,2280] ci=[0,112][0,0] vi=[0,112][0,0] or=1 D/PermissionHelper: onRequestPermissionsResult android.permission.CAMERA permission granted. D/ViewRootImpl@b6055ba[MainActivity]: stopped(false) old=false D/GlThread: Starting GL thread ExternalTextureConverter I/Camera2CameraInfo: Device Level: INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED D/Camera: Use cases [Preview:androidx.camera.core.Preview-afb5e548-f775-439c-9138-c5c4f7e5f098] ONLINE for camera 1 D/UseCaseAttachState: Active and online use case: [] for camera: 1 D/Camera: Resetting Capture Session D/Camera: releasing session in state INITIALIZED D/ViewRootImpl@b6055ba[MainActivity]: MSG_WINDOW_FOCUS_CHANGED 1 1 D/InputMethodManager: prepareNavigationBarInfo() DecorView@c84c675[MainActivity] getNavigationBarColor() -855310 D/Camera: Transitioning camera internal state: INITIALIZED --> OPENING Opening camera: 1 D/UseCaseAttachState: All use case: [androidx.camera.core.Preview-afb5e548-f775-439c-9138-c5c4f7e5f098] for camera: 1 D/InputMethodManager: prepareNavigationBarInfo() DecorView@c84c675[MainActivity] getNavigationBarColor() -855310 V/InputMethodManager: Starting input: tba=com.example.myapplication ic=null mNaviBarColor -855310 mIsGetNaviBarColorSuccess true , NavVisible : true , NavTrans : false D/InputMethodManager: startInputInner - Id : 0 I/InputMethodManager: startInputInner - mService.startInputOrWindowGainedFocus I/CameraManagerGlobal: postSingleUpdate device: camera id 1 status STATUS_NOT_AVAILABLE I/CameraManagerGlobal: postSingleUpdate device: camera id 1 status STATUS_NOT_AVAILABLE I/CameraManagerGlobal: postSingleUpdate device: camera id 1 status STATUS_NOT_AVAILABLE postSingleUpdate device: camera id 1 status STATUS_NOT_AVAILABLE I/CameraManagerGlobal: Camera 1 facing CAMERA_FACING_FRONT state now CAMERA_STATE_OPEN for client com.example.myapplication API Level 2 D/ViewRootImpl@b6055ba[MainActivity]: Relayout returned: old=[0,0][1080,2280] new=[0,0][1080,2280] req=(1080,2280)0 dur=9 res=0x1 s={true 507075100672} ch=false D/Camera: Use case Preview:androidx.camera.core.Preview-afb5e548-f775-439c-9138-c5c4f7e5f098 ACTIVE for camera 1 D/UseCaseAttachState: Active and online use case: [androidx.camera.core.Preview-afb5e548-f775-439c-9138-c5c4f7e5f098] for camera: 1 D/Camera: CameraDevice.onOpened(): 1 Transitioning camera internal state: OPENING --> OPENED D/UseCaseAttachState: All use case: [androidx.camera.core.Preview-afb5e548-f775-439c-9138-c5c4f7e5f098] for camera: 1 D/SurfaceView: surfaceCreated 1 #8 android.view.SurfaceView{dc5d9d V.E...... ......ID 0,0-1080,2021} D/SurfaceView: surfaceChanged (1080,2021) 1 #8 android.view.SurfaceView{dc5d9d V.E...... ......ID 0,0-1080,2021} E/OpenGLRenderer: [SurfaceTexture-0-29180-0] attachToContext: SurfaceTexture is already attached to a context E/AndroidRuntime: FATAL EXCEPTION: ExternalTextureConverter Process: com.example.myapplication, PID: 29180 java.lang.RuntimeException: Error during attachToGLContext (see logcat for details) at android.graphics.SurfaceTexture.attachToGLContext(SurfaceTexture.java:294) at com.google.mediapipe.components.ExternalTextureConverter$RenderThread.setSurfaceTextureAndAttachToGLContext(ExternalTextureConverter.java:197) at com.google.mediapipe.components.ExternalTextureConverter.lambda$setSurfaceTextureAndAttachToGLContext$1$ExternalTextureConverter(ExternalTextureConverter.java:120) at com.google.mediapipe.components.-$$Lambda$ExternalTextureConverter$5RBq2uHoPl06wf2nG84kcgcgP_U.run(Unknown Source:8) at android.os.Handler.handleCallback(Handler.java:883) at android.os.Handler.dispatchMessage(Handler.java:100) at android.os.Looper.loop(Looper.java:237) at com.google.mediapipe.glutil.GlThread.run(GlThread.java:134) D/InputMethodManager: prepareNavigationBarInfo() DecorView@c84c675[MainActivity] getNavigationBarColor() -855310 V/InputMethodManager: Starting input: tba=com.example.myapplication ic=null mNaviBarColor -855310 mIsGetNaviBarColorSuccess true , NavVisible : true , NavTrans : false D/InputMethodManager: startInputInner - Id : 0 D/CaptureSession: Opening capture session. I/Process: Sending signal. PID: 29180 SIG: 9
Could someone help me?If you want more message ,also can ask me.

there are my codes:
`
public class Camera2Proxy {

private static final String TAG = "Camera2Proxy";

private Activity mActivity;

private int mCameraId = CameraCharacteristics.LENS_FACING_FRONT; // 要打开的摄像头ID
private Size mPreviewSize; // 预览大小
private CameraManager mCameraManager; // 相机管理者
private CameraCharacteristics mCameraCharacteristics; // 相机属性
private CameraDevice mCameraDevice; // 相机对象
private CameraCaptureSession mCaptureSession;
private CaptureRequest.Builder mPreviewRequestBuilder; // 相机预览请求的构造器
private CaptureRequest mPreviewRequest;
private Handler mBackgroundHandler;
private HandlerThread mBackgroundThread;
private ImageReader mImageReader;
private Surface mPreviewSurface;
private SurfaceTexture mPreviewSurfaceTexture;
private OrientationEventListener mOrientationEventListener;

private int mDisplayRotate = 0;
private int mDeviceOrientation = 0; // 设备方向，由相机传感器获取
private int mZoom = 1; // 缩放

/**
 * 打开摄像头的回调
 */
private CameraDevice.StateCallback mStateCallback = new CameraDevice.StateCallback() {
    @Override
    public void onOpened(@NonNull CameraDevice camera) {
        Log.d(TAG, "onOpened");
        mCameraDevice = camera;
        initPreviewRequest();
    }

    @Override
    public void onDisconnected(@NonNull CameraDevice camera) {
        Log.d(TAG, "onDisconnected");
        releaseCamera();
    }

    @Override
    public void onError(@NonNull CameraDevice camera, int error) {
        Log.e(TAG, "Camera Open failed, error: " + error);
        releaseCamera();
    }
};

@TargetApi(Build.VERSION_CODES.M)
public Camera2Proxy(Activity activity) {
    mActivity = activity;
    mCameraManager = (CameraManager) mActivity.getSystemService(Context.CAMERA_SERVICE);
    mOrientationEventListener = new OrientationEventListener(mActivity) {
        @Override
        public void onOrientationChanged(int orientation) {
            mDeviceOrientation = orientation;
        }
    };
}

@SuppressLint("MissingPermission")
public void openCamera(int width, int height) {
    Log.v(TAG, "openCamera");
    startBackgroundThread(); // 对应 releaseCamera() 方法中的 stopBackgroundThread()
    mOrientationEventListener.enable();
    try {
        mCameraCharacteristics = mCameraManager.getCameraCharacteristics(Integer.toString(mCameraId));
        StreamConfigurationMap map = mCameraCharacteristics.get(CameraCharacteristics
                .SCALER_STREAM_CONFIGURATION_MAP);
        // 拍照大小，选择能支持的一个最大的图片大小
        Size largest = Collections.max(Arrays.asList(map.getOutputSizes(ImageFormat.JPEG)), new
                CompareSizesByArea());
        Log.d(TAG, "picture size: " + largest.getWidth() + "*" + largest.getHeight());
        mImageReader = ImageReader.newInstance(largest.getWidth(), largest.getHeight(), ImageFormat.JPEG, 2);
        // 预览大小，根据上面选择的拍照图片的长宽比，选择一个和控件长宽差不多的大小
        mPreviewSize = chooseOptimalSize(map.getOutputSizes(SurfaceTexture.class), width, height, largest);
        Log.d(TAG, "preview size: " + mPreviewSize.getWidth() + "*" + mPreviewSize.getHeight());
        // 打开摄像头
        mCameraManager.openCamera(Integer.toString(mCameraId), mStateCallback, mBackgroundHandler);
    } catch (CameraAccessException e) {
        e.printStackTrace();
    }
}

public void releaseCamera() {
    Log.v(TAG, "releaseCamera");
    if (null != mCaptureSession) {
        mCaptureSession.close();
        mCaptureSession = null;
    }
    if (mCameraDevice != null) {
        mCameraDevice.close();
        mCameraDevice = null;
    }
    if (mImageReader != null) {
        mImageReader.close();
        mImageReader = null;
    }
    mOrientationEventListener.disable();
    stopBackgroundThread(); // 对应 openCamera() 方法中的 startBackgroundThread()
}

public void setImageAvailableListener(ImageReader.OnImageAvailableListener onImageAvailableListener) {
    if (mImageReader == null) {
        Log.w(TAG, "setImageAvailableListener: mImageReader is null");
        return;
    }
    mImageReader.setOnImageAvailableListener(onImageAvailableListener, null);
}

public void setPreviewSurface(SurfaceHolder holder) {
    mPreviewSurface = holder.getSurface();
}

public void setPreviewSurface(SurfaceTexture surfaceTexture) {
    mPreviewSurfaceTexture = surfaceTexture;
}

private void initPreviewRequest() {
    try {
        mPreviewRequestBuilder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
        if (mPreviewSurfaceTexture != null && mPreviewSurface == null) { // use texture view
            mPreviewSurfaceTexture.setDefaultBufferSize(mPreviewSize.getWidth(), mPreviewSize.getHeight());
            mPreviewSurface = new Surface(mPreviewSurfaceTexture);
        }
        mPreviewRequestBuilder.addTarget(mPreviewSurface); // 设置预览输出的 Surface
        mCameraDevice.createCaptureSession(Arrays.asList(mPreviewSurface, mImageReader.getSurface()),
                new CameraCaptureSession.StateCallback() {

                    @Override
                    public void onConfigured(@NonNull CameraCaptureSession session) {
                        mCaptureSession = session;
                        // 设置连续自动对焦
                        mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_MODE, CaptureRequest
                                .CONTROL_AF_MODE_CONTINUOUS_PICTURE);
                        // 设置自动曝光
                        mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest
                                .CONTROL_AE_MODE_ON_AUTO_FLASH);
                        // 设置完后自动开始预览
                        mPreviewRequest = mPreviewRequestBuilder.build();
                        startPreview();
                    }

                    @Override
                    public void onConfigureFailed(@NonNull CameraCaptureSession session) {
                        Log.e(TAG, "ConfigureFailed. session: mCaptureSession");
                    }
                }, mBackgroundHandler); // handle 传入 null 表示使用当前线程的 Looper
    } catch (CameraAccessException e) {
        e.printStackTrace();
    }
}

public void startPreview() {
    Log.v(TAG, "startPreview");
    if (mCaptureSession == null || mPreviewRequestBuilder == null) {
        Log.w(TAG, "startPreview: mCaptureSession or mPreviewRequestBuilder is null");
        return;
    }
    try {
        // 开始预览，即一直发送预览的请求
        mCaptureSession.setRepeatingRequest(mPreviewRequest, null, mBackgroundHandler);
    } catch (CameraAccessException e) {
        e.printStackTrace();
    }
}

public void stopPreview() {
    Log.v(TAG, "stopPreview");
    if (mCaptureSession == null || mPreviewRequestBuilder == null) {
        Log.w(TAG, "stopPreview: mCaptureSession or mPreviewRequestBuilder is null");
        return;
    }
    try {
        mCaptureSession.stopRepeating();
    } catch (CameraAccessException e) {
        e.printStackTrace();
    }
}

public void captureStillPicture() {
    try {
        CaptureRequest.Builder captureBuilder = mCameraDevice.createCaptureRequest(CameraDevice
                .TEMPLATE_STILL_CAPTURE);
        captureBuilder.addTarget(mImageReader.getSurface());
        captureBuilder.set(CaptureRequest.CONTROL_AF_MODE, CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE);
        captureBuilder.set(CaptureRequest.JPEG_ORIENTATION, getJpegOrientation(mDeviceOrientation));
        // 预览如果有放大，拍照的时候也应该保存相同的缩放
        Rect zoomRect = mPreviewRequestBuilder.get(CaptureRequest.SCALER_CROP_REGION);
        if (zoomRect != null) {
            captureBuilder.set(CaptureRequest.SCALER_CROP_REGION, zoomRect);
        }
        mCaptureSession.stopRepeating();
        mCaptureSession.abortCaptures();
        final long time = System.currentTimeMillis();
        mCaptureSession.capture(captureBuilder.build(), new CameraCaptureSession.CaptureCallback() {
            @Override
            public void onCaptureCompleted(@NonNull CameraCaptureSession session,
                                           @NonNull CaptureRequest request,
                                           @NonNull TotalCaptureResult result) {
                Log.w(TAG, "onCaptureCompleted, time: " + (System.currentTimeMillis() - time));
                try {
                    mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_TRIGGER, CameraMetadata
                            .CONTROL_AF_TRIGGER_CANCEL);
                    mCaptureSession.capture(mPreviewRequestBuilder.build(), null, mBackgroundHandler);
                } catch (CameraAccessException e) {
                    e.printStackTrace();
                }
                startPreview();
            }
        }, mBackgroundHandler);
    } catch (CameraAccessException e) {
        e.printStackTrace();
    }
}

private int getJpegOrientation(int deviceOrientation) {
    if (deviceOrientation == android.view.OrientationEventListener.ORIENTATION_UNKNOWN) return 0;
    int sensorOrientation = mCameraCharacteristics.get(CameraCharacteristics.SENSOR_ORIENTATION);
    // Round device orientation to a multiple of 90
    deviceOrientation = (deviceOrientation + 45) / 90 * 90;
    // Reverse device orientation for front-facing cameras
    boolean facingFront = mCameraCharacteristics.get(CameraCharacteristics.LENS_FACING) == CameraCharacteristics
            .LENS_FACING_FRONT;
    if (facingFront) deviceOrientation = -deviceOrientation;
    // Calculate desired JPEG orientation relative to camera orientation to make
    // the image upright relative to the device orientation
    int jpegOrientation = (sensorOrientation + deviceOrientation + 360) % 360;
    Log.d(TAG, "jpegOrientation: " + jpegOrientation);
    return jpegOrientation;
}

public boolean isFrontCamera() {
    return mCameraId == CameraCharacteristics.LENS_FACING_BACK;
}

public Size getPreviewSize() {
    return mPreviewSize;
}

public void switchCamera(int width, int height) {
    mCameraId ^= 1;
    Log.d(TAG, "switchCamera: mCameraId: " + mCameraId);
    releaseCamera();
    openCamera(width, height);
}

private Size chooseOptimalSize(Size[] sizes, int viewWidth, int viewHeight, Size pictureSize) {
    int totalRotation = getRotation();
    boolean swapRotation = totalRotation == 90 || totalRotation == 270;
    int width = swapRotation ? viewHeight : viewWidth;
    int height = swapRotation ? viewWidth : viewHeight;
    return getSuitableSize(sizes, width, height, pictureSize);
}

private int getRotation() {
    int displayRotation = mActivity.getWindowManager().getDefaultDisplay().getRotation();
    switch (displayRotation) {
        case Surface.ROTATION_0:
            displayRotation = 90;
            break;
        case Surface.ROTATION_90:
            displayRotation = 0;
            break;
        case Surface.ROTATION_180:
            displayRotation = 270;
            break;
        case Surface.ROTATION_270:
            displayRotation = 180;
            break;
    }
    int sensorOrientation = mCameraCharacteristics.get(CameraCharacteristics.SENSOR_ORIENTATION);
    mDisplayRotate = (displayRotation + sensorOrientation + 270) % 360;
    return mDisplayRotate;
}

private Size getSuitableSize(Size[] sizes, int width, int height, Size pictureSize) {
    int minDelta = Integer.MAX_VALUE; // 最小的差值，初始值应该设置大点保证之后的计算中会被重置
    int index = 0; // 最小的差值对应的索引坐标
    float aspectRatio = pictureSize.getHeight() * 1.0f / pictureSize.getWidth();
    Log.d(TAG, "getSuitableSize. aspectRatio: " + aspectRatio);
    for (int i = 0; i < sizes.length; i++) {
        Size size = sizes[i];
        // 先判断比例是否相等
        if (size.getWidth() * aspectRatio == size.getHeight()) {
            int delta = Math.abs(width - size.getWidth());
            if (delta == 0) {
                return size;
            }
            if (minDelta > delta) {
                minDelta = delta;
                index = i;
            }
        }
    }
    return sizes[index];
}

public void handleZoom(boolean isZoomIn) {
    if (mCameraDevice == null || mCameraCharacteristics == null || mPreviewRequestBuilder == null) {
        return;
    }
    int maxZoom = mCameraCharacteristics.get(CameraCharacteristics.SCALER_AVAILABLE_MAX_DIGITAL_ZOOM).intValue()
            * 10;
    Log.d(TAG, "handleZoom: maxZoom: " + maxZoom);
    Rect rect = mCameraCharacteristics.get(CameraCharacteristics.SENSOR_INFO_ACTIVE_ARRAY_SIZE);
    if (isZoomIn && mZoom < maxZoom) {
        mZoom++;
    } else if (mZoom > 1) {
        mZoom--;
    }
    Log.d(TAG, "handleZoom: mZoom: " + mZoom);
    int minW = rect.width() / maxZoom;
    int minH = rect.height() / maxZoom;
    int difW = rect.width() - minW;
    int difH = rect.height() - minH;
    int cropW = difW * mZoom / 100;
    int cropH = difH * mZoom / 100;
    cropW -= cropW & 3;
    cropH -= cropH & 3;
    Log.d(TAG, "handleZoom: cropW: " + cropW + ", cropH: " + cropH);
    Rect zoomRect = new Rect(cropW, cropH, rect.width() - cropW, rect.height() - cropH);
    mPreviewRequestBuilder.set(CaptureRequest.SCALER_CROP_REGION, zoomRect);
    mPreviewRequest = mPreviewRequestBuilder.build();
    startPreview(); // 需要重新 start preview 才能生效
}

public void focusOnPoint(double x, double y, int width, int height) {
    if (mCameraDevice == null || mPreviewRequestBuilder == null) {
        return;
    }
    // 1. 先取相对于view上面的坐标
    int previewWidth = mPreviewSize.getWidth();
    int previewHeight = mPreviewSize.getHeight();
    if (mDisplayRotate == 90 || mDisplayRotate == 270) {
        previewWidth = mPreviewSize.getHeight();
        previewHeight = mPreviewSize.getWidth();
    }
    // 2. 计算摄像头取出的图像相对于view放大了多少，以及有多少偏移
    double tmp;
    double imgScale;
    double verticalOffset = 0;
    double horizontalOffset = 0;
    if (previewHeight * width > previewWidth * height) {
        imgScale = width * 1.0 / previewWidth;
        verticalOffset = (previewHeight - height / imgScale) / 2;
    } else {
        imgScale = height * 1.0 / previewHeight;
        horizontalOffset = (previewWidth - width / imgScale) / 2;
    }
    // 3. 将点击的坐标转换为图像上的坐标
    x = x / imgScale + horizontalOffset;
    y = y / imgScale + verticalOffset;
    if (90 == mDisplayRotate) {
        tmp = x;
        x = y;
        y = mPreviewSize.getHeight() - tmp;
    } else if (270 == mDisplayRotate) {
        tmp = x;
        x = mPreviewSize.getWidth() - y;
        y = tmp;
    }
    // 4. 计算取到的图像相对于裁剪区域的缩放系数，以及位移
    Rect cropRegion = mPreviewRequestBuilder.get(CaptureRequest.SCALER_CROP_REGION);
    if (cropRegion == null) {
        Log.w(TAG, "can't get crop region");
        cropRegion = mCameraCharacteristics.get(CameraCharacteristics.SENSOR_INFO_ACTIVE_ARRAY_SIZE);
    }
    int cropWidth = cropRegion.width();
    int cropHeight = cropRegion.height();
    if (mPreviewSize.getHeight() * cropWidth > mPreviewSize.getWidth() * cropHeight) {
        imgScale = cropHeight * 1.0 / mPreviewSize.getHeight();
        verticalOffset = 0;
        horizontalOffset = (cropWidth - imgScale * mPreviewSize.getWidth()) / 2;
    } else {
        imgScale = cropWidth * 1.0 / mPreviewSize.getWidth();
        horizontalOffset = 0;
        verticalOffset = (cropHeight - imgScale * mPreviewSize.getHeight()) / 2;
    }
    // 5. 将点击区域相对于图像的坐标，转化为相对于成像区域的坐标
    x = x * imgScale + horizontalOffset + cropRegion.left;
    y = y * imgScale + verticalOffset + cropRegion.top;
    double tapAreaRatio = 0.1;
    Rect rect = new Rect();
    rect.left = clamp((int) (x - tapAreaRatio / 2 * cropRegion.width()), 0, cropRegion.width());
    rect.right = clamp((int) (x + tapAreaRatio / 2 * cropRegion.width()), 0, cropRegion.width());
    rect.top = clamp((int) (y - tapAreaRatio / 2 * cropRegion.height()), 0, cropRegion.height());
    rect.bottom = clamp((int) (y + tapAreaRatio / 2 * cropRegion.height()), 0, cropRegion.height());
    // 6. 设置 AF、AE 的测光区域，即上述得到的 rect
    mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_REGIONS, new MeteringRectangle[]{new MeteringRectangle
            (rect, 1000)});
    mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AE_REGIONS, new MeteringRectangle[]{new MeteringRectangle
            (rect, 1000)});
    mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_MODE, CaptureRequest.CONTROL_AF_MODE_AUTO);
    mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_TRIGGER, CameraMetadata.CONTROL_AF_TRIGGER_START);
    mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AE_PRECAPTURE_TRIGGER, CameraMetadata
            .CONTROL_AE_PRECAPTURE_TRIGGER_START);
    try {
        // 7. 发送上述设置的对焦请求，并监听回调
        mCaptureSession.capture(mPreviewRequestBuilder.build(), mAfCaptureCallback, mBackgroundHandler);
    } catch (CameraAccessException e) {
        e.printStackTrace();
    }
}

private final CameraCaptureSession.CaptureCallback mAfCaptureCallback = new CameraCaptureSession.CaptureCallback() {

    private void process(CaptureResult result) {
        Integer state = result.get(CaptureResult.CONTROL_AF_STATE);
        if (null == state) {
            return;
        }
        Log.d(TAG, "process: CONTROL_AF_STATE: " + state);
        if (state == CaptureResult.CONTROL_AF_STATE_FOCUSED_LOCKED || state == CaptureResult
                .CONTROL_AF_STATE_NOT_FOCUSED_LOCKED) {
            Log.d(TAG, "process: start normal preview");
            mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_TRIGGER, CameraMetadata.CONTROL_AF_TRIGGER_CANCEL);
            mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_MODE, CaptureRequest
                    .CONTROL_AF_MODE_CONTINUOUS_PICTURE);
            mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.FLASH_MODE_OFF);
            startPreview();
        }
    }

    @Override
    public void onCaptureProgressed(@NonNull CameraCaptureSession session,
                                    @NonNull CaptureRequest request,
                                    @NonNull CaptureResult partialResult) {
        process(partialResult);
    }

    @Override
    public void onCaptureCompleted(@NonNull CameraCaptureSession session,
                                   @NonNull CaptureRequest request,
                                   @NonNull TotalCaptureResult result) {
        process(result);
    }
};


private void startBackgroundThread() {
    if (mBackgroundThread == null || mBackgroundHandler == null) {
        Log.v(TAG, "startBackgroundThread");
        mBackgroundThread = new HandlerThread("CameraBackground");
        mBackgroundThread.start();
        mBackgroundHandler = new Handler(mBackgroundThread.getLooper());
    }
}

private void stopBackgroundThread() {
    Log.v(TAG, "stopBackgroundThread");
    mBackgroundThread.quitSafely();
    try {
        mBackgroundThread.join();
        mBackgroundThread = null;
        mBackgroundHandler = null;
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
}

private int clamp(int x, int min, int max) {
    if (x > max) return max;
    if (x < min) return min;
    return x;
}

/**
 * Compares two {@code Size}s based on their areas.
 */
static class CompareSizesByArea implements Comparator<Size> {

    @Override
    public int compare(Size lhs, Size rhs) {
        // We cast here to ensure the multiplications won't overflow
        return Long.signum((long) lhs.getWidth() * lhs.getHeight() -
                (long) rhs.getWidth() * rhs.getHeight());
    }
}

}

public class Camera2TextureView extends TextureView {
public static CameraXPreviewHelper cameraHelper;
private static final String TAG = "CameraTextureView";
private Camera2Proxy mCameraProxy;
private SurfaceTexture mSurfaceTexture;
private int mRatioWidth = 0;
private int mRatioHeight = 0;
private float mOldDistance;

public Camera2TextureView(Context context) {
    this(context, null);
}

public Camera2TextureView(Context context, AttributeSet attrs) {
    this(context, attrs, 0);
}

public Camera2TextureView(Context context, AttributeSet attrs, int defStyleAttr) {
    this(context, attrs, defStyleAttr, 0);
}

public Camera2TextureView(Context context, AttributeSet attrs, int defStyleAttr, int defStyleRes) {
    super(context, attrs, defStyleAttr, defStyleRes);
    init(context);
}

private void init(Context context) {
    setSurfaceTextureListener(mSurfaceTextureListener);
    mCameraProxy = new Camera2Proxy((Activity) context);
}

private SurfaceTextureListener mSurfaceTextureListener = new SurfaceTextureListener() {
    @Override
    public void onSurfaceTextureAvailable(SurfaceTexture surface, int width, int height) {
        Log.v(TAG, "onSurfaceTextureAvailable. width: " + width + ", height: " + height);
        mSurfaceTexture=surface;
        mCameraProxy.setPreviewSurface(mSurfaceTexture);
    }

    @Override
    public void onSurfaceTextureSizeChanged(SurfaceTexture surface, int width, int height) {
        Log.v(TAG, "onSurfaceTextureSizeChanged. width: " + width + ", height: " + height);
    }

    @Override
    public boolean onSurfaceTextureDestroyed(SurfaceTexture surface) {
        Log.v(TAG, "onSurfaceTextureDestroyed");
        mCameraProxy.releaseCamera();
        return false;
    }

    @Override
    public void onSurfaceTextureUpdated(SurfaceTexture surface) {
    }
};

private void setAspectRatio(int width, int height) {
    if (width < 0 || height < 0) {
        throw new IllegalArgumentException("Size cannot be negative.");
    }
    mRatioWidth = width;
    mRatioHeight = height;
    requestLayout();
}

public Camera2Proxy getCameraProxy() {
    return mCameraProxy;
}
public SurfaceTexture getSurfaceTexture() {
    return mSurfaceTexture;
}
@Override
protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec) {
    super.onMeasure(widthMeasureSpec, heightMeasureSpec);
    int width = MeasureSpec.getSize(widthMeasureSpec);
    int height = MeasureSpec.getSize(heightMeasureSpec);
    if (0 == mRatioWidth || 0 == mRatioHeight) {
        setMeasuredDimension(width, height);
    } else {
        if (width < height * mRatioWidth / mRatioHeight) {
            setMeasuredDimension(width, width * mRatioHeight / mRatioWidth);
        } else {
            setMeasuredDimension(height * mRatioWidth / mRatioHeight, height);
        }
    }
}

@Override
public boolean onTouchEvent(MotionEvent event) {
    if (event.getPointerCount() == 1) {
        mCameraProxy.focusOnPoint((int) event.getX(), (int) event.getY(), getWidth(), getHeight());
        return true;
    }
    switch (event.getAction() & MotionEvent.ACTION_MASK) {
        case MotionEvent.ACTION_POINTER_DOWN:
            mOldDistance = getFingerSpacing(event);
            break;
        case MotionEvent.ACTION_MOVE:
            float newDistance = getFingerSpacing(event);
            if (newDistance > mOldDistance) {
                mCameraProxy.handleZoom(true);
            } else if (newDistance < mOldDistance) {
                mCameraProxy.handleZoom(false);
            }
            mOldDistance = newDistance;
            break;
        default:
            break;
    }
    return super.onTouchEvent(event);
}

private static float getFingerSpacing(MotionEvent event) {
    float x = event.getX(0) - event.getX(1);
    float y = event.getY(0) - event.getY(1);
    return (float) Math.sqrt(x * x + y * y);
}

}

public class MainActivity extends AppCompatActivity {
private static final String TAG = "MainActivity";

private ScaleGestureDetector mScaleGestureDetector = null;
public static CameraCharacteristics cameraCharacteristics;

private static final String BINARY_GRAPH_NAME = "handtrackinggpu.binarypb";
private static final String INPUT_VIDEO_STREAM_NAME = "input_video";
private static final String OUTPUT_VIDEO_STREAM_NAME = "output_video";
private static final String OUTPUT_HAND_PRESENCE_STREAM_NAME = "hand_presence";
private static final String OUTPUT_LANDMARKS_STREAM_NAME = "hand_landmarks";
public static final CameraHelper.CameraFacing CAMERA_FACING = CameraHelper.CameraFacing.FRONT;

// Flips the camera-preview frames vertically before sending them into FrameProcessor to be
// processed in a MediaPipe graph, and flips the processed frames back when they are displayed.
// This is needed because OpenGL represents images assuming the image origin is at the bottom-left
// corner, whereas MediaPipe in general assumes the image origin is at top-left.
private static final boolean FLIP_FRAMES_VERTICALLY = true;

static {
// Load all native libraries needed by the app.
System.loadLibrary("mediapipe_jni");
System.loadLibrary("opencv_java3");
}

// {@link SurfaceTexture} where the camera-preview frames can be accessed.
private SurfaceTexture previewFrameTexture;
// {@link SurfaceView} that displays the camera-preview frames processed by a MediaPipe graph.
private SurfaceView previewDisplayView;

// Creates and manages an {@link EGLContext}.
private EglManager eglManager;
// Sends camera-preview frames into a MediaPipe graph for processing, and displays the processed
// frames onto a {@link Surface}.
private FrameProcessor processor;
// Converts the GL_TEXTURE_EXTERNAL_OES texture from Android camera into a regular texture to be
// consumed by {@link FrameProcessor} and the underlying MediaPipe graph.
private ExternalTextureConverter converter;

// Handles camera access via the {@link CameraX} Jetpack support library.
public static CameraXPreviewHelper cameraHelper;

private Camera2TextureView mCameraView;

private Camera2Proxy mCameraProxy;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
previewDisplayView = new SurfaceView(this);
setupPreviewDisplayView();
initView();
// Initialize asset manager so that MediaPipe native libraries can access the app assets, e.g.,
// binary graphs.
AndroidAssetUtil.initializeNativeAssetManager(this);

eglManager = new EglManager(null);
processor =
        new FrameProcessor(
                this,
                eglManager.getNativeContext(),
                BINARY_GRAPH_NAME,
                INPUT_VIDEO_STREAM_NAME,
                OUTPUT_VIDEO_STREAM_NAME);
processor.getVideoSurfaceOutput().setFlipY(FLIP_FRAMES_VERTICALLY);

processor.addPacketCallback(
        OUTPUT_HAND_PRESENCE_STREAM_NAME,
        (packet) -> {
          Boolean handPresence = PacketGetter.getBool(packet);
          if (!handPresence) {
            Log.d(
                    TAG,
                    "[TS:" + packet.getTimestamp() + "] Hand presence is false, no hands detected.");
          }
        });

processor.addPacketCallback(
        OUTPUT_LANDMARKS_STREAM_NAME,
        (packet) -> {
          byte[] landmarksRaw = PacketGetter.getProtoBytes(packet);
          try {
            NormalizedLandmarkList landmarks = NormalizedLandmarkList.parseFrom(landmarksRaw);
            if (landmarks == null) {
              Log.d(TAG, "[TS:" + packet.getTimestamp() + "] No hand landmarks.");
              return;
            }
            // Note: If hand_presence is false, these landmarks are useless.
            Log.d(
                    TAG,
                    "[TS:"
                            + packet.getTimestamp()
                            + "] #Landmarks for hand: "
                            + landmarks.getLandmarkCount());
            Log.d(TAG, getLandmarksDebugString(landmarks));
          } catch (InvalidProtocolBufferException e) {
            Log.e(TAG, "Couldn't Exception received - " + e);
            return;
          }
        });

PermissionHelper.checkAndRequestCameraPermissions(this);

}

@Override
protected void onResume() {
super.onResume();
converter = new ExternalTextureConverter(eglManager.getContext());
converter.setFlipY(FLIP_FRAMES_VERTICALLY);
converter.setConsumer(processor);

if (PermissionHelper.cameraPermissionsGranted(this)) {
  startCamera();
}

}

@Override
protected void onPause() {
super.onPause();
converter.close();
}

@Override
public void onRequestPermissionsResult(
int requestCode, String[] permissions, int[] grantResults) {
super.onRequestPermissionsResult(requestCode, permissions, grantResults);
PermissionHelper.onRequestPermissionsResult(requestCode, permissions, grantResults);
}

private void setupPreviewDisplayView() {

previewDisplayView.setVisibility(View.GONE);
ViewGroup viewGroup = findViewById(R.id.frame_layout_rect);
viewGroup.addView(previewDisplayView);
previewDisplayView
        .getHolder()
        .addCallback(
                new SurfaceHolder.Callback() {
                  @Override
                  public void surfaceCreated(SurfaceHolder holder) {
                    processor.getVideoSurfaceOutput().setSurface(holder.getSurface());
                  }

                  @Override
                  public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
                    // (Re-)Compute the ideal size of the camera-preview display (the area that the
                    // camera-preview frames get rendered onto, potentially with scaling and rotation)
                    // based on the size of the SurfaceView that contains the display.
                    Size viewSize = new Size(width, height);
                    Size displaySize = cameraHelper.computeDisplaySizeFromViewSize(viewSize);
                    boolean isCameraRotated = cameraHelper.isCameraRotated();

                    // Connect the converter to the camera-preview frames as its input (via
                    // previewFrameTexture), and configure the output width and height as the computed
                    // display size.
                    converter.setSurfaceTextureAndAttachToGLContext(
                            mCameraView.getSurfaceTexture(),
                            isCameraRotated ? displaySize.getWidth() : displaySize.getHeight(),
                            isCameraRotated ? displaySize.getHeight() : displaySize.getWidth());
                  }

                  @Override
                  public void surfaceDestroyed(SurfaceHolder holder) {
                    processor.getVideoSurfaceOutput().setSurface(null);
                  }
                });

}

private void startCamera() {
cameraHelper = new CameraXPreviewHelper();
cameraHelper.setOnCameraStartedListener(
surfaceTexture -> {
previewFrameTexture = surfaceTexture;
// Make the display view visible to start showing the preview. This triggers the
// SurfaceHolder.Callback added to (the holder of) previewDisplayView.
previewDisplayView.setVisibility(View.VISIBLE);
});
cameraHelper.startCamera(this, CAMERA_FACING, /surfaceTexture=/ mCameraView.getSurfaceTexture());

}

private static String getLandmarksDebugString(NormalizedLandmarkList landmarks) {
int landmarkIndex = 0;
String landmarksString = "";
for (NormalizedLandmark landmark : landmarks.getLandmarkList()) {
landmarksString +=
"\t\tLandmark["
+ landmarkIndex
+ "]: ("
+ landmark.getX()
+ ", "
+ landmark.getY()
+ ", "
+ landmark.getZ()
+ ")\n";
++landmarkIndex;
}
return landmarksString;
}
private void initView() {
mCameraView = findViewById(R.id.camera_view);
mCameraProxy = mCameraView.getCameraProxy();
}
}

xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent">
android:id="@+id/frame_layout_rect"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
tools:ignore="MissingConstraints">
android:id="@+id/camera_view"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:layout_below="@+id/toolbar">

`

[eknight7]

Hi, can you share the code you are using for Camera2? Also, please share the code for using your Camera2 class inside MainActivity.java?

[a798347923]

Hello ,I shared my code and have an another question.If I want to use mediapipe with camera 2.0. I need to use CameraXPreviewHelper.startcamera() or the function of camera2 to open camera .I tried the two way .they all have problem . when I use CameraXPreviewHelper.startcamera()，it display normal but can not relize pinch zoom and other camera2's functions.And when I use the function of camera2 to open camera ,I can not get the output view. The log is always Failed to create capture session; configuration failed. I think the latter is correct,maybe my code is something wrong.But I am not sure ,I want to get an definite answer.

[afsaredrisy]

Hey sorry but your code is too much & difficult to understand please structure your code properly.
Here is Camera2Helper which you can use similarly as you are using provided CameraXHelper
Camera2Helper.java

public class Camera2Helper extends CameraHelper {
    public static final String TAG="Camera2Helper";
    private String cameraId;
    protected CameraDevice cameraDevice;
    protected CameraCaptureSession cameraCaptureSessions;

    protected CaptureRequest.Builder captureRequestBuilder;
    private Size imageDimension;
    private ImageReader imageReader;
    private static final int REQUEST_CAMERA_PERMISSION = 200;

    private Handler mBackgroundHandler;
    private HandlerThread mBackgroundThread;
    private SurfaceTexture outputSurface;
    private Size frameSize;
    private int frameRotation;
    private CameraHelper.CameraFacing cameraFacing;
    private Context context;
    public Camera2Helper(Context context){
        this.context = context;

    }
    public Camera2Helper(Context context, SurfaceTexture surfaceTexture){
        this.context = context;
        this.outputSurface = surfaceTexture;
    }


    @Override
    public void startCamera(Activity context, CameraFacing cameraFacing, @Nullable SurfaceTexture surfaceTexture) {
        this.cameraFacing =cameraFacing;
        closeCamera();
        startBackgroundThread();
        openCamera();

    }

    @Override
    public Size computeDisplaySizeFromViewSize(Size viewSize) {
        if (viewSize == null || frameSize == null) {
            // Wait for all inputs before setting display size.
            Log.d(TAG, "viewSize or frameSize is null.");
            return null;
        }

        // Valid rotation values are 0, 90, 180 and 270.
        // Frames are rotated relative to the device's "natural" landscape orientation. When in portrait
        // mode, valid rotation values are 90 or 270, and the width/height should be swapped to
        // calculate aspect ratio.
        float frameAspectRatio =
                frameRotation == 90 || frameRotation == 270
                        ? frameSize.getHeight() / (float) frameSize.getWidth()
                        : frameSize.getWidth() / (float) frameSize.getHeight();

        float viewAspectRatio = viewSize.getWidth() / (float) viewSize.getHeight();

        // Match shortest sides together.
        int scaledWidth;
        int scaledHeight;
        if (frameAspectRatio < viewAspectRatio) {
            scaledWidth = viewSize.getWidth();
            scaledHeight = Math.round(viewSize.getWidth() / frameAspectRatio);
        } else {
            scaledHeight = viewSize.getHeight();
            scaledWidth = Math.round(viewSize.getHeight() * frameAspectRatio);
        }

        return new Size(scaledWidth, scaledHeight);
    }

    final CameraCaptureSession.CaptureCallback captureCallbackListener = new CameraCaptureSession.CaptureCallback() {
        @Override
        public void onCaptureCompleted(CameraCaptureSession session, CaptureRequest request, TotalCaptureResult result) {
            super.onCaptureCompleted(session, request, result);
            Toast.makeText(context, "Saved:" + file, Toast.LENGTH_SHORT).show();
            createCameraPreview();
        }
    };

    private void closeCamera() {
        try{
            stopBackgroundThread();
            Log.d(TAG,"Closing camera ");
            if (null != cameraDevice) {
                cameraDevice.close();
                cameraDevice = null;
            }
            if (null != imageReader) {
                imageReader.close();
                imageReader = null;
            }
        }catch (Exception e){
            Log.d(TAG,e.toString());
        }

    }

    String new_cam_id;
    private void openCamera() {
        CameraManager manager = (CameraManager) context.getSystemService(Context.CAMERA_SERVICE);
        try {
            if(cameraId == null){
                cameraId = manager.getCameraIdList()[1];
            }
            Log.d(TAG,"CameraListSiize "+manager.getCameraIdList().length);
            if(cameraFacing==CameraFacing.BACK ){
                cameraId = manager.getCameraIdList()[0];
                new_cam_id = manager.getCameraIdList()[0];
                //CameraCharacteristics.LENS_FACING_BACK+"";
                Log.d(TAG,"Opening front cameraa.............................");
            }
            else{
                cameraId = manager.getCameraIdList()[1];//CameraCharacteristics.LENS_FACING_BACK+"";
                Log.d(TAG,"Opening back cameraa.............................");
                new_cam_id = manager.getCameraIdList()[1];
            }
            Log.e(TAG, "is camera open at 93 ");
            CameraCharacteristics characteristics = manager.getCameraCharacteristics(manager.getCameraIdList()[0]);
            StreamConfigurationMap map = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
            assert map != null;
            imageDimension = map.getOutputSizes(SurfaceTexture.class)[0];
            // Add permission for camera and let user grant the permission
            if (ActivityCompat.checkSelfPermission(context, Manifest.permission.CAMERA) != PackageManager.PERMISSION_GRANTED && ActivityCompat.checkSelfPermission(context, Manifest.permission.WRITE_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED) {
                ActivityCompat.requestPermissions((Activity) context, new String[]{Manifest.permission.CAMERA, Manifest.permission.WRITE_EXTERNAL_STORAGE}, REQUEST_CAMERA_PERMISSION);
                Log.d(TAG,"Permission issue");
                return;
            }
            Log.d(TAG,"Opening camera from manager "+cameraId);
            manager.openCamera(manager.getCameraIdList()[0], stateCallback, null);
            //fetchFrame();
            /*if(previewDisplayView!=null){
                previewDisplayView.setVisibility(View.VISIBLE);
            }*/
           /* if(outputTextureView!=null){
                outputTextureView.setVisibility(View.VISIBLE);
                initRecorder();
            }*/
           Log.d(TAG,"Camera debug start 113");

        } catch (CameraAccessException e) {
            e.printStackTrace();
            Log.d(TAG,e.toString());
        }
        Log.e(TAG, "openCamera X");
    }

    private final CameraDevice.StateCallback stateCallback = new CameraDevice.StateCallback() {
        @Override
        public void onOpened(CameraDevice camera) {
            //This is called when the camera is open
            Log.e(TAG, "onOpened");
            cameraDevice = camera;
            createCameraPreview();
            if(onCameraStartedListener!=null){
                onCameraStartedListener.onCameraStarted(outputSurface);
            }
        }

        @Override
        public void onDisconnected(CameraDevice camera) {
            cameraDevice.close();
        }

        @Override
        public void onError(CameraDevice camera, int error) {
            try{
                Log.d(TAG," Error on CameraDevice ");
                cameraDevice.close();
                cameraDevice = null;
            }catch (Exception e){
                Log.d(TAG,"ERROR: "+e.toString()+" ER "+error);
                e.printStackTrace();
            }
        }
    };
    private File file;
    //private CameraCaptureSession cameraCaptureSession;
   // @RequiresApi(api = Build.VERSION_CODES.O)
    protected void createCameraPreview() {
        try {
            Log.d(TAG,"Creating camera preview");
            outputSurface = (outputSurface==null)?new CustomSurfaceTexture(0):outputSurface;

            SurfaceTexture texture = outputSurface;//textureView.getSurfaceTexture();
            assert texture != null;
            texture.setDefaultBufferSize(imageDimension.getWidth(), imageDimension.getHeight());
            frameSize = imageDimension;
            Surface surface = new Surface(texture);
            captureRequestBuilder = cameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
            //captureRequestBuilder.set(CaptureRequest.CONTROL_VIDEO_STABILIZATION_MODE, CameraMetadata.CONTROL_VIDEO_STABILIZATION_MODE_ON);
            captureRequestBuilder.set(CaptureRequest.LENS_OPTICAL_STABILIZATION_MODE,CaptureRequest.LENS_OPTICAL_STABILIZATION_MODE_ON);
            captureRequestBuilder.addTarget(surface);
            cameraDevice.createCaptureSession(Arrays.asList(surface), new CameraCaptureSession.StateCallback() {
                @Override
                public void onConfigured(@NonNull CameraCaptureSession cameraCaptureSession) {
                    //The camera is already closed
                    if (null == cameraDevice) {
                        return;
                    }
                    // When the session is ready, we start displaying the preview.
                    cameraCaptureSessions = cameraCaptureSession;
                    updatePreview();
                }

                @Override
                public void onConfigureFailed(@NonNull CameraCaptureSession cameraCaptureSession) {
                    Toast.makeText(context, "Configuration change", Toast.LENGTH_SHORT).show();
                }
            }, null);
        } catch (CameraAccessException e) {
            e.printStackTrace();
            Log.d(TAG,e.toString());
        }
    }

    protected void updatePreview() {
        if (null == cameraDevice) {
            Log.e(TAG, "updatePreview error, return");
        }
        captureRequestBuilder.set(CaptureRequest.CONTROL_MODE, CameraMetadata.CONTROL_MODE_AUTO);
        try {
            cameraCaptureSessions.setRepeatingRequest(captureRequestBuilder.build(), null, mBackgroundHandler);
        } catch (CameraAccessException e) {
            e.printStackTrace();
        }
    }
    protected void startBackgroundThread() {
        mBackgroundThread = new HandlerThread("Camera Background");
        mBackgroundThread.start();
        mBackgroundHandler = new Handler(mBackgroundThread.getLooper());
    }

    protected void stopBackgroundThread() {
        mBackgroundThread.quitSafely();
        try {
            mBackgroundThread.join();
            mBackgroundThread = null;
            mBackgroundHandler = null;
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

You need SurfaceTexture object which is not attached to glContext to instantiate Camera2Helper for that use following CustomSurfaceTexture.
CustomSurfaceTexture.java

public class CustomSurfaceTexture extends SurfaceTexture {
    public CustomSurfaceTexture(int texName) {
        super(texName);
        init();
    }

    private void init() {
        super.detachFromGLContext();
    }
}

Instantiate Camera2Helper in your main activity like this.

// Inside Main Activity 
CameraHelper cameraHelper;
int textureName = 65; // Declare a TextureName here
  private void startCamera() {
        cameraHelper = new Camera2Helper(this,new CustomSurfaceTexture(textureName));
        cameraHelper.setOnCameraStartedListener(
                surfaceTexture -> {
                    previewFrameTexture = surfaceTexture;
                    // Make the display view visible to start showing the preview. This triggers the
                    // SurfaceHolder.Callback added to (the holder of) previewDisplayView.
                    videoTexture.setVisibility(View.VISIBLE);
                    Log.d(TAG,"Camera Started");
                });
        cameraHelper.startCamera(this, CAMERA_FACING, /*surfaceTexture=*/ null);
    }

Let me know you face any issue while using it.
You can also ask question on stack-overflow with tag mediapipe as its seems not an issue of mediapipe.

[a798347923]

thank you very mach! I will try and change my code.
I tried your suggestion,and deal with my trouble .It useful ,I really feel for you.

[eknight7]

@afsaredrisy thanks for sharing your code and responding to the question.

[mapleyuan]

@afsaredrisy thanks, u save my times! but i got a stretch frame, so i remove the method (computeDisplaySizeFromViewSize), it need to right resize.

[PrinceP]

@mapleyuan For a correct stretch we needed to have frameAspectRatio modified.

float frameAspectRatio = frameRotation == 0 || frameRotation == 180? frameSize.getHeight() / (float) frameSize.getWidth(): frameSize.getWidth() / (float) frameSize.getHeight();
@afsaredrisy Thanks for the help.

@eknight7 @afsaredrisy I was trying to remove the crash if the user pushes the app in the background and opens it back again. We have corrected the cameraHelper crash.
It's now giving following error, Any idea on the changes needed?
2020-04-27 12:00:49.555 27385-27882/com.signum.sigil E/OpenGLRenderer: [SurfaceTexture-65-27385-0] attachToContext: SurfaceTexture is already attached to a context 2020-04-27 12:00:49.556 27385-27882/com.signum.sigil E/AndroidRuntime: FATAL EXCEPTION: ExternalTextureConverter Process: com.signum.sigil, PID: 27385 java.lang.RuntimeException: Error during attachToGLContext (see logcat for details) at android.graphics.SurfaceTexture.attachToGLContext(SurfaceTexture.java:294) at com.google.mediapipe.components.ExternalTextureConverter$RenderThread.setSurfaceTextureAndAttachToGLContext(ExternalTextureConverter.java:197) at com.google.mediapipe.components.ExternalTextureConverter.lambda$setSurfaceTextureAndAttachToGLContext$1$ExternalTextureConverter(ExternalTextureConverter.java:120) at com.google.mediapipe.components.-$$Lambda$ExternalTextureConverter$5RBq2uHoPl06wf2nG84kcgcgP_U.run(Unknown Source:8) at android.os.Handler.handleCallback(Handler.java:883) at android.os.Handler.dispatchMessage(Handler.java:100) at android.os.Looper.loop(Looper.java:214) at com.google.mediapipe.glutil.GlThread.run(GlThread.java:134) 2020-04-27 12:00:49.563 27385-27385/com.signum.sigil E/CameraCaptureSession: Session 0: Exception while stopping repeating: android.hardware.camera2.CameraAccessException: CAMERA_DISCONNECTED (2): checkPidStatus:2039: The camera device has been disconnected at android.hardware.camera2.CameraManager.throwAsPublicException(CameraManager.java:799) at android.hardware.camera2.impl.ICameraDeviceUserWrapper.cancelRequest(ICameraDeviceUserWrapper.java:97) at android.hardware.camera2.impl.CameraDeviceImpl.stopRepeating(CameraDeviceImpl.java:1139) at android.hardware.camera2.impl.CameraCaptureSessionImpl.close(CameraCaptureSessionImpl.java:526) at android.hardware.camera2.impl.CameraCaptureSessionImpl$2.onDisconnected(CameraCaptureSessionImpl.java:737) at android.hardware.camera2.impl.CameraDeviceImpl$7.run(CameraDeviceImpl.java:239) at android.os.Handler.handleCallback(Handler.java:883) at android.os.Handler.dispatchMessage(Handler.java:100) at android.os.Looper.loop(Looper.java:214) at android.app.ActivityThread.main(ActivityThread.java:7356) at java.lang.reflect.Method.invoke(Native Method) at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:930) Caused by: android.os.ServiceSpecificException: checkPidStatus:2039: The camera device has been disconnected (code 4) at android.os.Parcel.createException(Parcel.java:2085) at android.os.Parcel.readException(Parcel.java:2039) at android.os.Parcel.readException(Parcel.java:1987) at android.hardware.camera2.ICameraDeviceUser$Stub$Proxy.cancelRequest(ICameraDeviceUser.java:658) at android.hardware.camera2.impl.ICameraDeviceUserWrapper.cancelRequest(ICameraDeviceUserWrapper.java:95) at android.hardware.camera2.impl.CameraDeviceImpl.stopRepeating(CameraDeviceImpl.java:1139)  at android.hardware.camera2.impl.CameraCaptureSessionImpl.close(CameraCaptureSessionImpl.java:526)  at android.hardware.camera2.impl.CameraCaptureSessionImpl$2.onDisconnected(CameraCaptureSessionImpl.java:737)  at android.hardware.camera2.impl.CameraDeviceImpl$7.run(CameraDeviceImpl.java:239)  at android.os.Handler.handleCallback(Handler.java:883)  at android.os.Handler.dispatchMessage(Handler.java:100)  at android.os.Looper.loop(Looper.java:214)  at android.app.ActivityThread.main(ActivityThread.java:7356)  at java.lang.reflect.Method.invoke(Native Method)  at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492)  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:930) 

[afsaredrisy]

@PrinceP as SurfaceTexture is already attached to a context indicates that some how surfacetexture object is already attached to context but attachToGLContext() is being called on this object. Probably you are reusing the SurfaceTexture object after app resume & calling SurfaceTextureAndAttachToGLContext on External texture converter. Either share your modified code along with onResume() method of Activity (To see the event on app coming back to foreground from background). If you are reusing SurfaceTexture object then try creating new object every time onResume() called.

[PrinceP]

@afsaredrisy

Following is the code.

1. Camera2Helper.java and CustomSurfaceTexture.java are same as comment
2. MainActivity kotlin code

private var cameraHelper: Camera2Helper? = null
var textureName = 65 // Declare a TextureName here
private var previewFrameTexture: CustomSurfaceTexture? = null
private lateinit var previewDisplayView: SurfaceView

override fun onResume() {
super.onResume()
convertor = ExternalTextureConverter(eglManager.context)
convertor!!.setFlipY(FLIP_FRAMES_VERTICALLY)
convertor!!.setConsumer(processor)
if (PermissionHelper.cameraPermissionsGranted(this)) {
startCamera()
}
}

private fun startCamera() {
    cameraHelper = Camera2Helper(this, CustomSurfaceTexture(textureName))
    cameraHelper!!.setOnCameraStartedListener(
        OnCameraStartedListener {
            previewFrameTexture = it  as CustomSurfaceTexture
            // Make the display view visible to start showing the preview. This triggers the
            // SurfaceHolder.Callback added to (the holder of) previewDisplayView.
            previewDisplayView.setVisibility(View.VISIBLE)
            Log.d(TAG, "Camera Started")
        }
    )
    cameraHelper!!.startCamera(this, CAMERA_FACING,  /*surfaceTexture=*/previewFrameTexture)

}

override fun onPause() {
    super.onPause()
    convertor!!.close()
}

private fun setupPreviewDisplayView() {
previewDisplayView.visibility = View.GONE
val container = preview_display_layout
container.addView(previewDisplayView)
previewDisplayView.holder.addCallback(object : SurfaceHolder.Callback {
override fun surfaceChanged(
holder: SurfaceHolder?, format: Int, width: Int,
height: Int
) {
val viewSize = Size(width, height)
Log.d(TAG, "Viewsize ${viewSize.height}")
val displaySize: Size = cameraHelper?.computeDisplaySizeFromViewSize(viewSize) ?: Size(1280,720)
convertor!!.setSurfaceTextureAndAttachToGLContext(
previewFrameTexture, displaySize.width,
displaySize.height
)
}
override fun surfaceDestroyed(holder: SurfaceHolder?) {
Log.d(TAG, "Destroyed surface")
processor.videoSurfaceOutput.setSurface(null)
}

        override fun surfaceCreated(holder: SurfaceHolder?) {
            processor.videoSurfaceOutput.setSurface(holder?.surface)
        }

    })
}

[yxchng]

@afsaredrisy

Why do you need to wrap the SurfaceTexture into a CustomSurfaceTexture that only does super.detachFromGLContext()? Is this strictly necessary?

public class CustomSurfaceTexture extends SurfaceTexture {
    public CustomSurfaceTexture(int texName) {
        super(texName);
        init();
    }

    private void init() {
        super.detachFromGLContext();
    }
}

[yxchng]

@afsaredrisy is the sample code working? it seems to be missing frameRotation? how to get it?

[afsaredrisy]

@yxchng It will work fine for portrait mode if you are changing the orientation then you can get the frameRotation with getWindowManager().getDefaultDisplay().getRotation(). Set the frameRotation value from activity and use the following orientation (May be you can use setter method or can pass getWindowManager().getDefaultDisplay().getRotation() in Camera2Helper constructor)

private static final SparseIntArray ORIENTATIONS = new SparseIntArray();

    static {
        ORIENTATIONS.append(Surface.ROTATION_0, 90);
        ORIENTATIONS.append(Surface.ROTATION_90, 0);
        ORIENTATIONS.append(Surface.ROTATION_180, 270);
        ORIENTATIONS.append(Surface.ROTATION_270, 180);
    }

In updatePreview() method of Camera2Helper add this line:

captureRequestBuilder.set(CaptureRequest.JPEG_ORIENTATION, ORIENTATIONS.get(frameRotation));

Hopefully it will work. if not please have look at official google Camera 2 Demo

For first question. see-this-stackoverflow. We need to use this surfaceTexture to render its EXT_Texture to frameBuffer for Mediapipe AppTextureFrame for that we need to detach from current gl context and attach ExternalTextureRenderer context.

[yxchng]

@afsaredrisy

I am getting this error following your code and suggestion:

ERROR: /mnt/sdd1/fuck_mediapipe_mobile/mediapipe/mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectioncpu/BUILD:76:1: Merging manifest for //mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectioncpu:facedetectioncpu failed (Exit 1) ResourceProcessorBusyBox failed: error executing command bazel-out/host/bin/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/ResourceProcessorBusyBox --tool MERGE_MANIFEST -- --manifest ... (remaining 11 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
Error: /home/abc/.cache/bazel/_bazel_abc/db07d7e613626d40c502a5051f1b083f/sandbox/linux-sandbox/565/execroot/mediapipe/mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectioncpu/AndroidManifest.xml:19:18-86 Error:
    Attribute application@appComponentFactory value=(androidx.core.app.CoreComponentFactory) from [@maven//:androidx_core_core] AndroidManifest.xml:19:18-86
    is also present at [@maven//:com_android_support_support_compat] AndroidManifest.xml:19:18-91 value=(android.support.v4.app.CoreComponentFactory).
    Suggestion: add 'tools:replace="android:appComponentFactory"' to <application> element at AndroidManifest.xml:17:3-31:17 to override.
May 14, 2020 10:49:17 PM com.google.devtools.build.android.ManifestMergerAction main
SEVERE: Error during merging manifests
com.google.devtools.build.android.AndroidManifestProcessor$ManifestProcessingException: Manifest merger failed : Attribute application@appComponentFactory value=(androidx.core.app.CoreComponentFactory) from [@maven//:androidx_core_core] AndroidManifest.xml:19:18-86
    is also present at [@maven//:com_android_support_support_compat] AndroidManifest.xml:19:18-91 value=(android.support.v4.app.CoreComponentFactory).
    Suggestion: add 'tools:replace="android:appComponentFactory"' to <application> element at AndroidManifest.xml:17:3-31:17 to override.
    at com.google.devtools.build.android.AndroidManifestProcessor.mergeManifest(AndroidManifestProcessor.java:183)
    at com.google.devtools.build.android.ManifestMergerAction.main(ManifestMergerAction.java:217)
    at com.google.devtools.build.android.ResourceProcessorBusyBox$Tool$9.call(ResourceProcessorBusyBox.java:114)
    at com.google.devtools.build.android.ResourceProcessorBusyBox.processRequest(ResourceProcessorBusyBox.java:237)
    at com.google.devtools.build.android.ResourceProcessorBusyBox.main(ResourceProcessorBusyBox.java:198)

Warning: 
See http://g.co/androidstudio/manifest-merger for more information about the manifest merger.
Target //mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectioncpu:facedetectioncpu failed to build

It doesn't seem to be compatible with the original MediaPipe code.

Do you have a working fork?

[afsaredrisy]

I am using it segmentation sample & its working even with latest Mediapipe, can you match your startCamera() method in MainActivity.

private void startCamera() {
        cameraHelper = new Camera2Helper(this);
        cameraHelper.setOnCameraStartedListener(
                surfaceTexture -> {
                    previewFrameTexture = surfaceTexture;
                    // Make the display view visible to start showing the preview. This triggers the
                    // SurfaceHolder.Callback added to (the holder of) previewDisplayView.
                    previewDisplayView.setVisibility(View.VISIBLE);
                });
        cameraHelper.startCamera(this, CAMERA_FACING, /*surfaceTexture=*/ new CustomSurfaceTexture(34));
    }

[yxchng]

@afsaredrisy Do you mind sharing all the files you modify? What are the imports used? Also, your code certainly need to modify files like BUILD and WORKSPACE to include Camera2 and remove CameraX. Without these information, the code is not runnable.

Do you remove all CameraX imports across all files?

[yxchng]

@afsaredrisy Also your code seems to have smaller field of view from Camera 2 Demo. How to enlarge the FOV?