高通 camera基于HAL1和hal3的说明
Summary
libcamera provides C++ API to interface with camera on multiple Linux based Snapdragon platform (APQ8074,APQ8096,APQ8009) used in drones and robotics application.
Currently libcamera library provides APIs abstracting two different frameworks
- Camera HAL1 :
APQ8074andAPQ8009based platform use this implementation.
Note: libcamera implementation forAPQ8009is present in this repository whereas implementation for 8074 is present at : https://source.codeaurora.org/quic/la/platform/hardware/qcom/camera/tree/libcamera?h=LNX.LER.1.2
- Camera HAL3 using QMMF-SDK :
APQ8096based platforms use this implementation.
The core framework and API remain the same on these platforms with slight differences in functionality provided due by underlying framework differences. There are also some improvisation in APIs. This document should help in migration to different platform.
Application notes
==================
APIs overview
There are two set of APIs. The first set is for one application to be the master of the camera. The second set is for other applications to access
the camera frame streams that were brought up by another master application.
APIs for the master application
The APIs are listed in header file
libcamera/inc/camera.h, mainly the following three interfaces: - camera::ICameraDevice to open/close a camera to start/stop preview/video streams, and to take a snapshot.
- camera::ICameraParameters to configure the camera.
- camera::ICameraListener and camera::ICameraFrame to access the camera frame content.
Call Flow Sequence diagram
Sample Code and Test Application
libcamera/test/camera-test.cpp
APIs for the camera subscriber applications
Once the camera master application starts the camera streams, other applications running in different processses may access the frame contenct
(readonly).
The APIs are listed in header filelibcamera/inc/camera_subscriber.handlibcamera/inc/camera_subscriber_meta_data.h- camera::ICameraSubscriber to subscribed to the frames published by the master application
- camera::ICameraSubsriberListener and ICameraSubsriberFrame to access the camera frame content. The content includes frame bits and meta data.
- camera::tFrameMetaData defines the meta data that the subscribe application can access
Call Flow Sequence diagram
Sample Code and Test Application
libcamera/test/camera-subscriber-test.cpp
Library configuration via system property setting ( for
APQ8096andAPQ8009)Using
setprop/getpropcan configure the library behavior.- libcam.enable.publish 0: disable the frame pulishing/subscribing to other applications. 1 (default): enable
- libcam.enable.publish.dump 1: dump the published frame meta data. 0(default): do not dump
- libcam.publish.buffer.policy max(default): the publisher caches frames as long as the buffer is not full. min: the publisher returns frame ASAP
- libcam.publish.buffer.max default is 3 frames
- libcam.debug.level silent, error, info(default), debug
- libcam.dump.dir where to put dump files. default:
/data/misc/camera/dumps
example:setprop libcam.enable.publish.dump 1getprop libcam.enable.publish.dump
Notes for porting applications from
APQ8074toAPQ8096 - New set of API for getting per frames information such exposure time, gain, readout time, readout timestamp are added.
- On HAL1, the timestamp in frames reported in preview and video callback was at start of readout. In HAL3 implementation timestamp reported is start of exposure of first line of the frame.
- Currently function to camera id function mapping is not working. Hence we need to use the hardcoded camera-id for camera. Note: the camera-id is assigned based on the order in which it was initialized during boot up. Hence if one sensor is disconnected the camera-id of the sensor can change.
- Burst snapshot is enabled
- HAL3 implementation has cleaner API's to set preview frame and Snapshot formats
- On 8996, only combined stereo frames can be obtained. individual stereo frames cannot be obtained.
- The range for manualExposure and manualGain is changed.
- For ManualExposure, on HAL1 based implementation, the function parameters values correspond to values directly written to sensor. In HAL3, the values is in unit of time (nS)
- The min/max exposure and gain can be obtained using functionsgetManualExposureRangeandgetManualGainRange. - Acquire/Release Ref: There is a hard requirement to release all pendng frames (frames on which acquire ref is done) before calling stopPreivew or stopRecording function. Note: startPreview or startRecording also requires release ref of all pending frames.
- The following callbacks are not yet enabled.
-onControl: This gave information such as status of Focus operation, etc.
-onMetadataFrame: This gave information such as bounding box for face detect, etc. - The following feature are not yet supported or verified.
- HDR
- Hardware face detect
- ZSL
- mobicat
- SW jpeg encoding for Snapshot
- Manual focus. Hence snapshot eg does not contain setting focus.
### New APIs in HAL3 Based Implementation
c++<br /> void setPictureFormat(const std::string& value);<br /> Range64 getManualExposureRange(ImageSize size, int fps);<br /> Range getManualGainRange(ImageSize size, int fps);<br /> uint64_t getFrameExposureTime(ICameraFrame* frame);<br /> int32_t getFrameGainValue(ICameraFrame* frame);<br /> uint64_t getFrameRollingShutterSkew(ICameraFrame* frame);<br /> uint64_t getFrameReadoutTimestamp(ICameraFrame* frame);<br /> uint64_t getFrameReadoutDuration(ICameraFrame* frame);<br />
### Changes in existing APIs
-virtual int takePicture() = 0;is changed tovirtual int takePicture(uint32_t num_images = 1) = 0;
Note: This does not affect application calling takePicture since it uses default value of num_images. But it will affect applications that create classes which inherits ICameraDevice.
-std::vector<ImageSize> getSupportedPictureSizes() const;is changed tostd::vector<ImageSize> getSupportedPictureSizes(std::string format = FORMAT_JPEG) const;
Note: You can query supported sizes based on image format. This was needed because in HAL3 based implementation, the supported resolution for jpeg and raw format are different for some sensors.
- get/set sharpness APIssetSharpnessnow changed. User first need to set sharpness modesetSharpnessModeand then set sharpness value usingsetSharpnessAPI.
-setToneMapModeandsetContrastTonemust be used instead ofsetContrastto set contrast.
### API not yet implemented
c++<br /> virtual int startAutoFocus() = 0;<br /> virtual void stopAutoFocus() = 0;<br />
### API's not available
c++<br /> int CameraParams::writeObject(std::ostream& ps) const;<br /> string CameraParams::get(const string& key) const;<br /> void CameraParams::set(const string& key, const string& value);<br /> Range CameraParams::getSupportedBrightness() const;<br /> int CameraParams::getBrightness() const;<br /> void CameraParams::setBrightness(int value);<br /> Range CameraParams::getSupportedContrast() const;<br /> int CameraParams::getContrast() const;<br /> void CameraParams::setContrast(int value);<br /> string CameraParams::toString() const;<br /> void CameraParams::setVerticalFlip(bool value);<br /> void CameraParams::setHorizontalMirror(bool value);<br /> void CameraParams::setStatsLoggingMask(int value);<br />
Only skeleton exist for the following API no implementation yet
c++<br /> void CameraParams::setAntibanding(const string& value)<br />
FAQs
How to map camera-id to function id?
Each camera is given a unique function id in the sensor driver.:
HIRES = 0
LEFT SENSOR = 1
TRACKING SENSOR = 2
RIGHT SENSOR = 3
STEREO = 4
getNumberOfCameras gives information on number of camera connected on target.
getCameraInfo provides information on each camera loop.
* camid is obtained by looping through available cameras and matching info.func
with the requested camera.
How to perform vertical flip and horizontal mirror on individual images in stereo ?
In stereo since the image is merged. It makes it harder to perform these operation on individual images which may be required based on senor orientation on target.setVerticalFlipandsetHorizontalMirrorperform these operation by changing the output configuration from the sensor.
Note: only available onAPQ8074
How to set FPS ?
Preview fps is set using the function :setPreviewFpsRange. Video fps is set using the function :setVideoFPS.setFPSindexscans through the supported fps values and returns index of requested fps in the array of supported fps.
How to change the format to NV12 ?
On Camera HAL1 based implementation, default preview stream YUV format is NV21. To change the format to NV12 use the "preview-format" key. params.set(std::string("preview-format"), std::string("nv12"));<br /> ```<br /> On HAL3 only nv12-venus is availble amoung YUV formats.<br /> <br /> __How to enable RAW mode for tracking camera sensor ?__<br /> - RAW stream is only available for OV7251
- RAW stream currently returns images in the preview callback.
- When configuring RAW stream, video stream on the same sensor must not be enabled. Else you will not see preview callbacks.
- When configuration RAW, these parameters must be set in addition to other parameters for tracking camera
* params_.set("preview-format", "bayer-rggb");<br /> * params_.set("picture-format", "bayer-mipi-10gbrg");<br /> * params_.set("raw-size", "640x480");<br /> ```<br /> <br /> <br /> <br /> Known Limitations<br /> =================<br /> <br /> ### ```AQP8096``` based platforms<br /> - The following callbacks are not yet enabled.<br /> - ```onControl``` : This gave information such as status of Focus operation, etc.<br /> - ```onMetadataFrame``` : This gave information such as bounding box for face detect, etc.<br /> - The following feature are not yet supported or verified.<br /> - HDR<br /> - Hardware face detect<br /> - ZSL<br /> - mobicat<br /> - SW jpeg encoding for Snapshot<br /> - Manual focus. Hence snapshot eg does not contain setting focus.<br /> <br /> ### ```AQP8074``` based platforms<br /> - left/right Naming:
code is written with reference to schematic but for end users the left and right sensor appears swapped.
so for user experience in the current app left option is changed to right.
right sensor with reference to schematic always goes in stereo mode, so no left option for end users. - Snapshot is not supported for tracking camera.
- The following parameters can be set only after starting preview :
-Manual exposure
-Manual gain
-setVerticalFlip
-setHorizontalMirror - Do not set the following parameters for tracking and stereo cameras
- PictureSize
- focus mode
- white balance
- ISO
- preview format
libcamera提供C ++ API，用于在无人机和机器人应用程序中使用的多个基于Linux的Snapdragon平台（```APQ8074```，```APQ8096```，````APQ8009```）上与摄像头连接。
目前，libcamera库提供了抽象两个不同框架的API
- [Camera HAL1]（https://source.android.com/devices/camera/）：```APQ8074```和```APQ8009```平台使用此实现。
注意：此库中存在“`APQ8009```的libcamera实现，而8074的实现存在于：https：//source.codeaurora.org/quic/la/platform/hardware/qcom/camera/tree/libcamera ？H = LNX.LER.1.2
- [使用QMMF-SDK的Camera HAL3]（https://source.android.com/devices/camera/camera3）：基于``APQ8096```的平台使用此实现。
核心框架和API在这些平台上保持不变，由于底层框架差异而提供的功能略有不同。 API中也有一些即兴创作。本文档应有助于迁移到不同的平台。
应用笔记
==================
## API概述
有两组API。第一组是一个应用程序的主摄像头。第二组用于访问由另一个主应用程序启动的相机帧流的其他应用程序。
###主应用程序的API
API列在头文件```libcamera / inc / camera.h```中，主要有以下三个接口：
- camera :: ICameraDevice打开/关闭相机以启动/停止预览/视频流，并拍摄快照。
- camera :: ICameraParameters配置摄像头。
- camera :: ICameraListener和camera :: ICameraFrame访问相机帧内容。
####调用流程序列图
！[alt text]（documentation / libcamera / camera-sequence-diagram.jpg）
####示例代码和测试应用程序
- ```libcamera / test / camera-test.cpp```
###相机订阅者应用程序的API
一旦摄像机主应用程序启动摄像机流，在不同进程中运行的其他应用程序可以访问帧协议
（只读）。
API列在头文件```libcamera / inc / camera_subscriber.h```和```libcamera / inc / camera_subscriber_meta_data.h```中
- camera :: ICameraSubscriber订阅主应用程序发布的帧
- camera :: ICameraSubsriberListener和ICameraSubsriberFrame访问相机帧内容。内容包括帧位和元数据。
- camera :: tFrameMetaData定义订阅应用程序可以访问的元数据
##将应用程序从```APQ8074```移植到```APQ8096```的注意事项
- 用于获取每帧信息的新API集合，例如曝光时间，增益，读出时间，读出时间戳。
- 在HAL1上，预览和视频回调中报告的帧中的时间戳是在读数开始时。在HAL3实现中，报告的时间戳是帧的第一行的曝光开始。
- 目前功能到摄像机ID功能映射不起作用。因此我们需要使用硬编码的摄像机ID作为摄像机。注意：camera-id是根据启动期间初始化的顺序分配的。因此，如果一个传感器断开，传感器的摄像机ID可能会改变。
- 启用了突发快照
- HAL3实现具有更清晰的API来设置预览帧和快照格式
- 在8996，只能获得组合的立体帧。无法获得单个立体帧。
- 手动曝光和手动增益的范围已更改。
- 对于ManualExposure，在基于HAL1的实现中，函数参数值对应于直接写入传感器的值。在HAL3中，值以时间为单位（nS）
- 使用函数```getManualExposureRange```和```getManualGainRange```可以获得最小/最大曝光和增益。
- 获取/释放参考：在调用stopPreivew或stopRecording函数之前，很难释放所有pendng帧（获取ref的帧）。注意：startPreview或startRecording还需要所有挂起帧的版本ref。
- 尚未启用以下回调。
- ```onControl```：这给出了Focus操作状态等信息。
- ```onMetadataFrame```：这提供了诸如面部检测的边界框等信息。
- 尚不支持或验证以下功能。
- HDR
- 硬件面部检测
- ZSL
- mobicat
- 用于快照的SW jpeg编码
- 手动对焦。因此，快照例如不包含设置焦点。
###基于HAL3的实现中的新API
```C ++
void setPictureFormat（const std :: string＆value）;
Range64 getManualExposureRange（ImageSize size，int fps）;
范围getManualGainRange（ImageSize size，int fps）;
uint64_t getFrameExposureTime（ICameraFrame * frame）;
int32_t getFrameGainValue（ICameraFrame * frame）;
uint64_t getFrameRollingShutterSkew（ICameraFrame * frame）;
uint64_t getFrameReadoutTimestamp（ICameraFrame * frame）;
uint64_t getFrameReadoutDuration（ICameraFrame * frame）;
```
###现有API的变化
- ```virtual int takePicture（）= 0;```改为```virtual int takePicture（uint32_t num_images = 1）= 0;```
注意：这不会影响应用程序调用takePicture，因为它使用默认值num_images。但它会影响创建继承ICameraDevice的类的应用程序。
- ```std :: vector <ImageSize> getSupportedPictureSizes（）const;```改为```std :: vector <ImageSize> getSupportedPictureSizes（std :: string format = FORMAT_JPEG）const;```
注意：您可以根据图像格式查询支持的尺寸。这是必需的，因为在基于HAL3的实现中，对于某些传感器，支持的jpeg和原始格式的分辨率是不同的。
- 获取/设置锐度API```setSharpness```现在已更改。用户首先需要设置锐度模式```setSharpnessMode```然后使用```setSharpness``` API设置锐度值。
- 必须使用```setToneMapMode```和```setContrastTone```代替```setContrast```来设置对比度。
### API尚未实现
```C ++
virtual int startAutoFocus（）= 0;
virtual void stopAutoFocus（）= 0;
```
### API不可用
```C ++
int CameraParams :: writeObject（std :: ostream＆ps）const;
string CameraParams :: get（const string＆key）const;
void CameraParams :: set（const string＆key，const string＆value）;
范围CameraParams :: getSupportedBrightness（）const;
int CameraParams :: getBrightness（）const;
void CameraParams :: setBrightness（int value）;
范围CameraParams :: getSupportedContrast（）const;
int CameraParams :: getContrast（）const;
void CameraParams :: setContrast（int value）;
string CameraParams :: toString（）const;
void CameraParams :: setVerticalFlip（bool value）;
void CameraParams :: setHorizontalMirror（bool value）;
void CameraParams :: setStatsLoggingMask（int value）;
```
以下API仅存在骨架，尚未实现
```C ++
void CameraParams :: setAntibanding（const string＆value）
```
#FAQs
__如何将camera-id映射到功能id？__
*每个摄像机在传感器驱动程序中都有一个唯一的功能ID：
* HIRES = 0
* LEFT SENSOR = 1
*跟踪传感器= 2
*右传感器= 3
* STEREO = 4
- getNumberOfCameras提供有关目标上连接的摄像机数量的信息。
- getCameraInfo提供有关每个摄像机循环的信息。
- camid是通过循环浏览可用的摄像头并匹配info.func获得的
与请求的相机。
如何在立体声中的单个图像上执行垂直翻转和水平镜像？
在立体声中，因为图像被合并。这使得对基于目标上的传感器取向可能需要的单个图像执行这些操作变得更加困难。setVerticalFlip和setHorizontalMirror通过改变传感器的输出配置来执行这些操作。
注意：仅适用于APQ8074
如何设置FPS？
预览fps使用以下函数设置：setPreviewFpsRange。视频fps使用以下函数设置：setVideoFPS。setFPSindex扫描支持的fps值，并返回支持的fps数组中所请求的fps的索引。
如何将格式更改为NV12？
在基于Camera HAL1的实现上，默认预览流YUV格式是NV21。要将格式更改为NV12，请使用“预览格式”键。
params.set（std :: string（“preview-format”），std :: string（“nv12”））;<br /> ```<br /> 在HAL3上，只有nv12-venus可用于YUV格式。<br /> <br /> __如何启用RAW模式跟踪相机传感器？__<br />- RAW流仅适用于OV7251
- RAW流当前返回预览回调中的图像。
配置RAW流时，不得启用同一传感器上的视频流。否则你将看不到预览回调。
配置RAW时，除了跟踪摄像机的其他参数外，还必须设置这些参数
* params_.set（“preview-format”，“bayer-rggb”）;<br /> * params_.set（“picture-format”，“bayer-mipi-10gbrg”）;<br /> * params_.set（“raw-size”，“640x480”）;<br /> ```<br /> <br /> <br /> <br /> 已知限制<br /> =================<br /> <br /> ###```基于AQP8096```的平台<br /> - 尚未启用以下回调。<br /> - ```onControl```：这给出了Focus操作状态等信息。<br /> - ```onMetadataFrame```：这提供了诸如面部检测的边界框等信息。<br /> - 尚不支持或验证以下功能。<br /> - HDR<br /> - 硬件面部检测<br /> - ZSL<br /> - mobicat<br /> - 用于快照的SW jpeg编码<br /> - 手动对焦。因此，快照例如不包含设置焦点。<br /> <br /> ###```基于AQP8074```的平台<br /> - __left / right命名：__<br /> 代码是参考原理图编写的，但对于最终用户，左右传感器显示为交换。<br /> 因此，对于当前应用程序中的用户体验，左侧选项更改为右侧。<br /> 右侧传感器参考原理图始终进入立体声模式，因此最终用户没有左侧选项。<br /> - 跟踪相机不支持快照。<br /> - 只有在开始预览后才能设置以下参数：<br /> - ```手动曝光```<br /> - ```手动增益```<br /> - ```setVerticalFlip```<br /> - ```setHorizontalMirror```<br /> - 请勿为跟踪和立体摄像机设置以下参数<br /> - PictureSize<br /> - 对焦模式<br /> - 白平衡<br /> - ISO<br /> - 预览格式<br /> <br /> </div>分类文章
- getNumberOfCameras提供有关目标上连接的摄像机数量的信息。
