overview.rst 5.5 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124
  1. Overview
  2. ========
  3. Libmegapixels is a library for accessing V4L2 cameras in Linux. It's the camera
  4. set-up code originally written for Megapixels but split off and improved as a
  5. library to make it easier to debug and integrate.
  6. The way you'd normally open a webcam in Linux and capture frames is roughly:
  7. - Open a :code:`/dev/video*` file descriptor
  8. - Run the :code:`VIDIOC_QUERYCAP` ioctl to check the device has the :code:`V4L2_CAP_VIDEO_CAPTURE` flag and check which methods of capturing frames are valid.
  9. - Run the :code:`VIDIOC_S_FMT` and :code:`VIDIOC_G_FMT` ioctls a few times to narrow down a format to use.
  10. - Set-up mmapped video capture on the device and run :code:`VIDIOC_STREAMON` to start the capture
  11. - Get frames from the kernel
  12. This works for UVC webcams you usually encounter on Linux devices since these are
  13. relatively simple devices.
  14. If you try to use the camera on a modern phone the whole workflow is completely different.
  15. Instead of dealing with just a video device you also have to deal with
  16. a :code:`/dev/media*` device for setting up hardware pipelines and then deal with
  17. various :code:`/dev/v4l-subdev*` devices to configure all the nodes in that media pipeline.
  18. One (or more) of the nodes in that media pipeline will be the regular old :code:`/dev/video*`
  19. device again that will provide your application with the frames, but due to the way
  20. the pipelines work a lot of the normal auto-configuration things applications do no longer work.
  21. In the new pipelines the video device only deals with getting the video frames into
  22. userspace. Due to this the video device does not actually know what formats and
  23. resolutions are valid so using the old ioctls to query this information is useless.
  24. One of the v4l-subdev devices will be representing the sensor in the device, this does know what
  25. modes are available but to know that mode will work you need to make sure all the nodes in the
  26. pipeline can run at that mode and all these nodes need to be manually configured. But in
  27. reality it's even more complicated becaues this only describes what resolutions and modes
  28. the drivers for these components support and it does not account for limitations in
  29. the actual hardware like not all the MIPI lanes being connected between the sensor
  30. and the SoC limiting the possible bandwidth, or even more basic bandwidth limitations
  31. due to the length of PCB traces.
  32. Using libmegapixels
  33. -------------------
  34. The solution for this mess in Megapixels which is now transferred to a library
  35. is using config files that provide the pipeline information. This hardcodes a
  36. series of known working modes and also provides the metadata for producing
  37. high quality pictuers from the data coming from the pipeline.
  38. The general use of libmegapixels is this:
  39. .. code-block:: c
  40. #include <libmegapixels.h>
  41. // Create the empty device config object
  42. libmegapixels_devconfig *config = {0};
  43. libmegapixels_init(&config);
  44. // Find the config file path for this device
  45. char configpath[PATH_MAX];
  46. int ret = libmegapixels_find_config(configpath);
  47. // Load the config
  48. libmegapixels_load_file(config, configpath);
  49. // Additionally load UVC cameras with autodetection
  50. libmegapixels_load_uvc(config);
  51. printf("Found %d cameras\n", config->count);
  52. // Open the first camera
  53. libmegapixels_camera *camera = config->cameras[camera_id];
  54. libmegapixels_open(camera);
  55. printf("Camera has %d modes\n", camera->num_modes);
  56. // Select the first mode
  57. libmegapixels_mode *mode = camera->modes[mode_idx];
  58. struct v4l2_format format = {0};
  59. libmegapixels_select_mode(camera, mode, &format);
  60. // Now you can do regular V4L2 things on the camera with the
  61. // supplied FDs. video_fd for the /dev/video device.
  62. ioctl(camera->video_fd, VIDIOC_QUERYCAP, &cap)
  63. Libmegapixels replaces most of the original V4L2 setup code and provides a
  64. filled in :code:`struct v4l2_format` to get all the exact mode information.
  65. To pick a camera and mode you'd iterate over the found cameras and then over
  66. the mode structs to find something that matches the needs of the application.
  67. The camera struct provides the FDs for the various devices you might need:
  68. - The :code:`video_fd` field has the FD for :code:`/dev/video*` for actually
  69. getting the frames.
  70. - The :code:`sensor_fd` field is for the :code:`/dev/v4l-subdev*` device that
  71. represents the sensor. This is for setting camera V4L2 controls while capturing.
  72. - The :code:`media_fd` field is for the :code:`/dev/media*` device that has the
  73. pipeline for the current camera. This is mostly for the libmegapixels internals.
  74. The config files
  75. ----------------
  76. The libmegapixels library ships with configuration files for some devices. The
  77. device configuration can be stored in multiple locations so it can be overridden
  78. by packagers and end users.
  79. /usr/share/megapixels/config/%model.conf
  80. This is the location where the config files from packages are stored. This is
  81. also where the build script from libmegapixels will place the internal config
  82. files here.
  83. /etc/megapixels/config/%model.conf
  84. This is the place for extra and/or override config files
  85. $cwd/config/%model.conf
  86. The config is loaded from the current working directory to make testing and
  87. debugging the code easier and to run it from the root of the git repository.
  88. The %model argument in the path is referring to the device-tree compatible names
  89. of the device. This can be found in :code:`/proc/device-tree/compatible`. This
  90. stores the name of the hardware seperated by null-bytes in decreasing specificity
  91. order. Libmegapixels will check all of them in order for every location above.