media: staging/imx: update TODO

Update TODO file:

- Remove TODO info about the OV564x driver, while this still needs
  to be done (add a OV5642 driver or merge with OV5640 driver), it
  is not relevant here.

- Update TODO about methods for retrieving CSI bus config.

- Add some TODO's about OF graph parsing restrictions.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

1 files changed, 51 insertions, 12 deletions

diff --git a/drivers/staging/media/imx/TODO b/drivers/staging/media/imx/TODO
index 0bee3132b26f..9eb7326f3fc6 100644
--- a/drivers/staging/media/imx/TODO
+++ b/drivers/staging/media/imx/TODO
@@ -1,19 +1,14 @@
 
-- Clean up and move the ov5642 subdev driver to drivers/media/i2c, or
-  merge support for OV5642 into drivers/media/i2c/ov5640.c, and create
-  the binding docs for it.
-
 - The Frame Interval Monitor could be exported to v4l2-core for
   general use.
 
-- At driver load time, the device-tree node that is the original source
-  (the "sensor"), is parsed to record its media bus configuration, and
-  this info is required in imx-media-csi.c to setup the CSI.
-  Laurent Pinchart argues that instead the CSI subdev should call its
-  neighbor's g_mbus_config op (which should be propagated if necessary)
-  to get this info. However Hans Verkuil is planning to remove the
-  g_mbus_config op. For now this driver uses the parsed DT mbus config
-  method until this issue is resolved.
+- The CSI subdevice parses its nearest upstream neighbor's device-tree
+  bus config in order to setup the CSI. Laurent Pinchart argues that
+  instead the CSI subdev should call its neighbor's g_mbus_config op
+  (which should be propagated if necessary) to get this info. However
+  Hans Verkuil is planning to remove the g_mbus_config op. For now this
+  driver uses the parsed DT bus config method until this issue is
+  resolved.
 
 - This media driver supports inheriting V4L2 controls to the
   video capture devices, from the subdevices in the capture device's
@@ -21,3 +16,47 @@
   link_notify callback when the pipeline is modified. It should be
   decided whether this feature is useful enough to make it generally
   available by exporting to v4l2-core.
+
+- The OF graph is walked at probe time to form the list of fwnodes to
+  be passed to v4l2_async_notifier_register(), starting from the IPU
+  CSI ports. And after all async subdevices have been bound,
+  v4l2_fwnode_parse_link() is used to form the media links between
+  the entities discovered by walking the OF graph.
+
+  While this approach allows support for arbitrary OF graphs, there
+  are some assumptions for this to work:
+
+  1. All port parent nodes reachable in the graph from the IPU CSI
+     ports bind to V4L2 async subdevice drivers.
+
+     If a device has mixed-use ports such as video plus audio, the
+     endpoints from the audio ports are followed to devices that must
+     bind to V4L2 subdevice drivers, and not for example, to an ALSA
+     driver or a non-V4L2 media driver. If the device were bound to
+     such a driver, imx-media would never get an async completion
+     notification because the device fwnode was added to the async
+     list, but the driver does not interface with the V4L2 async
+     framework.
+
+  2. Every port reachable in the graph is treated as a media pad,
+     owned by the V4L2 subdevice that is bound to the port's parent.
+
+     This presents problems for devices that don't make this port = pad
+     assumption. Examples are SMIAPP compatible cameras which define only
+     a single output port node, but which define multiple pads owned
+     by multiple subdevices (pixel-array, binner, scaler). Or video
+     decoders (entity function MEDIA_ENT_F_ATV_DECODER), which also define
+     only a single output port node, but define multiple pads for video,
+     VBI, and audio out.
+
+     A workaround at present is to set the port reg properties to
+     correspond to the media pad index that the port represents. A
+     possible long-term solution is to implement a subdev API that
+     maps a port id to a media pad index.
+
+  3. Every endpoint of a port reachable in the graph is treated as
+     a media link, between V4L2 subdevices that are bound to the
+     port parents of the local and remote endpoints.
+
+     Which means a port must not contain mixed-use endpoints, they
+     must all refer to media links between V4L2 subdevices.