Reverse Engineering Closed-Source GMSL Camera Drivers

Written By Summit Embedded

GMSL camera modules are frequently supplied with closed-source drivers: a prebuilt .ko, a devicetree overlay, and a short README naming the one platform it was tested on. That platform is often a Jetson Orin Dev Kit on a specific JetPack & kernel version. The closed driver won't work on any other hardware.

In the real world, production devices rarely ship on development boards with old kernel versions. It's much more common to ship a System-On-Module (Jetson, i.MX95, etc) on a custom carrier board with the latest security-patched kernel. Also, embedded vision devices frequently need advanced V4L2 controls (i.e. triggering), that may not be exposed by a closed-source driver.

Clients often approach Summit Embedded with a specific GMSL camera in mind and ask if we can create a custom driver for their system. This blog post details the first step in our process: collecting information from the camera vendor's closed-source reference driver and using it to adapt the mainline Linux kernel's open-source GMSL drivers.

Why is GMSL so complex?

Most cameras today interface directly with a Linux SOM via MIPI CSI-2 over an unshielded ribbon cable. This works great when the camera is a few inches from the SOM. Ethernet cameras are an alternative to span long distances, but Ethernet processing adds significant latency. This is where GMSL is used: long-distance applications that need low latency (like an automotive backup camera). GMSL tunnels the video data, power, and I2C commuincation over a single-wire coax cable.

Such a tunnel requires a pair of serializer/deserializer chips to encode & decode the data. The serializer is soldered next to the image sensor and converts MIPI CSI-2 (or parallel) video data into a coax signal. The deserializer lives on the SOM's carrier board to decode the data back into MIPI CSI-2. Typically the end user selects a camera based on the image sensor, without regard to the serializer embedded within. And the designer of the embedded Linux module selects a deserializer chip without knowing what cameras may be paired with it in the future. Thus the components are mixed & matched, which leads to an incredibly complex Linux driver. The image sensor, serializer, deserializer, and SOM video pipeline all need drivers that work in sync with each other. Closed-source GMSL drivers often support only one combination of sensor + serializer + deserializer + SOM.

The Summit Embedded approach

Everything the vendor knows about the hardware is expressed in the .ko (a compiled driver binary) and the device tree overlay. We just need to extract it. The .ko file contains I2C registers, timing/sequencing information, and kernel API calls. The device tree overlay contains video format and endpoint information.

1. Purchase the reference hardware and bring up the reference example

This step is straightforward. It is well-documented and supported by the camera vendor. The important part of this step is to budget the time for purchasing & setting up the reference system.

For this example, we'll be using a Waveshare GMSL-1MP-Camera-A with a RaspberryPi5 and Waveshare's MAX9296-GMSL-HAT. Their closed-source driver can be downloaded from their documentation here.

2. Decompile the devicetree overlay

A "compiled" .dtbo is not a black box the way a .ko is. It's simply an encoded text file. Use the Linux device tree compiler (dtc) to convert back to readable text. Phandle names are lost, but all the important information is still present. This tells you which bus, which chips, and which addresses to focus on when you trace the I2C bus in the next step.

sudo apt install device-tree-compiler;
dtc -I dtb -O dts -o gmsl-1mp-camera-a.dts /boot/firmware/overlays/gmsl-1mp-camera-a.dtbo;

Real output from the vendor's Raspberry Pi 5 overlay:

isx031@1a {
        compatible = "ws,gmsl-1mp-camera-a";
        reg = <0x1a>;
        status = "okay";
        dser-addr = <0x48>;
        ser-addr = <0x40>;
        clocks = <0xffffffff>;
        clock-names = "xvclk";
        avdd-supply = <0xffffffff>;
        dovdd-supply = <0xffffffff>;
        dvdd-supply = <0xffffffff>;
        sensor-type = "GMSL_1CH_LINKA";

        port {
                endpoint {
                        clock-lanes = <0x00>;
                        data-lanes = <0x01 0x02 0x03 0x04>;
                        link-frequencies = <0x00 0x47868c00>;
                };
        };
};

The above tells us:

  • the sensor is at i2c address 0x1a
  • the serializer at i2c address 0x40
  • the deserializer at i2c address 0x48
  • the CSI-2 output uses 4 lanes at 1.2 GHz (0x47868c00 = 1,200,000,000)
  • The binding convention is a single-cell reg + separate named ser-addr and dser-addr properties,
    • This differs from the two-cell reg = <ser sensor> form used on other GMSL drivers.
    • Matching the vendor's convention here is important for decompiling the .ko file later.

The <0xffffffff> values are unresolved phandles, an artifact of decompiling an overlay outside the tree it attaches to. The symbolic targets are recoverable from the __fixups__ section further down the same file.

3. Trace the I2C traffic the driver puts on the wire

GMSL deserializers, serializers, and image sensors are all configured via I2C commands. There is typically a sequence of writes in a specific order to initialize the "link" between the GMSL chips, then begin the transfer of image data through that GMSL coax tunnel.

Fortunately, most Linux kernels (if you have root access, which we do on the Raspberry Pi), allow you to trace all i2c messages going out onto any bus. To ensure your kernel supports this, run the below read-only command:

$ ls /sys/kernel/tracing # if present, tracing is enabled
$ grep -E "^CONFIG_(FTRACE|TRACING|DEBUG_FS)=" /lib/modules/$(uname -r)/build/.config
CONFIG_DEBUG_FS=y
CONFIG_TRACING=y
CONFIG_FTRACE=y

# `zcat /proc/config.gz | grep CONFIG_FTRACE` works too, where that file exists.

The above configs are typically enabled by default, especially on Raspberry Pi and Jetson development platforms. If they are not, you can recompile the Linux kernel for your device with these configurations enabled. See the documentation for your Linux platform to do this.

Next, let's find the right I2C bus for our device. From the device tree above, we know there is a bus with ID's 0x1A, 0x40, and 0x48 (image sensor, serializer, and deserializer respectively). Let's find it:

$ i2cdetect -l
# i2c-11  i2c             Synopsys DesignWare I2C adapter         I2C adapter
# i2c-13  i2c             107d508200.i2c                          I2C adapter
# i2c-14  i2c             107d508280.i2c                          I2C adapter
$ i2cdetect -y -r 11
#      0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
# 00:                         -- -- -- -- -- -- -- -- 
# 10: -- -- -- -- -- -- -- -- -- -- UU -- -- -- -- -- 
# 20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
# 30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
# 40: UU -- -- -- -- -- -- -- UU -- -- -- -- -- -- -- 
# 50: 50 51 52 53 -- -- -- -- 58 59 5a 5b -- -- -- -- 
# 60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
# 70: -- -- -- -- -- -- -- --

This shows those devices on I2C bus 11. UU means they are bound to the vendor's driver. If you want to confirm, unplug the camera and ensure those "UU" markings disappear from i2cdetect. - Note: that number doesn't always match the device tree name, which was i2c0 in this case.

I2C boot tracing

Once tracing is enabled, it's tempting to dive right into tracing while the device is running. There are many tutorials online about this, but we won't cover runtime tracing here because it won't work for GMSL. If a 3rd-party GMSL driver is well written, it should be possible to probe it at runtime. But often the developer only tests their driver when auto-loaded at boot. It's extremely likely that 3rd-party GMSL drivers have bugs that won't allow them to be probed outside of the normal boot sequence. Let's skip right to i2c tracing at boot

We need to add this string to the "kernel command line":

trace_event=i2c:i2c_write,i2c:i2c_read,i2c:i2c_reply,i2c:i2c_result trace_buf_size=2M

Follow the instructions for your development device to edit it's "kernel command line". This is a common debugging tweak that will be well documented.

  • On Raspberry Pi, put that string in /boot/firmware/cmdline.txt
  • On U-Boot (ie i.MX8 or i.MX95), add it to the bootargs variable from the serial console (contact Summit Embedded for help with this on your specific platform).
  • On a Jetson, edit the APPEND line in /boot/extlinux/extlinux.conf.

If the ring buffer below fills up too quickly (ie if HDMI EDID probing sends a lot of messages), increase trace_buf_size.

To view the trace data, cat the trace buffer with the bus id that we found above (bus #11):

# format is: a=<address> f=<flags> l=<length,bytes> [payload]

$ sudo cat /sys/kernel/tracing/trace | grep i2c-11

modprobe-515     [003] .....     6.226156: i2c_write: i2c-11 #0 a=048 f=0000 l=2 [00-00]
modprobe-515     [003] .....     6.226158: i2c_read: i2c-11 #1 a=048 f=0001 l=1
modprobe-515     [003] .....     6.228010: i2c_reply: i2c-11 #1 a=048 f=0001 l=1 [94]
modprobe-515     [003] .....     6.228012: i2c_result: i2c-11 n=2 ret=2

modprobe-515     [003] .....     6.228016: i2c_write: i2c-11 #0 a=048 f=0000 l=2 [00-10]
modprobe-515     [003] .....     6.228016: i2c_read: i2c-11 #1 a=048 f=0001 l=1
modprobe-515     [003] .....     6.229794: i2c_reply: i2c-11 #1 a=048 f=0001 l=1 [11]
modprobe-515     [003] .....     6.229795: i2c_result: i2c-11 n=2 ret=2

modprobe-515     [003] .....     6.229799: i2c_write: i2c-11 #0 a=048 f=0000 l=3 [00-10-91]
modprobe-515     [003] .....     6.230686: i2c_result: i2c-11 n=1 ret=1

# ... and so on

These are the first few transactions with the deserializer at address 0x48! Each register is 2 bytes. Thus, when the kernel writes 2 bytes onto the bus, it's requesting to read that register (confirmed by the "read" and "reply" lines below each read). A write is 3 bytes: 2 address bytes plus a value. So the above trace does:

  • read addr 0x48, register 0x0000 at 6.226156 seconds
    • value is 0x94
  • read addr 0x48, register 0x0010 at 6.228016 seconds
    • value is 0x11
  • write addr 0x48, register 0x0010 at 6.229799 seconds, with a value of 0x91
    • write succeeds (n == ret, details beyond scope of this post)

The timestamps are important, especially if one of the i2c registers is a "reset" register. Your custom driver may need to sleep briefly after one write before writing another.

The above log is abbreviated. There are 102 I2C reads & writes in the probing of this driver. Further down in the log, you'll see a few writes to the serializer (0x40) and the image sensor itself (0x1a).

102 i2c operations are manageable, but a lot to decipher yourself. At this point, we recommend that you feed the log into your favorite AI tool. Provide the datasheets for the serializer and deserializer and let it parse the meaning of each register. In this case, we know from the device datasheets that the deserializer is a MAX9296a and the serializer is a MAX96701. If we did not have this information, it's typically available in the first few i2c reads (drivers often confirm they're talking to the right chip first). Again, your favorite AI agent can help you determine that now that we have the raw data. - Warning: Analog Devices MAX GMSL datasheets are public - no NDA required. But image sensor datasheets are often covered under NDA. Review any NDA terms & conditions before sharing with an AI tool.

4. List kernel symbols called by the module

A .ko module, like any dynamically-linked binary file, has to have some text strings inside it to say what functions it must jump to when running. Because it's not compiled into the kernel binary itself, the nm (or names) command can dump all the calls this driver makes to the kernel, and any other libraries or modules:

$ nm gmsl-1mp-camera-a.ko
#                  U alt_cb_patch_nops
# 0000000000000000 T cleanup_module
#                  U _dev_err
#                  U _dev_info
#                  U devm_gpiod_get_optional
#.....
#                  U media_entity_pads_init
#                  U msleep
#                  U __mutex_init
#.....                
#                  U v4l2_async_register_subdev_sensor
#                  U v4l2_async_unregister_subdev
#                  U v4l2_ctrl_handler_free
#                  U v4l2_ctrl_handler_init_class
#                  U v4l2_ctrl_new_int_menu
#                  U v4l2_i2c_subdev_init
#                  U v4l2_subdev_link_validate
#                  U __v4l2_subdev_state_get_format
# # ... and so on

From this output, we know:

  • msleep() is used. We will likely need delays between certain register writes
  • there are devm_gpiod_get_optional() & gpiod_set_value_cansleep() calls. In this case, we already know there are no *-gpio properties in the device tree, so the optional GPIO aren't used. But, if there were, we could enable gpio tracing in /sys/kernel/tracing to see how it aligns with the I2C traffic.
  • v4l2*() calls will all be needed in our driver to set up the media pipeline.

In many cases, the vendor's .ko binary also contains internal function names for "oops backtraces". In this case, the module has a goldmine of internal function information. You can view the internal function call structure with:

objdump --disassemble --reloc gmsl-1mp-camera-a.ko | grep '<.*>'
# 00000000000001a8 <gmsl_remove>:
#  1cc:   94000000        bl      0 <v4l2_async_unregister_subdev>
#  1d4:   94000000        bl      0 <v4l2_ctrl_handler_free>
#  1e0:   94000000        bl      0 <__pm_runtime_disable>
# 0000000000000208 <gmsl_open>:
#  22c:   94000000        bl      0 <mutex_lock>
#  23c:   94000000        bl      0 <__v4l2_subdev_state_get_format>
#  25c:   94000000        bl      0 <mutex_unlock>
# 0000000000000288 <gmsl_set_format>:
#  2b8:   94000000        bl      0 <mutex_lock>
#  310:   94000000        bl      0 <__v4l2_subdev_state_get_format>
#  330:   94000000        bl      0 <mutex_unlock>
# ..... and so on

Provide this function call structure, along with the I2C trace from above, to your favorite AI tool. These two pieces of information can generate the majority of the driver code almost instantly.

One other small helper is the strings command. It dumps all human-readable strings in the binary. This contains function names, as well as any error strings that any error handling code might print. It's not essential to reverse engineering the driver, but could help make your driver more robust.

5. Media Graph

The above steps give us everything we need to make a basic Linux kernel driver for this GMSL camera (I2C register writes & timing). Those I2C transactions set up the GMSL tunnel over coax and tell the image sensor to start streaming. Now the video data is reaching the CSI pins. But we need to tell the CSI driver how to decode this data (pixel format, resolution, color space, etc).

This can be done in the driver itself to deliver one monolithic .ko and .dtbo for a given camera + platform. But the best-practice is to do this in userspace. In this case, Waveshare provides a script/ws_camera_ctrl.sh userspace script alongside their driver. This is readable source code. They use many variables, but the script essentially boils down to:

media-ctl -r -d /dev/media0
media-ctl -d /dev/media0 -l "'csi2':4 -> 'rp1-cfe-csi2_ch0':0 [1]"
media-ctl -d /dev/media0 -V "'csi2':0 [fmt:UYVY8_1X16/1280x960 field:none colorspace:smpte170m]"
media-ctl -d /dev/media0 -V "'csi2':1 [fmt:UYVY8_1X16/1280x960 field:none colorspace:smpte170m]"
media-ctl -d /dev/media0 -V "'csi2':2 [fmt:UYVY8_1X16/1280x960 field:none colorspace:smpte170m]"
media-ctl -d /dev/media0 -V "'${sensor_entity}':0 [fmt:UYVY8_1X16/1280x960 field:none colorspace:smpte170m]"
v4l2-ctl --device=/dev/video0 --set-fmt-video=width=1280,height=960,pixelformat=UYVY
# and if 2 cameras:
media-ctl -d /dev/media0 -l "'csi2':6 -> 'rp1-cfe-csi2_ch2':0 [1]"
media-ctl -d /dev/media0 -V "'${sensor_entity}':2 [fmt:UYVY8_1X16/1280x960 field:none colorspace:smpte170m]"
v4l2-ctl --device=/dev/video2 --set-fmt-video=width=1280,height=960,pixelformat=UYVY

Be sure to include a systemd service that configures the media map in this way. If you'd like to move this into a monolithic .ko file, contact Summit Embedded and we would be happy to help!

6. Write Your Custom Driver

After collecting all the above information, you're ready to write your own driver source code. You can start from scratch, or copy an existing driver from upstream and modify for your serializer/deserializer. If you copy from upstream, be sure to adhere to any applicable open-source license requirements (GPL).

A note on upstream GMSL drivers

At the time of writing, the GMSL drivers in the mainline kernel are in a state of disarray. Only a few have been upstreamed, and only for certain serializer/deserializer combinations. Most only support GMSL2, which greatly simplified the Link Negotiation sequence. GMSL1 has a more complex sequence, thus most drivers don't implement it unless absolutely needed.

Analog Devices is currently developing a "proper" GMSL driver for their chips. See https://lwn.net/Articles/1093365/ for details. They plan to support all of their modern GMSL2/3 chips in one framework. This will allow easier mixing & matching of serializers & deserializers, more examples to pull from for new implementations, and general improvements to Linux's GMSL support. However, there is no target date as to when this will be available.

At Summit Embedded, we have reverse engineered several GMSL drivers (Waveshare, TechNexion, and others under NDA). Many are GMSL1-based. We used a mid-2026 version of Analog Device's proposed max_serdes framework. Hopefully it only requires small changes to update when their work is finally upstreamed to the mainline kernel. At that point, Summit Embedded plans to upstream expanded support for GMSL1 in those drivers.

In the meantime, if you need a custom GMSL driver for your platform, please contact Summit Embedded. We even provide support for GMSL1, which most embedded firms do not support.

Summit Embedded https://summitembedded.com
Next

Using SmartGit To Supercharge Your Git Workflow