Payload Integrations
An overview of payload integrations that can be fully controlled by the onboard Jetson and our gstreamer solutions.
Last updated
Was this helpful?
An overview of payload integrations that can be fully controlled by the onboard Jetson and our gstreamer solutions.
Last updated
Was this helpful?
The onboard Jetson is an incredibly powerful computer, and we made sure that it can connect to any relevant system on the drone. This includes the mavlink system, multiple onboard networks, and USB interfaces. There is a lot of flexibility with the type of payloads that can be integrated with the F-11, and we chose to highlight some of the integrations we have that highlight this flexibility.
The Sony LR1 provides a tremendous amount of control through its developer library, and we made sure to utilize that to its fullest extent. Our integration supports all major features that are required by standard drone missions, as well as many features that are needed for high quality drone photography. With our integration, users and developers can take pictures, record videos, digitially zoom in and our, and even control optical zoom in and out when using powered zoom lenses. Additionally, we implemented ways to utilize the camera's autofocus intuitively, set the exposure bias, and even set the white balance. Additional features may be available on request, though we are confidant that we have a full set of features that can satisfy very demanding missions.
The Sony LR1 can be connected directly to the Jetson via USB 3.0, which allows us to both control the camera and get a live stream with blazing speeds.
We have also optimized our integration to feed the sensor data directly into a gstreamer pipeline, resulting in a huge minimization in streaming latency. Our solution, which streams incredibly high resolution video over radio, is nearly indistinguishable from a direct HDMI connection in terms of its latency and quality.
Both the F1 and G1 versions of the Gremsy Vio are fully supported on our drones. We have implemented full gimbal control that are intuitive and allow extremely precise movements. The Vio integration supports taking pictures and videos, both of which capture both EO and IR cameras simultaneously. The gimbal pitch and laser range finder range values are always reflected on the drone controller with extremely low latency. For the IR camera, we have the option of choosing between many different color palettes, including, but not limited to, WhiteHot, Ironbow, and Rainbow. We also support digital zoom for the IR camera, as well as up to 30x optical zoom on the EO camera. The Vio gimbal position can also be set to a wide range of very precise angles.
The Vio is connected to the Jetson in two ways. For gimbal, camera, and LRF control, the Vio is connected to the onboard mavlink system and can be controlled through mavlink messages.
For video feeds, the Vio is connected to our gigabit ethernet switch and lives in the same network as the Jetson, which gives us high quality video with low latency.
The GoPro Hero 12 Black delivers great video quality for a relatively low price, and is a great example that highlights the versatility of our system. The GoPro is connected through USB, and can be fully controlled through the Jetson. This includes taking pictures and videos, but allows users to change the GoPro settings to their needs. We are able to stream HQ video and record 4K video, as well as dynamically changing framerate, resolution, bitrate, bit depth, image stabilization, and any video or picture profiles that are required by the user.
Our system is fully modular, allowing for payloads to dynamically be added or removed while the drone is already on. This results in a very complex system that maintains the payload services and running streams. To this end, the controller and onboard Jetson work together to allow streams to be put up and torn down without any interruptions for the user. We were required to overcome tremeandous challenges and limitations of gstreamer in order to have the controller dynamically respond to stream changes on the drone. To this end, developers looking to integrate their own payloads only need to register their payload and stream services with the onboard drone service, and the controller automatically handles the rest.
The drone service and payload framework is currently used internally to develop all of our integrations, and will be made available for public in the future.
