Customizing the User Interface

There are several ways to extend or customize the web User Interface by loading custom JavaScript modules and CSS files into it. This is configured in the robot’s phntm_bridge.yaml config file. None of the following techniques require you to to host any part of the Phantom Bridge cloud infrastructure besides your own extension sources.

phntm_bridge.yaml

 /**:
   ros__parameters:

     ui_custom_includes_js:
       - https://localhost:1234/custom-input-driver.js
       - https://www.some-server.com/custom-service-slider-widget.js
     ui_custom_includes_css:
       - https://www.some-server.com/example.css

Note

In the example above, we are injecting CSS and JS files both from the localhost as well as a public web hosting. Localhost can be indeed used for development puposes, keep in mind that all your extensions need to be available on the internet. Included file that fails to load will likely break the UI, or cause unexpected behavior.

In the Bridge UI Extras repository, you will find all examples mentioned below and a lighweight Node.js server that can serve your extensions with the proper CORS policy and SSL certificate, both required for security reasons by all modern browsers.

Customizing of the Bridge Interface can be split into the following categories:

Customizing CSS

With custom CSS, you can override the default look, change colors, and load 2D graphics as needed. A mechanism for pre-loading of custom images from your CSS files is included, use it to load graphics ahead of use to provide a smooth user experience. See examples in css-example, the demo-tweaks.css file in fact cointains modifications made for the live demos.

Custom Input Drivers

On top of the built-in Input Drivers that can generate Twist, TwistStamped and Joy messages, you can also easily create a custom drivers that map user’s keyboard, gamepad or touchscreen input to any other ROS message types.

See more under User Input & Teleoperation.

Single-type Panel Widgets

The UI comes with several built-in widgets for visualizing ROS topics based on their type. You can make your own custom panel widgets by extending the SingleTypePanelWidgetBase class. You can use either 2D graphics or create a custom 3D scene.

There is a simple example widget displaying Bool messages available in the Extras repo and in our live demos.

Composite Panel Widgets

Composite widgets offer the possibility to mesh together different topic types in order to present them jointly either in 2D or 3D. A showcase of this would be the World Model 3D widget, or a much simpler example which creates a map of Wi-Fi signal strength in combination with robot’s odometry.

For a composite panel widget, you will need to extend the CompositePanelWidgetBase class.

2D Video Overlays

The built-in Video widget allows to create custom data overlays to display information over the video frames. In the live demos, you can see this work with vision_msgs/msg/Detection2DArray messages. There is one other example built-in displaying TwistStamped messages as a visualisation of the input keys.

In order to create a new video overlay plugin, extend the VideoPluginBase class.

World Model 3D Plugins

In a simlar fashion, the World Model 3D widget can also be extended to show custom data in 3D. One example of that would be the virtual battery LEDs shown on the robot in our demos. The widget can already combine several data types and show them on the URDF model animated with TF messages. You can add any other data visualisation or interactive element and map these to the correct reference frames on the URDF model.

To create a custom plugin for the World Model 3D, extend the WorldModel3DPluginBase class.

Notes

The public User Interface API allows to add extra items to the panel’s menu, create functional buttons in the title bar, open dialogs or store and load custom panel variables. You can read custom config parameters passed from the robot’s phntm_bridge.yaml file both for the whole UI and for specific topics or services. You can also call ROS services programatically and write data into your topics via fast UDP Data Channels.

jQuery, D3.js, CanvasJS Charts and Three.js are all available to your extensions out of the box and used by the built-in widgets. If you need some inspiration, take a look at how these are implemented.

You can load custom graphics from the internet as well as 3D models and other sources from the robot.

Browse the Bridge UI Extras repository for example code and to see how your files need to be hosted. Feel free to have a look at how the built-in widgets are implemented and use the API.

If you create some useful extensions worth sharing with others, consider letting us know.