BlackBerry QNX’s self-driving Lincoln MKZ – what’s under the hood?

Kerry Johnson

Sr. Product Manager

BlackBerry QNX

At CES 2017, BlackBerry-QNX unveiled its self-driving Lincoln MKZ. 

In years past , BlackBerry QNX has become known for displaying its innovative technology in its concept cars, which included infotainment, mobile device connectivity, digital instrument clusters and ADAS. This year BlackBerry QNX has outfitted a Lincoln MKZ to demonstrate a self-driving vehicle. The Lincoln MKZ is much more than a demonstration vehicle – it is an engineering prototype that allows BlackBerry QNX engineers to experiment with and develop new technologies for the autonomous vehicle market.

You may wonder why BlackBerry QNX chose a Lincoln MKZ for its autonomous driving car. The reason is straight forward. The 2017 Lincoln MKZ comes equipped, from the factory, with all the necessary drive-by-wire capabilities. All of the driving systems (throttle, gearbox, steering and braking) can be completely controlled electronically. By using this capability as a starting point, BlackBerry QNX and its partners are able to focus on adding other self-driving capabilities such as the sensors, route planning, and maneuvering.

While providing the foundational software, BlackBerry QNX did not build this self driving vehicle alone. We worked closely with Renesas, University of Waterloo, Polysync and Cogent, to put the car on the road.

The following is a brief walkthrough of the technologies inside the Lincoln MKZ:

BlackBerry QNX

BlackBerry QNX’s goal was to build an autonomous vehicle using commercial embedded processors and safety certified embedded operating system (OS). At the core of the design was QNX’s safety certified OS, which powers all of  the intelligent software modules. QNX’s middleware serves to integrate RADAR, LIDAR sensors , multiple camera inputs and vehicle networking. BlackBerry QNX provided a port of the OpenCV library to help with the vision processing functions delivered by Cogent. 

BlackBerry QNX also provided a port of Robot OS (ROS), so that the University of Waterloo could easily bring their self-driving software algorithms to the car without having to re-write large portions of code.

The ROS software components are not truly embedded, production oriented software. However, in building an autonomous car we chose a phased approach. We chose to use existing software to test and validate the solution. This saves time and allows flexible prototyping. Once the code is finalized we can convert it into an embedded solution.   

University of Waterloo

The University of Waterloo, one of Canada’s leading autonomous driving research institutions, contributed several software components, including static and dynamic environment perception, path planning, maneuvering and dispatching control commands to the various actuators. It should be noted that, at the outset of the project, the University of Waterloo already had a number of these components operational. Part of the activity was to port the software from Linux to QNX – a task made simple by BlackBerry QNX’s support for the POSIX standard.

Polysync

Polysync provided their framework for distributed communications and sensor integration. They also provided system data visualization tools, so the engineers could see how the system was operating from a central console.

Cogent

Cogent provided a number of vision processing algorithms that processed input from multiple camera sensors.

Renesas

The compute horsepower in the Lincoln MKZ comes from two Renesas R-Drive reference boards. Each Renesas R-Drive board has two Renesas R-Car system on chips (SoCs), each with quad-core ARM processors and image processing accelerators. Two R-Drive systems were used so that fail-over scenarios could be tested.

Sensors

The following sensors were used to construct a 360-degree view of the surroundings and to achieve accurate positioning of the car:

• 1 Delphi long range radar
• 1 Delphi short & medium range radar
• 2 Velodyne LIDARs
• 1 forward-facing Point Grey camera
• High precision GPS and IMU (Inertial Management Unit)

The car is now running on a test track.  In the following years BlackBerry QNX will continue to refine the system towards production oriented hardware and software. 

On 64-bit and roadmap alignment

By Romain Saha
Strategic Alliances Manager
Blackberry QNX
 

One of the coolest things about my job is getting to see all the silicon roadmaps. OK, I’m a nerd. That is not a surprise to anyone I’m sure. Still, there’s a lot of amazing innovation going on. You just haven’t heard about. Yet. And you won’t hear it from me. Sorry.

Except maybe that the embedded world – at least in the areas we play – is going 64-bit. Pretty much already gone 64-bit actually. Intel architecture has been 64-bit for as long as I can remember. ARMv8 is almost exclusively 64-bit. It’s here. It’s real. You can buy it. Pretty much everybody has it.

Graphics is another area that is moving fast. The latest embedded GPUs are really impressive. I know of one chip that actually has two full-blown GPUs on a single die. The things I’m seeing on the bench are amazing. Light-years ahead of where we were only a generation ago.

Roadmap alignment is key for us. We need to make sure our products sing with our silicon partner’s technology. We need to make sure customers can take the latest SoCs and build the things stuff the world wants.

One of the things the world wants these days is a digital cockpit - a unified experience across multiple displays in the cabin.  That is happening today but it takes two SoCs to do it, one for cluster and one for infotainment system. It also takes space, power, cabling, connectors and inter-processor communication. Hassle. Lots of hassle I bet.

The obvious dream is to eliminate all this cost and complexity and just use a single SoC for both. Easy. Except digital instrument clusters and infotainment systems are different. Very different.

Infotainment systems need lots of horsepower. They use lots of memory. Navigation systems alone can drive addressing past 4 Gigabytes. Huge state diagrams. They also need lots of eye-candy. GPU performance is key. Complicated indeed.

Digital instrument clusters need incredibly smooth graphics performance but are relatively simple otherwise. Except that they are safety critical. Enter ISO26262 certification. Overlay that with making sure what you think you are rendering is actually what gets displayed. If the screen says P(ark) and the car is in R(everse) people get hurt. Or worse. Clusters are complicated too.

What do you need to make all this work? For sure you need a 64-bit safety-certified embedded OS. You need a hypervisor with the ability share graphics across virtual machines. You need ISO26262 top to bottom as well as a way to ensure cluster rendered output matches the intended output. And you need an SoC with the juice to make it all happen. That’s a lot. No wonder people think the single chip digital cockpit is still a dream.

QNX, NVIDIA team up to deliver infotainment solutions

Today, at SAE Convergence, QNX announced that it is working with graphics leader NVIDIA to bring infotainment solutions to the automotive market. As part of this initiative, the companies will integrate support for the NVIDIA Tegra processor into the QNX CAR 2 application platform.

The Tegra system-on-chip is the size of thumbnail, yet it incorporates a quad-core ARM CPU and a GeForce GPU, as well as dedicated audio, video, and image processors.

The NVIDIA Tegra visual
computing module

“QNX Software Systems and NVIDIA have a proven track record of delivering on production programs for Audi... and we’re excited to add support for Tegra to the latest generation of our automotive platform,” said Linda Campbell, QNX director of strategic alliances.

Speaking of Audi, NVIDIA is bringing an Audi A6 to SAE Convergence, equipped with an infotainment system powered by technology from QNX and NVIDIA. The system bristles with high-end features, including 3D navigation with Google Maps and Google Earth, as well as natural voice recognition.

For more information on this announcement, read the press release, and for more information on QNX activities at SAE Convergence, visit our Convergence overview page.

Setting the Pace for Automotive Electronic Innovation

Welcome to the first installment in a series of guest posts from Paul Sykes of Freescale’s driver information systems team.

Recently, I traded in my MY2002 SUV for a new MY2012 vehicle. At the time, the MY2002 was quite advanced in its in-cabin electronics and styling, but wow — times have changed! Gone are the aftermarket satellite radio and PND that were attached, with wires dangling, at various places in my cabin.

The newest generation of vehicles offers complete and total integration, including new features that didn’t exist in 2002, such as USB/iPod interfaces, HD Radio, and a rear view camera.

But here’s the problem. I work in this great industry of automotive electronics and have some view of what’s coming in the next wave of vehicles. It’s both a blessing and a curse. Do I buy now or wait for the next model year? It’s like trying to time when to jump into your next cell phone or tablet purchase, only the time scale is a bit different.

The pace of electronic innovation has increased in this industry and you don’t have to wait 10 model years (like I did) to see it. It’s exciting to be a part of the supply base that is helping this industry move faster while maintaining some of the highest standards of quality and reliability over a long product life.

Fundamentally, at the heart of every embedded electronic vehicle system, incoming data needs to get processed and acted upon, using complex software algorithms. At Freescale, sensor and processor innovations make the future possible by doing these fundamental elements better, faster, and more reliably.

Ecosystem partners like QNX Software Systems provide many of the complex algorithms required to realize infotainment and instrument cluster systems. These same systems are often powered with Freescale i.MX processors.

In the latest generation, the i.MX 6 Series, Freescale has provided the most scalable line-up of products available. Scalability means not only performance and function scalability, but also pin-to-pin hardware compatibility across the entire series. This is one example of how the processor can help pick up the innovation pace. With hardware and software compatibility, system makers can develop more products to meet a broader range of market needs, in a shorter amount of time.


The Freescale i.MX 6 Series has been chosen to power the next-generation
GM OnStar system.

In future posts, I will offer Freescale’s perspective on many of the current trends in driver information systems as well as our product collaborations with QNX to bring unique value to the industry.

---

Here’s a little more about Paul and the Freescale Driver Information Team:

Paul has more than 15 years’ experience in the semiconductor industry, including product development, program management, and marketing positions. For the past several years, Paul has lived in Michigan and focuses exclusively on the automotive telematics, audio/infotainment, and instrument cluster application spaces.

Freescale’s Driver Information Team is driving the global strategy and product development for solutions to address the multitude of applications in the rapid growth and innovation area of Driver Information Systems. This includes instrument cluster, graphics displays, audio and infotainment, and telematics.
 

QNX concept car makes detour at TI headquarters

Guest post by Kroy Zeviar, QNX strategic alliances

My colleague Mark Rigley must feel (justifiably) proud these days. He is, after all, head of the team that created the new QNX concept car, which took home a Best of Show award from 2012 CES.

You'd think that Mark and I would be anxious to get home after a hectic, albeit gratifying week at CES. And indeed, we were. But we made a slight detour and stopped over in Dallas to show off the car to our friends at Texas Instruments.

Everyone loved it. The car, based on a Porsche 911, takes advantage of TI's OMAP4 and wireless connectivity solutions, so folks were naturally excited to see what their technology had helped enable.

Better yet, TI brought in a pro photographer to take these cool pix...



Upwards of 400 folks came out to see the car


Checking out the new head unit and digital cluster


Mark, in the passenger seat, giving one of many demos


Best of CES 2012 winner. Woo-hoo!


This shot is just cool.


And speaking of cool, a huge shoutout to the TI OMAP and
wireless solutions teams for all their great support!
 

CrackBerry posts first peek at OnStar RemoteLink for BlackBerry PlayBook

Paul Leroux

This morning at CES, CrackBerry.com met up with QNX's Andrew Poliak for a walkthrough of the new OnStar RemoteLink app for the BlackBerry PlayBook.

If you aren't familiar with RemoteLink, it provides a very cool and powerful connection to OnStar-equipped vehicles. From the convenience of your tablet or phone, you can access gas mileage, tire pressure, and other information in real time; you can even remotely start your vehicle and unlock its doors.

Cooler yet, the new version running on the PlayBook boasts a user interface built entirely in HTML5. But enough blather from me. Roll the tape...



Did you know? QNX is the core OS for the PlayBook, but it's also the OS for OnStar and OnStar FMW.
 

QNX-based nav system helps Ford SUVs stay on course down under

Paul Leroux

This just in: SWSA, a leading electronics supplier to the Australian automotive industry, and NNG, the developer of the award-winning iGO navigation software, have created a QNX-based navigation system for Ford Australia. The new system has been deployed in Ford Territory SUVs since June of this year.

To reduce driver distraction, the system offers a simplified user interface and feature set. And, to provide accurate route guidance, the system uses data from an internal gyroscope and an external traffic message channel, as well as standard GPS signals. Taking the conditions of local roads into account, the software provides a variety of alerts and speed-camera warnings; it also offers route guidance in Australian English.

The navigation system is based on the iGO My way Engine, which runs in millions of navigation devices worldwide. To read NNG's press release, click here.


SWSA's new nav system for the Ford Territory is based on the Freescale
i.MX31L processor, QNX Neutrino RTOS, and iGO My way Engine.