Autonomous Cars – Part 3: Technology Consolidation

Kaivan Karimi
SVP of Strategy and Business Development
BlackBerry Technology Solutions

The amount of software in a car is mushrooming with there being over 100 million lines of code in a modern car, which is more than most any other system.

Today cars are controlled via hardware electronic control units (ECUs) running the millions of lines of code.   60 to 100 ECUs are found in most newer cars today and that number is growing.  High end cars can have even more.  Reducing the number of ECUs in favor of reduced number of domain/area controllers is the new trend.  The idea is to reduce the complexities associated with software development, reduce the weight of the car, and reduce the overall cost. It also makes software upgradability less complex, where software functionalities can be enhanced to extend the life of a platform and offer a very large return on investment. 

Another benefit is that software can be more easily upgraded Over-The-Air (OTA) for minor or major fixes, respond to security issues, and provide other enhancements without the need to bring a car to the dealership.  This not only saves time, but also adds to the safety, security, and reliability of the car, while lowering the overall maintenance cost for the vehicle. According to research firm IHS, about 4.6 million cars received OTA software updates for telematics applications last year, and by the year 2022 forty-three million cars are expected to be using OTA services.  That is clearly a huge increase.

Some of the other technology components of Advanced Driver Assistance Systems (ADAS) are noted below:

Maps

While most people do not consider maps as a component of an ADAS system, in the future they will play a key role in assisting drivers to operate vehicles safely and adapt to driving changes based on location, such as changing what side of the road you drive on when you hit a border crossing. Maps provide a necessary input to augment the information that is provided by the various sensors in the car. This is not just macro-level geological data for finding directions, but also for augmenting functions such as camera-based traffic sign and roadway information detection, as well as infrastructure information. Cloud based processing will then be used to integrate the data sent by all vehicles into a global map that gets updated cooperatively by all drivers, including the road pot-holes to avoid, new roadway signs added, or rerouting due to construction.

Sensor Fusion

Sensor fusion means combining information and data from different sensors, leveraging the individual advantages of each sensor to complement and cover the weaknesses other sensors. The whole is greater than the sum of the parts, which means the individual sensors’ functions. This is very similar to what our brain does. You do not need to touch a pot of boiling water to know it is very hot, because your eyes to see the bubbling water and the steam on the top of the pot. In an ADAS system, the same thing happens: The sensor inputs are fused together for the ADAS domain controller to formulate a conclusive opinion about an event with better situational awareness, rather than just relying on a certain sensor’s data individually. This notion is at the heart of how any robot operates, but is especially important with the mission critical functionalities needed by connected autonomous cars.

HW & SW Roadmap to Consolidation

As the modern CPU increases in processing power, and decreases in electrical power consumption due to smaller process geometries, it would lead one to believe that consolidating multiple ECU functions onto one physical processor may result in significant cost savings. While that is true, consolidation needs to be balanced with a few important factors:

1.  The increase in leakage current as semiconductor process geometries get smaller (this is a downside of Moore’s Law) 
2. Thermal issues increase as clock speeds increase 
3. The extent to which the software can be multithreaded to take advantage of new multi-core  processors.

The auto industry will be going through a transformation with ECU consolidation into single powerful multi-core processors that is similar to what happened in early 2000s in the networking industry.  At that time I had a front seat to the networking debate as I was driving some products a large semiconductor company. What happened was that most network and baseband processor semiconductor suppliers for both wired and wireless infrastructure business moved from single to dual to quad-core processors.  I remember a day when people were planning to pack as many as 80 cores into a single chip.

There is a huge difference between the software requirements for mutli-core processing in the networking and automotive industries.  

The elephant in the automotive room is the need to combine mission-critical with non-mission critical functionalities into the same processor, while separating and isolating these functions effectively from each other from a safety and security perspective. This single fundamental requirement becomes the basis for what types of software framework and architecture needs to be used.

Multi-core Processing

At a very high-level, all multi-core processors pack multiple processing units (cores) into a single    physical package—just like it sounds. But, this is where the similarities end. Other architectural factors come into play and determine the application fit, throughput, bandwidth, effective horsepower, and software architectures suitable for an optimal processing environment. Some of the considerations are noted below:

• Choice and configuration of interconnect buses and shared memory schemes

• Choice of homogeneous multi-core systems with identical cores sharing the same instruction sets, vs. heterogeneous multi-core systems with identical cores (some with same instruction set, and some with different ones 

• Heterogeneous multi-core systems that mix different types of processor cores for application specific use cases (e.g. mix of MPUs, DSPs, GPUs, etc.).

• Mix of the above core with localized memories and predefined high-level functions such as micro-coded engines and vector processors

• Mix of cores and architectures that allow control and data path processing in a single core for communication applications 

• Choice of architectural implementations such as VLIW, vector or multithread processors, fine-grain vs. coarse grain processors, etc.

The improvement in performance by using multi-core processors can only happen if the software running on the processor can take advantage of every last cycle that the multiple core device can offer. It also assumes that the interconnect buses and interfaces between the cores and the world outside of the chip, as well as between the cores, and the interaction between the cores and the memory architecture are properly modeled and designed for the end application, so that there are no design bottle necks introduced. 

This situation is analogous to adding multiple streets and multiple lanes in and out of a parking lot. If the electronic door to go in and out of that parking lot is too slow to accommodate the extra traffic, you will cause bad congestion, and the traffic throughput in and out of the parking lot would be as good as the speed of that electronic door. You may need to open the gate altogether, but have a traffic cop that coordinates the flow of traffic in and out of different entrances, into different parking spots. That is exactly what you would also need in the world of software, namely a traffic cop for the processes running in the given multi-core architecture. That is where a hypervisor comes in, which is to act as that traffic cop.

QNX offers a hypervisor and other safety- and mission-critical software for make connected autonomous cars safe reliable, secure, and trusted.

The next blog will address the hypervisor/traffic cop, and describe how they make the software-defined future more autonomous and safe.

                                                  

Kaivan Karimi is the SVP of Strategy and Business Development at BlackBerry Technology Solutions (BTS). His responsibilities include operationalizing growth strategies, product marketing and business development, eco-system enablement, and execution of business priorities. He has been an IoT evangelist since 2010, bringing more than two decades of experience working in cellular, connectivity, networking, sensors, and microcontroller semiconductor markets. Kaivan holds graduate degrees in engineering (MSEE) and business (MBA). Prior to joining BlackBerry, he was the VP and General Manager of Atmel's wireless MCUs and IOT business unit.

Autonomous Cars Part 1-- And Now for Something Completely Different: The Autonomous Accident

Kaivan Karimi
SVP of Strategy and Business Development
BlackBerry Technology Solutions (BTS)

A few weeks ago a self-driving Tesla Model S in Autopilot mode crashed into a large semi-trailer in Williston Florida.   This is pretty much what lawyers call a case of first impression, and rightfully so.  This unprecedented event brought up a bunch of questions, and it is clear that we are now on the cusp of the autonomous (i.e. robot-driven) automotive future.  With that comes a completely different mix of risks, liabilities, safety concerns, responsibilities, ownership models, insurance platforms, and regulatory oversight. 

Car crashes are, and should be, a big deal. They are the number one reason for death among young people and number five overall, claiming over 32,000 American lives each year. Some news outlets have questioned the sanity of allowing driverless cars on the road all together. Fairly or unfairly, the whole notion of driverless cars is experiencing knee-jerk reactions. It is easy to see why the first known death caused by a self-driving car in the history has focused everyone’s attention on autonomous vehicles.

This incident is much like how Bridget Driscoll made the history in 1896 by being the first pedestrian being struck and killed by a gas-powered car (at a top speed of four miles per hour).   Thanks to the sensationalism of the press, the Florida crash got much more coverage in the news cycles than the more positive story about the Missouri man who used his Tesla Model X in autopilot mode to get to the hospital when he suffered from a debilitating blood clot on the highway.  Tesla Autopilot saved his life, and that is real (and good) news.  Nothing like that has happened before—a robot saving a man’s life.  Amazing.

These issues have made so much headlines that it made it to the white house, and president Obama wrote an op-ed mostly in support of the technology.  President Obama wrote that safer, more accessible driving, and less congested, less polluted roads are what harnessing technology for good can look like referring to self-driving car technologies. He also said that we have to get it right. Americans deserve to know they’ll be safe today even as we develop and deploy the technologies of tomorrow.

The accident has given rise to discussions about what types of sensors should have been in place to avoid that accident.  Also, as you would expect, there is a lot of questioning by legislators about the need for such technology, and how it can be regulated. A proper outcome of the crash has been awareness that autonomous driving is a public safety issue.  This is multi-faceted and includes technology (i.e. hardware, software, and architecture), economics, policy implementation, liability, and oversight factors.

I started following the development of autonomous vehicles when I first heard about Google’s so-called “self-driving car” project back in 2009. While I knew about DARPA’s initiative around this idea in mid 2000s, a commercial entity like Google picking up the project lends real credibility. Back in the 2010-2011 timeframe, my team and I were working on Freescale’s MCU strategies, and through that I got to understand the role of Active Driver Assistant System (ADAS) and the numerous architectural considerations and technologies needed to make autonomous driving a reality. 

Now at BlackBerry, I am working with our QNX software team on ADAS development. The QNX perspective, of course, comes from the software side with expertise in instrument clusters, functional safety, hypervisor  infotainment, and telematics. When you add that to  Certicom’s cryptographic security expertise,  and BlackBerry’s Over-the-Air (OTA),  updates for automotive security life cycle management, you have what you need for safety and security of the software-defined autonomous future. The evolution to connected autonomous vehicles is transitioning through different stages that in fact were defined by the U.S. Department of Transportation's National Highway Traffic Safety Administration. SAE has defined levels as well.  

                                                          

                                                           Source: NHTSA

Most car OEMs that we are working with have autonomous driving pilot programs in place. That is no surprise.  Even before the Tesla Autopilot accident, it was hard to open a technology magazine or website and not see a mention of self-driving cars and various pilot programs around the world. Cars are becoming cool again due to new technological evolution.   This is similar to how cellphones became cool in the early 2000s when the emergence of 3G made the notion of smartphones real.   Cars are much more than a phone, obviously, and the sky is the limit.  Software, semiconductor, networking, cryptography, sensors, communications, electric/hybrid engine, charging, display, augmented reality, smart highways, retail, and other technologies all converge on the car platform.  These things are quickly redefining the car, the highway, ownership models, insurance, and society itself. 
 
Some of the items to consider are the forms that vehicles will adopt due to automation, such as autonomous cars,  to  self-driving busses  , self-driving trucks,  and DARPA’s 132-foot long Sea Hunter unmanned Submarine-Hunter Drone .

Hardware + Software

Self-driving vehicles, or self-propelled anything, are based on an intimate relationship between electronics hardware and software to create not only a perceiving, processing, and actuating system, but a system that is safe, secure, and reliable.   While that last part seems obvious, it is not all that easy to accomplish.   Safety, security, and reliability come only from careful design based upon experience—experience that can make hardware and software work seamlessly.
 
Starting with the hardware, if you look at automotive microprocessors and microcontrollers, you can see that their complexity has skyrocketed to meet real time requirements of active safety elements such as vision processing, sensor fusion, and control algorithms, while still maintaining stringent power budgets.

Advanced driver assistance systems (ADAS) are the backbone of autonomous vehicles, obviously, and that it is based upon multiple application cores and hardware accelerators.   ADAS, software platforms must provide high performance by combining symmetric multiprocessing on application cores with support for built in accelerators such as vision processing engines or graphics processing units (GPUs).   Examples of applications range from four camera surround view systems, to a single camera forward facing collision avoidance system, to a sensor fusion hub.
Of course, the most important aspect of anything automotive is safety.  The old adage of safety first is still valid, and getting even more so as robotic cars start to drive themselves.  Therefore, there has to be real safety know-how at the core of the design and implementation of ADAS.  This is where safety standards compliance comes in.  The QNX Platform for ADAS is a great example of safety-centered software for the autonomous car.   The platform is certified by TÜV Rheinland to ISO 26262 ASIL-D.

More details will be addressed in a future blog, but are presented here to illustrate that software must be compliant with safety standards if it is to be taken seriously.  How safety is achieved by a software architecture is by ensuring that system faults in one area do not affect other areas.   This is accomplished by using a microkernel architecture the operating system (OS) to create isolation of failed components, and allowing them to be restarted dynamically while the rest of the system continues to operate. This type of adaptive partitioning technology safeguards the operation of the safety-critical components by ensuring they are never starved of CPU cycles. With a microkernal approach, traditional OS services can be contained in separate, hardware-protected address spaces in the same manner as applications.

The next blog will focus on the individual subsystems used in an ADAS platform in the connected autonomous car.    In addition, other connected autonomous car technologies will be covered in subsequent blogs, including security, Domain/Area-controller evolution, more about safety, and other technologies needed, plus use-case and financial considerations related to autonomous cars.   The story of the software-defined automotive future is just starting to be written.   For more see the QNX web site.

 

_____________________________________________________________________________

Kaivan Karimi is the SVP of Strategy and Business Development at BlackBerry Technology Solutions (BTS). His responsibilities include operationalizing growth strategies, product marketing and business development, eco-system enablement, and execution of business priorities. He has been an IoT evangelist since 2010, bringing more than two decades of experience working in cellular, connectivity, networking, sensors, and microcontroller semiconductor markets. Kaivan holds graduate degrees in engineering (MSEE) and business (MBA). Prior to joining BlackBerry, he was the VP and General Manager of Atmel wireless MCUs and IOT business unit.