Everything but the kitchen sink

Using a single SoC to drive a full-blown infotainment stack, 3D surround view, driver monitoring, smartphone connectivity, and dual HD displays.

TI and QNX have been working together in the infotainment space for a long time. The nice thing about this ongoing relationship is that lots of cool technology gets built along the way. Speaking of which, TI have put together a compelling demo that they will show at the TU-Automotive Detroit conference, on June 8 and 9. I’m pretty pumped about it and invite you to take the time to check it out.

The demo is built on the DRA75x (Jacinto 6 EP) SoC, which includes dual ARM Cortex-A15 processors, an Imagination SGX544MP2 GPU, dual TI C66x DSPs, and an IVA-HD video processing core. For starters, it runs the QNX CAR Platform for Infotainment with navigation, multimedia, speech recognition — all the goodies you’ve come to expect. Not surprising, as the platform has been running on Jacinto 6 longer than on any other SoC.

One SoC, two displays, many applications

It’s what they’ve managed to pile on beyond the QNX CAR Platform that makes this demo so exciting. You may not know it but we also work closely with TI on the informational ADAS (infoADAS) front. There’s a full port of the InfoADAS stack available today on the QNX platform and it’s included in the demo. Taking 4 camera inputs and the processing power available on the C66x and SGX, TI can demonstrate full 3D surround view concurrently with everything else. If that wasn’t enough, they’ve added a fifth camera and partnered with FotoNation to add driver monitoring and identification, which runs on the second C66x.

Normally, the smartphone projection runs on ARM, but for this demo, it runs on the IVA-HD to further demonstrate the capabilities of the chip. And to top it all off, the demo drives two HD displays. One display shows the QNX CAR Platform and the other shows the 3D surround view, along with the driver monitoring and identification.

So, to summarize, on one dual-core ARM A15 part, TI is showing a full-blown high-end infotainment system, driver monitoring, the ability to see everything around the car in real time, and the ability to connect to pretty much any smartphone in the world. Take a second to think back to just 5 years ago. It’s amazing how fast this industry moves.

If you aren’t going to be at TU this year, reach out to TI. I’m willing to bet they’d be happy to show it to you…

Why is software the key to bringing augmented reality to cars?

Guest post by Alex Leonov, marketing director, Luxoft Automotive.

While self-driving vehicles are gradually becoming a reality, more and more of today’s cars roll out from factories featuring advanced driver assistance systems (ADAS). We are quickly getting used to adaptive cruise control, blind spot monitoring, parking assistance, lane departure warning, and many other features that make driving safer and the driver’s job easier. Data from cameras, sensors, and V2X infrastructure feed into ADAS systems, increasing their accuracy and efficiency. These systems are important steps toward fully autonomous driving, but the ultimate responsibility for decision making still lies with a driver.

The more that cars become connected, the more the average driver can be bombarded by information while driving. “In 500 feet make a right turn.” “You have an incoming call from Christine.” “You have a new message on Facebook.” “You are over the speed limit.” This may not be so big of a distraction under normal conditions. But sometimes, when driving in hectic city traffic or in a snow storm, it is critical to keep eyes on the road, while still receiving essential information. The good news is, the technology is already there to remedy this.

Heads up for HUDs
Keeping the driver’s eyes on the road is a priority, and head-up displays (HUDs) can accomplish just that. They project alerts and navigation prompts right on the windshield. Analysts predict an explosive growth of HUDs with the market reaching close to US$100 billion by 2020. The bulk of HUDs are relatively simple combiners, but more advances in wide-field-of-view HUDs are coming soon.

Projecting alerts and navigation prompts directly on the windshield.

HUDs are perfect for presenting information in a convenient, natural way, and giving the driver a feeling of being in control. But HUDs are only as good as the information they display. That is why it is critical to have solid and reliable data processing and decision-making algorithms, running on a reliable OS, that can prioritize and filter data. The resulting alerts and prompts must be communicated to a driver in a clear, transparent way.

Computer vision, also known as machine vision, is a key to processing the endless flow of data. With its human-like image recognition ability, computer vision processes road scenes, and the system fuses data from multiple sources. Add in a natural representation of processing outcomes in the form of augmented reality, while tracking driver’s pupils, and you have a completely new level of driver’s experience — safe and intuitive.

Next-generation driving experience
At Luxoft, we’ve been working on making this experience a reality. The result is CVNAR, a computer vision and augmented reality solution. CVNAR is a powerful software framework containing mathematical algorithms that process a vast amount of road data in real time to generate intuitive prompts and alerts. CVNAR has built-in algorithms for road and pedestrian detection, vehicle recognition and tracking, lane detection, facade recognition and texture extraction, road sign recognition, and parking space search. It performs relative and absolute positioning and easily integrates with navigation, the map database, sensors, and other data sources. A unique feature of CVNAR is its extrapolation engine for latency avoidance.

Detecting and recognizing road signs, pedestrians, traffic lanes, gas stations, and other objects.

CVNAR works perfectly with LCD displays and smartglasses, but it is ultimately built for HUDs. Data from cameras, sensors, CAN, and navigation maps are fused and processed to create an extendable metadata output that describes all augmented objects. It takes a HUD and an eye-tracking camera to implement CVNAR in a vehicle. CVNAR will track the driver’s gaze and adjust the position of the augmented objects in the driver’s line of sight to make sure they don’t obstruct anything important — all in real time.

Alerting the driver to an empty parking spot.

This is not all that CVNAR can do. New car models come packed with infotainment features that take time to learn and memorize. The CVNAR-based smartphone app can help. It turns your smartphone into an interactive guide. Point your phone camera to your dashboard and use augmented prompts to find out more about a particular car function. It can work under the hood, too.

Era of a software-defined car
A modern car runs on code as much as it runs on gasoline (or a battery-powered electric motor). Today, it takes over 100 million lines of software code to get a premium car going, and the amount of software necessary keeps expanding. At Luxoft, we are excited about the car’s digital future, and we work every day to help bring it about, by developing cutting-edge automotive solutions for leading global vehicle manufacturers.

Offering a wide range of embedded software development and integration services for in-vehicle infotainment and telematics systems, digital instrument clusters, and head-up displays, Luxoft has developed User Experience (UX) and Human Machine Interface (HMI) technology for millions of vehicles on the road today. We push the envelope of technology in such areas as situation-aware HMI, computer vision and augmented reality, while Luxoft’s products, the Populus and Teora UX and HMI design tool chains, power the development of award-winning automotive HMIs and slash time to market.

Software holds the key to the future of cars. It is essential to creating a customized user experience in vehicles. With over-the-air updates, software offers unmatched flexibility and scalability. Finally, it takes safety to the next level with its ability to simulate human-like logic through complex algorithms.

You can view Luxoft’s CVNAR solution running on a QNX-based ADAS demo this week at CES, in the BlackBerry booth: LVCC North Hall, #325.

---

About Alex
Alex Leonov has been in the automotive and IT industry for over 18 years in various business development and marketing roles. Currently, Alex leads the global marketing efforts of Luxoft Automotive.

Video: Paving the way to an autonomous future

Lynn Gayowski

CES 2016 is now underway, and our kickoff to the year wouldn’t be complete without a behind-the-scenes look at the making of our new technology concept vehicle and updated reference vehicle.

The video below follows the journey of building our vehicles for CES 2016 and highlights the technologies we’re using to speed progress towards automated driving — and the list of tech that QNX covers is impressive! It includes advanced driver assistance systems (ADAS), V2X, and augmented reality, not to mention digital instrument clusters, in-car communication, and infotainment:



QNX Software Systems continues to innovate in automotive, with a vision for the evolution of automated driving and a trusted foundation for building reliable, adaptable systems. At risk of giving away the big finale, I think John Wall, head of QNX, sums up perfectly what QNX is on target for in the automotive industry: “We will dominate the cockpit of the car.” It’s a bold statement but we’re already amassing some imposing stats that back this up:

What if…

Imagine if your car could help you become more connected to friends and family — and to the road ahead. Enter a new video that peers into the not-so-distant future.

It blows my mind, but some people still see connectivity in the car as the enemy. They think that, the more connected the car, the more distracting and dangerous it will be. But you know what? Responding to their concerns is easy. I simply ask them what if.

For instance, what if connectivity helped you drive with greater situational awareness? What if it helped you sidestep traffic jams and axle-busting pot holes? What if it helped you detect a stop sign hidden behind a tree? And what if it helped you become more connected to the people important to you, as well as to the road and the cars around you?

When we talk connectivity at QNX, that’s the kind of connectivity we envision. It isn’t just about Bluetooth or Wi-Fi or LTE — that’s only the plumbing. Rather, it’s about keeping you in tune and in sync with your car, your environment, your business, your friends. Your life.

Hanging out with the cool kids at the EcoCAR booth

If you read Jin Xu's post earlier this week, you are already up to speed on the EcoCAR 2 competition established by General Motors and the U.S. Department of Energy. If you didn't read it, here's the skinny: To drive home with first prize, university teams must reduce the environmental impact of a 2013 Chevy Malibu without compromising performance, safety, or consumer acceptability. If that sounds hard, it is. Which explains why, out of 150 university teams that applied to compete, only 15 made the grade.

Today, at SAE Convergence, I was lucky enough to meet two of the talented young people participating in this competition: Ahmed Uddin from Wayne State University and Andrew Palmer from Ohio State University. Ahmed and Andrew had just finished delivering remarks at the EcoCAR booth when they stopped to chat with me about their projects.

The EcoCAR 2 Chevy Malibu

The Wayne State team dub themselves the Hybrid Warriors, and they are modifying the Malibu with a parallel-through-the-road PHEV. In a nutshell, the modified Malibu has two power trains, with an electric motor in back and a 2.4L engine in front. By taking this parallel approach, the team has actually upped performance, even though they replaced the stock engine with a power plant that cranks out less power, takes up less room, and puts out fewer emissions. Before these modifications, the car went from 0 to 60 in 9.5 seconds; now it takes only 8.9 seconds.

Meanwhile, the Ohio State team has opted for a series-parallel PHEV that uses an electric motor for the rear axle and a 1.8 L engine for the front. The systems can operate in charge-depleting, charge-sustaining series, and charge-sustaining parallel modes. Personally, I was fascinated by Andrew Palmer's description of the team's infotainment system (redesigning the center stack is an optional component of the EcoCAR 2 competition) and how they aim to make phone connectivity more seamless.

Cooler yet, the team is working on augmented reality, using a BlackBerry PlayBook. Picture this: You hold a PlayBook over the engine of your car, and the screen overlays a transparent view of the engine. The possibilities for this kind of functionality are enormous, and I invite you to check out two blog posts (here and here) from another Andrew — QNX's Andrew Poliak — for examples of how augmented reality could pimp your next ride.

Before you go, remember to follow @QNX_Auto on Twitter, where I will continue to tweet out reports from SAE Convergence.

Pimp your ride with augmented reality — Part II

Last week, I introduced you to some cool examples of augmented reality, or AR, and stated that AR can help drivers deal with the burgeoning amount of information in the car.

Now that we’ve covered the basics, let’s look at some use-cases for both drivers and passengers. Remember, though, that these examples are just a taste — the possibilities for integrating AR into the car are virtually endless.


AR for the driver
When it comes to drivers, AR will focus on providing information while reducing distraction. Already, some vehicles use AR to overlay the vehicle trajectory onto a backup camera display, allowing the driver to gauge where the car is headed. Some luxury cars go one step further and overlay lane markings or hazards in the vehicle display.

Expect even more functionality in the future. In the case of a backup camera, the display might take advantage of 3D technology, allowing you to see, for example, that a skateboard is closer than the post you are backing towards. And then there is GM's prototype heads-up system, which, in dark or foggy conditions, can project lane edges onto the windshield or highlight people crossing the road up ahead:



AR can be extremely powerful while keeping distraction to a minimum. Take destination search, for example. You could issue the verbal command, “Take me to a Starbucks on my route. I want to see their cool AR cups”. The nav system could then overlay a subtle route guidance over the road with a small Starbucks logo that gets bigger as you approach your destination. The logo could then hover over the building when you arrive.

You'll no longer have to wonder if your destination is on the right or left, or if your nav system is correct when it says, “You have arrived at your destination.” The answer will be right in front of you.

AR for the passenger
So what about the passenger? Well, you could easily apply AR to side windows and allow passengers to learn more about the world around them, a la Wikitude. Take, for example, this recent video from Toyota, which represents one of the best examples of how AR could make long road trips less tedious and more enjoyable:


 

Pimp your ride with augmented reality — Part I

The use of electronics is exploding in automotive. Just last week, Intel proclaimed that the connected car “is the third-fastest growing technological device, following smartphones and tablets.”

Ten years ago, you’d be hard-pressed to find a 32-bit processor in your car. Now, some cars have 4 or more 32 bitters: one in the radio, another in the telematics module, yet another in the center display, and still another in the rear-seat system.

Heck, in newer cars, you’ll even find one in the digital instrument cluster — the QNX-powered cluster in the Range Rover, for example. Expect to see a similar demand for more compute power in engine control units, drive-by-wire systems, and heads-up displays.


The Range Rover cluster displays virtual speedometers and gauges, as well as warnings, suspension settings, and other info, all on a dynamically configurable display.

What do most of these systems have in common? The need to process tons of information, from both inside and outside of the vehicle, and to present key elements of that data in a safe, contextually relevant, and easy-to-digest fashion.

The next generation of these systems will be built on the following principles:

• Fully integrated cockpits — Vehicle manufacturers see system consolidation as a way to cut costs and reduce complexity, as well as to share information between vehicle systems. For instance, your heads-up display could discreetly let you know who is calling you, without forcing you to take your eyes off of the road. And it could do this even if the smarts integrating your phone and your car reside in another cockpit component — the telematics module, say.
   
• Augmented reality — With all of the data being generated from phones, cloud content services and, perhaps more importantly, the vehicle itself, presenting the right information at the right time in a safe way will become a major challenge. This is where augmented reality comes in.

Augmented reality is a cool use of cameras, GPS, and data to create smart applications that overlay a virtual world on top of the real world. Here are some of my favorite examples:

AR Starbucks cups — Use your phone to make your coffee cup come alive:



AR Starwars — Blast the rebel alliance squirrels!



AR postage stamp — Add a new dimension (literally) to an everyday object:



And here are a couple more for good measure:

AR ray gun — Blast aliens around the house!

Wikitude AR web browser — Explore the world around you while overlaying social networks, images, video, reviews, statistics, etc.

Stay tuned for my next post, where I will explore how AR could enhance the driving experience for both drivers and passengers — Andrew.