Emerging Tech Inspired by Self-Driving Cars

Alternate Title: The Self-Driving ABCs

My boyfriend’s dad’s car was recently broken into. By itself, this would be a pretty low note to start a blog post with, but rest assured that nothing was stolen and only one window of the car had to be replaced. The situation was made 100 times better by the fact that he was lent a Tesla Model X while his own car was being serviced. And not just a basic Model X, either, but the P100D, the ultra luxury model complete with space-age wing doors that open when you approach the car, “ludicrous” driving mode that goes 0 to 60 in 2.8 seconds, and leather interior and heated seats. It was definitely the most expensive car any of us had ever been in.

We were especially curious to try out the Model X’s self-parallel-parking feature. We decided to try it to park in front of my boyfriend’s house, as the calm, residential street presented a pretty low-risk opportunity. We enabled the feature, instructed it to parallel park, and sat back in our seats to let it do its thing. But something was off. The car sure was angling out quite a lot for a parallel parking spot. And then it hit us — rather than parallel park, the car was trying to back into the narrow stone walkway through the front yard, thinking it was a driveway it could pull into! Thankfully we realized and stopped it in time, saving the nicely manicured groundcover and flowers around the walkway from certain death.

Everywhere I look, I see another article about Tesla or self-driving Ubers or the improvements/regressions of self-driving technology, but our experience with trying to self-park the Tesla Model X certainly got me thinking more about the state of self-driving technology in 2018. But rather than lecture you here about the news you are seeing everywhere else, I thought I’d cover something that has not had quite the same presence in the spotlight and talk to you about some emerging tech inspired by self-driving cars.

Now, I’m not at all an expert when it comes to self-driving tech or the AI or the hardware behind it. My goal is not to provide a comprehensive analysis of how the tech works but rather to share some interesting and… unique developments out there that you may or may not have heard about. My hope is that you will find at least some of them as creative or thought-provoking or entertaining as I did while researching them. As the alternate title suggests, I’m going to work through these concepts in alphabetical order, but I will not necessarily devote an equal amount of time to each idea.

Ambulances (and other emergency vehicles!)

“Now wait a minute,” you’re probably going. “These are just cars!” And you would be technically correct. But, the self-driving “problem” for ambulances and other emergency vehicles to solve involves a lot of added factors that makes the concept even more interesting and means we probably will not be seeing auto-driving emergency vehicles any time in the near future.

For one, regular drivers respond inconsistently and outright erratically to approaching emergency vehicles. Sometimes people move, sometimes they don’t. When they do move, they move all over the place depending on what they think “out of the way” means, how much time they think they have, how other people are reacting, and how likely they think they are to be in the way if they do not move. Sometimes people stop, sometimes they don’t. Sometimes people actively try to adjust to what the emergency vehicle is doing next, sometimes they just sit there passively and let the emergency vehicle work out how to proceed around them. Self-driving technology is no better: current technology does pretty poorly at predicting the movement of emergency vehicles and thus often makes uninformed decisions about how to react. While responding appropriately to emergency vehicles is already very challenging for regular self-driving vehicles, you can imagine the complexities a self-driving ambulance would face when responding to regular vehicles’ confusing responses.

The rules an ambulance follows differ substantially from the rules an everyday driver follow, in that ambulances are expected to break everyday driving rules. While a regular self-driving vehicle should err on the side of caution and adhere more strictly to the rules of the road in order to minimize risks to the driver and other drivers around it (essentially the “defensive driving” philosophy), emergency vehicles risk the lives of the passengers they carry if they drive too defensively or stop at every stop light like regular drivers do. Self-driving technology would need to learn to drive “more aggressively” without increasing the risk of ramming into other vehicles of obstacles.

Outside of driving, auto-piloted firetrucks or ambulances would need to be able to quickly identify the safest place to stop when arriving at the scene of an emergency that is also the closest to a fire hydrant or people who need help. There are multiple methods it could use to make this decision, be it using lots of additional data, requesting input from trained responders on board, or switching control to a remote driver who could guide the vehicle. But however it would work, the efficacy of self-driving ambulances would hinge on the ability for the vehicle arrive on the scene safely and effectively.

Understandably, self-driving technology has a ways to go before we can expect to see any real work done on self-driving ambulances or other emergency vehicles. But projects like Google’s Waymo cars are actively working to improve their AI’s understanding of how emergency vehicles behave by practicing on-site with local fire and police departments. They have also been making specific hardware improvements for this purpose, such as extending their audio sensors’ ranges and improving their directional precision to help provide earlier information about oncoming emergency vehicles.

Would people be comfortable being driven to the hospital in a self-driving ambulance? The important thing to remember here is that paramedics and other first-responders would not be removed from the situation entirely, they would just not be driving the vehicles anymore. The focus would be on equipping people on board the ambulance or firetruck to better prepare for the situation they are being taken to or offer more undivided attention to patients on board. With this in mind, hopefully by the time this technology hits the streets, people can be more at ease about the care they would receive on a self-driving ambulance.

Boats

It doesn’t take that much stretching of the imagination to conceive of self-driving boats, but the picture in your mind probably differs from the picture the people working on this tech are aiming for. We are not just talking about self-driving yachts, after all — we are talking about any kind of boat! Autopilot ferries, autopilot cruise ships, autopilot tugboats, even massive autopilot cargo ships! For virtually every class of boat there is someone out there trying to develop self-driving technology for its unique needs.

In a lot of ways, boats have an easier navigational task than cars. They have lots of wide open space to work with, free of pedestrians and bicyclists. A boat will just encounter way fewer other vehicles in general. The relative sameness of the task over the duration of the journey is, in a lot of ways, the perfect activity to automate. In fact, a lot of the most devastating maritime accidents are often attributed to just how calm and open the seas can be: even the most diligent of human crews tend to lose focus or not recognize a problem until it’s too late. Self-driving cruise or cargo ships could be monitored remotely, and crews could be waiting at either end of the journey to help with the more involved tasks of docking and undocking. This would enable on-board crews to focus on other tasks that need to happen, such as cleaning up oil spills, working to prevent oil spills in the first place, or actively handling other issues — a lot like the crew that remain on board a self-driving ambulance.

As for concerns, boats on autopilot with more limited crews would be more susceptible to piracy and outright hijacking. Additionally, public trust in autonomous vessels passing close to coastal settlements correlates strongly with the cargo contained in said vessels — an auto-piloted ship full of fruit is a lot less unnerving than an auto-piloted ship containing nuclear weapons.

Military entities have been working on unmanned maritime vessels for close to a century, but the idea only reached the commercial scene more recently. Several start-ups have signed deals with undisclosed companies in a race to put out the first autonomous commercial containerships; one competitor, oddly enough, is a Norwegian fertilizer company! To put into perspective how lucrative this business has the potential to be here are some statistics.

• The Port of Oakland is a pretty large port (one of the largest in the US, its exact rank depending on the year); it handles ~2.5mil containers a year
• This pales in comparison to the Port of Shanghai, which handles nearly 40mil containers a year
• It takes between 15-25 days for a ship to make the journey from Shanghai to Oakland

Imagine how many man-hours crews spend at sea navigating this path every year! China is certainly imagining it — they are reportedly planning on having the first self-navigating cargo ships set sail in just three years.

Rolls-Royce has been experimenting with remote-controlling boats from super sci-fi-esque controls rooms with 360 degrees of screens and an operator’s chair at the center. They imagine these rooms could also serve as monitoring stations for unmanned vessels. Their expectation is that self-steering ships will be introduced to most fleets within 10 years, and they are working on the technology to get there themselves. The company has created an AI navigational system to operate in medium-to-large vessels, but the tech is still in development, so its practical viability is yet to be seen.

A Dutch big data company called Xomnia built the “first” self-driving boat as an internal company challenge using an old lifeboat, some standard computer webcams, a deep-learning network developed in-house, and only a minuscule budget. You can catch their boat navigating itself around the canals of Amsterdam today! The project’s apparent success and popularity attracted the attention of a major Dutch shipping company that they have since partnered with, working to outfit some of their mid-size container ships with the same self-driving technology.

MIT researchers are also getting in on the fun. They have been developing 3D-printable autonomous boats that were also being tested in Amsterdam. These unique vessels can physically attach to each other to assemble into structures such as docks and bridges. Another motivation for the research is alleviating traffic congestion on the streets by bringing more public services into the water; the 3D-printed boats could be used for deliveries or waste management in cities built around water networks, removing a sizable portion of trucks and garbage vehicles from the streets.

Delivery Bots

You are probably already familiar with this smaller take on self-driving technology. Delivery bots are short distance, small units with wheels, insulated compartments, user-friendly lighting and screens, and sensors to help overcome sidewalk obstacles, cross intersections by reading traffic lights, and most importantly, not run into people! The bots are becoming pretty popular in urban areas, with companies like Domino’s, DoorDash, Yelp Eat24, and many more all trying to out-deliver the others.

Though most robots are still being tested with chaperones, robots in certain well-tested areas, such as at Intuit’s campus in Mountain View, are now driving solo, only needing to ping their remote monitors if they encounter unforeseen construction or an unmarked crosswalk. But unmanned delivery bot sightings are very close to becoming a common occurrence. The biggest obstacle stopping these from making a larger appearance on city sidewalks is not technical — it’s legal. As with the recent legislative issues concerning the scooters popping up everywhere in downtown San Francisco, many groups are voicing concerns about these bots clogging sidewalk traffic, creating obstacles for handicapped folks, or disrupting regular businesses. Until the proper guidelines and authorization procedures are fully established, delivery bots will remain relegated to very specific testing locations.

While the robots are certainly susceptible to theft or damage, most are equipped with two-way audio and cameras if human intervention with an attacker becomes necessary. The bots food-carrying compartments are locked and supposedly inaccessible until the delivery recipient presents a code from their delivery app or another means of identification. But the bots are not all that heavy, so you have to wonder if the locked compartments would really be enough to stop someone or several someones who are really hungry.

Fridges

Speaking of being really hungry, how many times have you felt the stomach rumbles but struggled to find the motivation to leave the comforts of your couch or bed to retrieve food from your refrigerator? Panasonic heard your first-world whining and tasked a team of skilled engineers to build you an icebox on a glorified Roomba. This one really needs no explanation other than the following video.

Does it actually serve a practical purpose? I would certainly hope so, considering this project probably received more funding than most university-level research projects do in a year. The stated goal for its design was to provide an autonomous means of acquiring perishable food for elderly, handicapped, or otherwise bedridden individuals who have trouble getting to the refrigerator on their own. While the product is not currently commercially available, it was tested at length at Panasonic’s own offices, delivering cool beverages and heated catering (the top of the refrigeration unit also featured a heating plate) to various business meeting attendees.

In my opinion, the most interesting thing about a self-driving fridge wouldn’t be the fact that it’s self-driving, but rather the fact that we would suddenly have all this data available from it about what we eat and when we eat it!

Grocery Stores

Several engineering start-ups are also looking to capitalize on the self-driving refrigerator’s concept of bringing the food to you, but with a slightly different approached. Their idea is to mass-produce specialized self-driving cars that grocery retail chains could lease as an alternative to setting up a whole new store.

The No. 1 reason consumers do not typically use online grocery delivery services is that they want to see/feel/smell the groceries before they pick them. Online grocery shopping is the one area of online shopping that has yet to take off, but it is steadily growing in popularity with services like Instacart and AmazonFresh, as well as many more meal-based delivery services.

But many companies are actively working to overcome this dilemma by turning to grocery stores on wheels. Larger retailers like Amazon, Walmart, and Safeway have already begun looking into self-driving grocery store technology in order to gain the edge in the online grocery market before its projected takeoff.

Leasing self-driving mobile grocery units would usually be cheaper than setting up a new physical store, and the costs of maintenance are also fairly comparable. Units could even alleviate traffic during shopping rush hours such as directly before and after work and in the afternoon on weekends.

A company called Robomart is testing its grocery vans in Santa Clara. Like Uber, the service provides a smartphone app that allows consumers to call for an autonomous grocery vehicle the same way they’d call for a car from a ride-hailing service. Upon arrival, the app would unlock the vehicle’s doors, and consumers would be able to select their desired products. The vehicle would use a proprietary checkout process similar to that of the Amazon stores, tracking consumers’ selections with cameras and sensors and charging them accordingly when they are finished. Such a system optimizes for getting the vehicles between shoppers as speedily as possible, but it also makes the transaction faster and easier for the consumer.

Another company, Wheelys Cafe, is testing 24-hour, solar-powered, mobile shops called Moby Marts in Shanghai, the current world leader in adopting mobile payment technologies. Although self-driving vehicles are not yet permitted on the streets there, the grocery unit promises the ability to not only drive itself but also restock itself as soon as the city will let it. The Moby Marts are more spacious and traincar-like than the Robomarts — consumers actually enter them through doors and can comfortably walk around and browse their shelves once inside. If the Robomarts are the Ubers of autonomous grocery vehicles, Moby Marts are the food trucks.

Autonomous grocery units present a particularly appealing opportunity to both the most dense and the least dense of inhabited areas. Being both cheaper and mobile, units like the Moby Marts could allow neighbors from a community to easily open their own local grocer without needing to find the space (in the case of dense, high-rent areas) or a large customer base (in the case of rural, low-traffic areas).

One of the biggest challenges for self-driving grocery stores will be restocking. It is essentially a very complicated optimization problem where the goal is to maximize the amount of time the grocery stores spend on the road and the amount of goods each unit has available for consumers to purchase.

Take an example: say we have two units, Unit A and Unit B. Unit A runs out of milk at the same time that Unit B runs out of eggs. Rather than send both units back to the restocking facility, we could optimize by having the units meet each other so that Unit B can transfer milk to Unit A. That way, only Unit B has to return to the restocking facility, and Unit A can resume accepting customers with both products available.

But what if Unit B ran out of eggs just a couple minutes later, and Unit A was already on its way back to a restocking facility or to meet up with another third unit? Or what if Unit B runs out of a less popular product like spam but still has eggs — should it still attempt to coordinate a transfer or return to a restocking facility? It’s a very interesting optimization problem with a ton of different factors to consider!

Homes

Here we have a concept that simultaneously feels like it is out of the 1970’s AND the 2070’s. Honda has been thinking about all that wasted time that cars are spent parked at their owner’s homes and wants to do something useful with that time. Their conclusion? Why not turn the car into an extended part of your house or apartment?

Honda leMobi is a mobile, autonomous living room that connects seamlessly to the wall of your house. It comes complete with chairs that can move between the car and the home, and its controls could transition from driving controls to home automation controls automatically. It epitomizes a flexible, mobile, multipurpose space. Honda built a full-scale proof-of-concept that they showed off at a Tokyo Motor Show last year, and it really does connect seamlessly with the rest of the house. The idea may sound silly or far-fetched, but this might actually not be too far off from what we can expect of the future. Young, urban professionals, especially, are on the move for so much of the day that some people speculate that self-driving, automated RVs could become viable homes of the future. Even in its very early stages, Honda shows us that they are already preparing for that day.

Lawn Mowers

Self-driving lawn mowers are quite similar to the refrigerators in that they are essentially just oversized Roombas for your lawn, but these robots are actually available for purchase from most online marketplaces and home improvement outlets. Interestingly, although Honda lags behind most other motor vehicle companies in self-driving technology, they have produced their own line of autonomous lawn mowers are their only “real” self-driving product right now (as we cannot really count the leMobi homes… yet).

One thing that makes self-driving lawn mowers more interesting than regular Roombas is the added challenge of preventing your robots from going rogue and escaping your lawn to potentially ravage neighbors’ yards. Typically, these lawn mowers are restricted to their lawn areas with specially-installed wires that are buried underground and mark the area’s perimeter. The lawn mowers are then equipped with sensors to detect and respond to these electrical walls as Roombas would with the physical walls of your home.

Machine Guns

:(
As with any emerging technology, someone somewhere will find a way to repurpose it for war. Self-driving tech is no different. The future of war will almost inevitably be fought by autonomous machines deployed and monitored remotely by humans. The remote-controlled guns are the easy part of this, so it’s only a matter of time before weapons are augmented with autopiloting capabilities.

NASCAR

Who would win: an AI-driven race car or pro-driver RYAN TUERCK? Assuming they are both controlling vehicles with the same specs, of course, let’s take a moment to consider it.

VS.

The pro-driver would win, obviously! Come on, our Tesla Model X couldn’t even properly distinguish a garden path from a driveway! But the AI-driven car was not entirely crushed: Ryan Tuerck’s best lap time was 1:51, and the AI-driven race car’s was only 2:18. And this gap is steadily closing. Roborace’s all-electric DevBot weighs about 2,200 pounds, boasts 550 horsepower, and is powered by an NVIDIA Drive PX AI. When preparing for a race, the car first takes an exploratory lap where it gathers data and maps out the course it will follow during the real race.

Most people think it will still be a while before we see full races of high-intensity self-driving cars, but the biggest obstacle might not even be a technical one — Roborace and other companies must first convince the public that racing can still be exciting without flashy human drivers behind every wheel.

Office Chairs

In line with other auto companies demonstrating their self-driving technology in more contained, mundane settings, Nissan has rolled out a line of autonomous office chairs that park themselves at the table in response to clapping.

The chairs apparently use a strategy similar to what their Leaf cars will use to park themselves, although the example feels a bit contrived. The chairs hook up to cameras positioned around the office with wifi so that the main points on the chair can be identified to steer it back into place, tools that will obviously not be available to Leaf cars when they try parking. But the chairs are still cool to see in action!

Unfortunately, Nissan’s chairs are not a real product for sale and were only created as a demonstration of their parking technology. But like other concepts in this collection, the idea fits the depiction of a lazy, fully-automated future predicted by such fictional media as Wall-E and other sci-fi stories. One farcical article I found provided amusing illustrations of an office chair that follows alongside its absent-minded owner, moving into the perfect sitting position for them to lean back anywhere, anytime, and connecting to their personal calendar to lead them to their next meeting room (or certain death, but hopefully the former) so that they can respond to Slack messages as they walk. I would love to see those in action at Grio!

Planes

Airplanes may just features the oldest self-driving (or should I say, self-flying) technology. Original airplane autopilot technology used gyroscopes and altitude indicators to help stabilize the plane during flight. More advanced autopilot systems, a big part of which are called Flight Management Systems (FMS) have been an essential part of airplanes for years. These days, FMS and other auto-piloting features handle the bulk of work pilots needs to do. By constantly monitoring the plane itself and current flying conditions, autopilot systems can make decisions about when to adjust the speed, what the rate of climb should be, and other factors with amazing precision.

Pilots are still needed to monitor weather conditions, track fuel consumption, manage navigation and communication, and of course take control during severe turbulence and other situations, but autopilot systems can even perform some of these functions, now, too. Over 90% of the duration of most standard passenger flights are flown with the autopilot engaged, primarily leaving takeoff, landing, and taxiing for manual control.

Of course, much of what autopilot does is respond to instructions from the pilots, so it’s not like you just plug in your starting and ending points and it’s good to go. It’s more complicated than that – much more complicated, in fact! In my opinion, for any type of self-driving technology, it seems that striking a balance between the automatic and the manual is absolutely critical: reducing manual control reduces likelihood of human error, but if an autopilot system fails and the crew does not know how to take control, the result can be catastrophic and deadly.

I think it’s equally critical, then, that we improve these interfaces between the automation system and the human pilots to ensure taking manual control in an emergency is foolproof. The autopilot systems for most modern commercial aircraft have not seen much, if any, recent development as far as interfaces go. The system is not exactly intuitive to operate and requires lots of experience to master. Human misinterpretation of the display is not uncommon, especially in novice pilots, and can lead to poor navigational choices.

Boeing has recently announced advances in their AI technology for autonomous plane flight and acquisitions of smaller autonomous flight companies. They will likely ship this technology with cargo planes before passenger planes. The turning point for full self-flying AI in planes may actually be a shortage of pilots in the coming ~20 years, some say. But it is still going to be some time before such systems are perfected, and even longer until the authorities and the public are comfortable enough to remove pilots entirely. But the technology is well on its way, and Boeing is on an accelerated path to get there.

Potatoes

We’ve all seen the grade school science experiment where sticking a couple of electrodes into a potato produces enough current to power a small light bulb, but one guy took the experiment to the next level. By having that current charge a battery, putting the potato on wheels, and having it take off in a somewhat random direction once the battery was charged, he was essentially able to engineer his own autonomous house pet, which he aptly named Pontus the Potato.

The potato required sitting stationary for a quarter of an hour to charge its battery and move a few inches, but then again, does a sleepy cat move any more frequently than this on average?

Slippers

If Nissan’s frivolous, self-driving office chairs gimmick wasn’t enough, they upped their game again in February with self-driving slippers that tidy themselves away to a designated location with the push of a button, no matter where you leave them in the house. Alongside self-straightening tables and remotes and self-driving floor cushions, apparently, the slippers debuted and are exclusively available at ProPILOT Park Ryokan, a pop-up hotel in Hakone, Japan, which opened in March!

Like the office chairs, the slippers are hooked up to cameras and wifi in the hotel and are equipped with tiny wheels, motors, and sensors. While the slippers are not yet commercially available, who knows, there could be a huge market for these with parents of teenage children with messy bedrooms!

Vacuums

Before I finish this blog post, I have to give a shout out to everyone’s favorite autonomous cleaning robots, the Roombas — and in particular, this cat in a shark suit on top of a Roomba being chased by a duckling, because this probably isn’t even the weirdest concept in this post!