When I say we should worry about self-driving cars, I don’t just mean worry about getting hit by one—though that’s definitely part of it too. I mean “worry” as in, “We really need to think about this.” Autonomous cars offer a lot of advantages, but they also come with some pretty serious problems. We need to start thinking about those problems now, because even though the rise of autonomous cars has been very gradual so far, it is pretty clear that as the technology improves, such vehicles will become more and more common. Someday they may even “take over,” becoming the standard way to drive (or be driven).

This is no longer a distant, futuristic scenario. Car companies are racing against each other to develop autonomous driving technology. For instance, Volvo says a third of their cars will be driverless within the next decade. Some national governments are funding research and development of self-driving vehicles. The United Kingdom’s Department for Transport predicts that the first completely autonomous vehicles will be on British streets by 2021, and that autonomous vehicles will quickly become a normal part of life (and traffic) there. In the United States, self-driving vehicles are already being tested on public streets. Some cars that are commercially available to the public already incorporate certain elements of autonomous cars, like automatically slowing or stopping before hitting obstacles, or making sure a car stays in its lane if the driver lets it veer off. Therefore it’s not that big a leap to imagine fleets of automated taxis replacing human operators in the near future.

There are definitely some potential advantages to autonomous cars. A lot of deaths and injuries on the road occur because people drive while intoxicated. Accidents like these would theoretically happen much less frequently in cars with autopilot turned on. Self-driving vehicles also never get distracted or sleepy. They don’t text while driving, or get mad at other drivers. They simply won’t make human errors. Eliminate all of these problems and you can imagine how quickly accidents might decrease! Furthermore, if all cars were autonomous, there would be an increase in organization on all roadways. In crowded metropolitan areas, commute times would drop and there might even be less traffic on the streets. Autonomous cars can work together to find the fastest routes for everyone involved.

However, there are still problems to worry about. First of all, there is the quality of the technology. We don’t want to start depending too much on autonomous cars before they are fully reliable. Earlier this year, a woman in Arizona was killed by a self-driving car, and other people have been killed or injured while using autonomous features in their cars. Obviously autonomous driving technology still needs to improve before we can fully trust it with our safety.

Part of the problem is also human beings. As driverless technology develops, we will be in a period of transition for a while. People don’t always know what to expect from this technology, and sometimes they expect too much. After all, if you are riding in an “automated” vehicle, it seems like you shouldn't have to do anything, right? Yet people still need to be ready to take control away from automated systems when they encounter situations for which these systems are not prepared. In the Arizona crash, there was a driver in the car but she did not have her eyes on the road. In another fatal incident, the driver did not have his hands on the steering wheel while his car was using the autopilot feature. To use self-driving cars safely, we have to understand their limits.

Even if the technology is perfected, however, there will still be some tricky questions to figure out. For instance, does “perfected” mean that autonomous cars need to be as safer than human-driven ones? If so, how much safer do they have to be before we allow them on the road?

What about the actual definition of road safety? Imagine you are in a driving situation where you or another driver skids out of control. In a split second, you might have to decide which way to turn your car. One direction might be safer for you and your passengers, but would endanger nearby pedestrians or other drivers. Another direction might be more dangerous for you, but less dangerous for others. Is it ethical for you to want your self-driving car to “look out” for you first even if it means others might take more of the impact? Or, should your car communicate with other autonomous vehicles involved in the impending crash to choose a maneuver that leads to the fewest injuries or deaths overall?

If autonomous cars do get into accidents (which seems inevitable), who is legally to blame? The car’s owner? The company that manufactured the car? Finally, what effect will autonomous cars have on the way we live? Will the rise of this technology increase our car use, and so possibly also the pollution we produce? Will it change the ways we design whole cities? Who will be affected by those changes, and how?

Obviously I don’t have the answers to all these questions—but that’s my point! These are things we all need to think about. We need to figure out the rules for self-driving cars before they become the norm. It’s better to worry now than later, before unexpected problems pop up and we get caught unprepared.