Fear of driving is a common and troubling problem, but it's a problem you can solve. The first step is to identify the specific nature of your driving fear.
People who experience panic attacks are often afraid of having one at the wheel. They're licensed to drive and generally have a good driving record. They probably drove without any problems, maybe for years, until they started having panic attacks, and then they became afraid.
Their fear isn't really about driving. It's about panic attacks while driving. They worry that they won't drive safely during a panic attack, and that they won't be able to flee the scene as quickly as they want.
They often still do some driving, by limiting where and when they drive. They avoid highways. They stay in the right lane, where it's easier to pull off the road. They avoid bridges, tunnels, and any other road conditions which limit their ability to leave the scene as quickly as they wish, in the event of a panic attack. They avoid left turns, unless there's a traffic signal with a turn arrow, because they fear getting "trapped" in the intersection. They avoid driving at rush hour.
They also use distraction and various safety objects. They may sing, or call people on their cell phones, to distract themselves from the thoughts of a panic attack. They may drink from a water bottle, and feel reassured by this. They may only drive when accompanied by a "safe" person.
People who experience this type of driving fear learned to drive and received their license, but they often didn't get much experience after that, and lack confidence in their driving ability. Unlike the first group, they probably don't experience panic attacks outside the driving situation.
They may worry about accidents, but also worry that they'll drive so poorly that they'll attract negative attention, and worry further that this negative attention will make them even more afraid. With this type of driving phobia, you worry that you'll drive too slowly, or that you won't move quickly enough when the light turns green. You fear that the driver behind you will get impatient and honk his horn.
You also worry that you'll become more anxious when this happens, maybe so much that you stall the car, or make some other mistake. In your mind, this will quickly turn to a terrible scene of hopelessly snarled traffic, the loud din of horns honking, and a large crowd watching you fall apart behind the wheel as paramedics and policemen rush to your vehicle.
You might suppose there would be a third group of people with driving phobia. These would be people who drive so poorly that they have a history of accidents and traffic tickets. However, this is not really a phobia. A phobia is an irrational fear. If you are a reckless or incompetent driver, it's good to avoid driving until you get some retraining.
In my experience, however, the people who have come to me for help with a fear of driving fall into one of the two categories listed above. They don't have an actual history of harmful results from their driving. They just worry about that.
If your fear of driving is mostly about panic attacks, the way to solve this problem is to learn how to better handle panic attacks. If you have panic attacks in situations other than driving, it might help to start with a situation in which your role is more passive - perhaps waiting in a long line, or walking through a crowded mall. You can then work with driving after you've made progress in the more passive role first.
If your fear is more like the performance anxiety described above, you need a similar progressive practice with driving in which you can practice accepting your fearful thoughts and feelings while doing the work of driving. You can use the same techniques as the person with panic attacks, because the underlying fear, of becoming so afraid as to be unable to drive, is similar to the fear of losing control to a panic attack. The main difference in the thoughts with performance anxiety is the extent to which you worry that others are watching, and judging, you.
My Panic Attacks Workbook has a chapter devoted to the fear of driving.
Want to read it? You can download a free copy for your own personal use.