Below is a listing of picturesque towns and villages in France which you may enjoy visiting. I did not list those towns that have either a significant castle or cathedral/church. For those lists you should go to Cathedrals and Churches in France and Châteaux and Castles in France.
Aigues-Mortes, Languedoc-Roussillon
Alésia, Bourgogne (Burgundy)
Andlau, Alsace
Arras, Le Nord
Aurillac, Massif Central
Avignon, Provence
Beaune, Bourgogne (Burgundy)
Colmar, Alsace
Compiègne, Le Nord & Picardie
Dijon, Bourgogne (Burgundy)
Domremy, Lorraine
Epernay, Champagne
Espalion, Massif Central
Estaing, Massif Central
Jura Mountains, Franche-Comté
Le Linge, Alsace
Nancy, Lorraine
Omaha/Utah Beaches, Normandie
Pointe du Hoc, Normandie
Riquewihr, Alsace
Rouen, Normandie
St.-Malo, Bretagne
Verdun, Lorraine
Wissembourg, Alsace
Vitré, Bretagne