No, you don't have to at all, I was just wondering whether you had and if so whether it had influenced your results. If you want to reach all countries and not target a particular one the geographic target should be set as unlisted, which I believe is the default.
It's going to be very difficult to get good rankings in all countries, but one thing you can try is to get backlinks from different country sites, for example if you wanted to improve in Google Germany go for some .de backlinks.
Maybe you could narrow the field down? For example, most foreign visitors to my town come from the UK and the Scandinavian countries. So my main site is now targeted at the UK and I have a subdomain that is in the Norwegian language. I know that most visitors to Crete come from other European countries but are there any that come mainly from particular ones? You could perhaps have content on your site, perhaps a page, for those people.