Japanese is SHY?
Maybe some Japanese people look away from you if you make your eyes contact with them. Set your mind at ease. It is not that Japanese dislike you. Japanese is very shy, afraid to communicate to foreigner. Why is is? Please remember that Japan is the island country, not multinational, constituted single race. Therefore, Japanese still has difficulty communicate with foreigner a little. However, don't worry. Could I tell you Japanese personality again? Japanese are only shy.
Japanese is KIND?
Probably nobody speaks to you when you lost your way in town. At that time I want you to speak to someone, don't be afraid. Japanese try to solve your question or trouble in English though they might be not good at speaking English. However, because they cannot understand English, they may be not able to solve your question or trouble. Could you forgive them? If you ask some question to Japanese please speak in English slowly. If you possibly can, please speak Japanese or show the map or figure. Japanese maybe make an effort to solve your question and trouble.
If you speak to Japanese.
I will tell you the "magic" word in Japan. It is "sumimasen". "sumimasen" means "Excuse me" in English. If you say "sumimasen" don't say "excuse me", maybe Japanese is relieved, becomes more empathetic to you. If your question or trouble is solved please say "arigatou" to Japanese. "arigatou" means "Thank you" in English.
Although there are many Japanese isn't good at English, they try to solve your question or trouble. I think if you will speak in English slowly and show the map or figure, most of Japanese can solve your question or trouble.
Don't be afraid to ask your question and speak it to Japanese in Japan.