I was getting same thing when I had my Azure VPN gateway VPN Type set to Route-Based(Azure Default). After I recreated it to Policy-based, it started to work fine. 
-James