We have an azure VPN gateway setup with two site-to-site connections that are successfully configured. Site A (local network gateway A) and Site B (local network gateway B) can each see everything in the azure VNET where the VPN gateway resides.
We cannot, however, see resources between our sites. For example, I cannot RDP to a machine from my local network gateway A to a machine in local network gateway B even though both local network gateways are connected to the Azure VPN gateway. Both sites (A and B) have added static routes to ensure our respective routers forward requests through the VPN tunnel for our respective local gateway address spaces.
Is this type of connectivity turned off by default for security reasons or should it work with default settings?
This is possible only with a Route Based VPN, and will work best with BGP Enabled. You might also need to create custom routes on your on-prem devices so that traffic destined for your other on-prem network will go to the azure gateway.
It seems you can not connect with site A to site B.
As shown in the diagram, the Azure VPN gateway has traffic selectors
from the virtual network to each of the on-premises network prefixes,
but not the cross-connection prefixes. For example, on-premises site
2, site 3, and site 4 can each communicate to VNet1 respectively, but
cannot connect via the Azure VPN gateway to each other. The diagram
shows the cross-connect traffic selectors that are not available in
the Azure VPN gateway under this configuration.
For more details, see Connect Azure VPN gateways to multiple on-premises policy-based VPN devices.