I get nervous with 40mph sustained. The issue is there are usually also gusts, so at 40, getting a 45-50 gust is not uncommon; those are the ones that make me change lanes.
FYI - 40mph is just a bit over 34kts, which is the Beaufort definition of a gale. It is also the bottom end for the wind range of a Tropical Storm (34-63kts). Not weather to be out in, since at that wind speed, things like tree branches start coming off the trees
40mph is Level 1 restrictions (some RVs, trailers) on the Chesapeake Bridge-Tunnel.