I believe the attribution of this true and eloquent statement to Yamamoto is uncertain at best - as is the more well known attribution of another quote that was probably said (and thought) by many more informed Japanese officers after Dec. 7, 1941: "I fear we have wakened a sleeping giant and filled him with a terrible resolve". The movie "Tora Tora Tora" had Yamamoto uttering this to quiet his ectstatic younger staff officers on board the Nagato after the code phrase "tiger tiger tiger" was relayed from Nagumo's strike force, signifying complete surprise had been achieved in Hawaii.
There is no doubt that Yamamoto spent time in the US (grad school at Harvard, a stint in the attache's office in Washington) and traveled widely. He was especially impressed not just by the oil fields of Texas, but those of next-door Mexico as well. In fact he ended up being "rescued" with some US funds when Mexico let the US Embassy know they had a flat-broke Japanese naval officer (I believe in Monterrey or Veracruz) at a flea-bag hotel who belonged back in DC. A colorful and brilliant character (we won't mention his preposterously ill-designed Midway strike plan).
Also no doubt Yamamoto was dubious about attacking the US, and literally gave the high command 6 months of room following a first-strike on US forces - after which he could promise nothing.
The "blade of grass" quote probably reflected the views of the most worldly Japanese officers. And while (comparatively speaking) true, the shelling of a pier northwest of Santa Barbara by a Japanese submarine on Feb. 26, 1942 led to more than that - coastal artillery was quickly installed in San Diego (Point Loma) and San Pedro, and casements built in the Marin Headlands north of the Golden Gate (guns never got there, as Midway ended any fears of west coast raids). And I spoke with one woman, a teenager at the time, who lived on a ranch near SB and close to Elwood, site of the Jap shelling - country sheriffs came around and handed out shotguns and ammo to those who needed them and told people to black out their windows. The panic in SoCal (the basis, sort of, for the movie "1941") was real.
For history buffs there's a good little exhibit on this bit of histor at Cabrillo Nat. Monument on Point Loma, and when you're driving up 101 north of Santa Barbara you can stop at the Timbers Restaurant, partially constructed of wood from the pier damaged by the Imperial Japanese Navy what seem like a billion years ago .....