But trading was choppy, with stocks initially extending losses following the minutes while U.S. Treasury yields and the U.S. dollar index jumped, as the minutes also showed Fed policymakers hinted at a possible shift in future policy.
Minutes of the U.S. central bank's April 27-28 meeting said a number of Fed policymakers thought that if the economy continued rapid progress, it would be appropriate "at some point" in upcoming meetings to begin discussing tapering government bond purchases.
The mere hint of "taper talk" was enough to spark a selloff in bonds and send stocks lower, said Patrick Leary, chief market strategist and senior trader at Incapital.
Investors have been speculating whether rising U.S. inflationary pressures could prompt the Fed to pare back its support sooner than many anticipate.
"Anything short of reinforcing the uber-dovish stance the Fed has had is, at a time when the market is already getting a little jittery with regard to inflation, is what's compounding the sell-off, which could have been catalyzed by next to anything," said Mark Luschini, chief investment strategist at Janney Montgomery Scott in Philadelphia.
The Dow Jones Industrial Average fell 164.62 points, or 0.48%, to 33,896.04, the S&P 500 lost 12.15 points, or 0.29%, to 4,115.68 and the Nasdaq Composite dropped 3.90 points, or 0.03%, to 13,299.74.
The pan-European STOXX 600 index lost 1.51% and MSCI's gauge of stocks across the globe shed 0.58%.
The dollar gained ground following the release of the Fed minutes, snapping a four-day losing streak.
The dollar index rose 0.42%, with the euro up 0.02% to $1.2174.
Cryptocurrencies plunged after regulatory moves by China.
Bitcoin briefly plunged to its lowest level since January in the wake of China's decision to ban financial and payment institutions from providing digital currency services, but pared its losses after some of its prominent backers reiterated their support.
Rival cryptocurrency ethereum was last down 22% at $2,623.
In the Treasury market, the yield on 10-year notes was up 3.8 basis points at 1.680%, pulling back slightly from a day's high of 1.6920%.
Oil prices dropped over $2 a barrel to their lowest levels in three weeks amid worries that surging COVID-19 cases in Asia would hurt demand for crude.
Brent futures fell $2.05, or 3.0%, to settle at $66.66 a barrel, while U.S. West Texas Intermediate (WTI) crude fell $2.13, or 3.3%, to settle at $63.36.
U.S. gold futures fell 0.22% to $1,863.60 an ounce.
(Reporting by Caroline Valetkevitch; additional reporting by Herbert Lash, Sinead Carew, Scott DiSavino and Stephen Culp in New York; Tom Arnold in London; editing by William Maclean, Jonathan Oatis and Richard Pullin)
By Caroline Valetkevitch