Currency exchange values are seen in the buy-sell board of an exchange bureau in Buenos Aires, Argentina on May 07, 2018.

IMF Managing Director Christine Lagarde said in a statement on Tuesday that "discussions have been initiated on how we can work together to strengthen the Argentine economy and these will be pursued in short order." Lagarde also said: "I very much welcome President Macri's statement today and look forward to our continuing partnership with Argentina."

The Financial Times and Reuters also reported that Argentina is seeking a credit line.

Bloomberg News reported that Latin America's third-largest economy could get a $30 billion loan from the IMF. The report — citing a person with direct knowledge on the matter — said that Argentine President Mauricio Macri reached out to the IMF after the peso began to free-fall, affecting other financial markets in the country.

Argentina's peso rebounded from a record low against the U.S. dollar on Tuesday following a report the Latin American nation could get a credit line from the International Monetary Fund.

The peso traded at 22.4 against the dollar, down about 2 percent, after falling to an all-time low of 23.1. But for the year, the peso is down more than 20 percent against the greenback despite Argentina sporting the highest overnight interest rates in the world.

On Friday, Argentina's central bank hiked rates for the third time in eight days to a whopping 40 percent as it tries to reign in the country's rampant inflation. In March, consumer prices in Argentina rose 2.3 percent, bringing the 12-month inflation rate to 25.4 percent, well above the central bank's 15 percent target.

"Argentina is in a dilemma," wrote Gabriel Gersztein, head of Latin America global markets strategy at BNP Paribas, in a note on April 26. If the central bank "hikes the way it should, the fiscal deficit will increase. … If the monetary authority decides to do nothing and manage expectations through the FX rate, the damage will be done via higher risk premium in the credit spreads of Argentina (lower reserves and higher depreciation expectations)."

The peso's sharp drop also dragged Argentine stocks lower. The Merval index pulled back as much as 5.4 percent before closing 4.2 percent lower.

"There is no silver bullet for managing Argentina's long-standing and conflicting economic and political challenges," Hasnain Malik, head of equity research at Exotix Capital, said in a note.

After years of populism and protectionist policies, "the adjustment is painful and there is no way to keep all of Argentina's economic metrics heading in the right direction," Malik said. "It is this prognosis that has driven our caution on Argentina equities."

—CNBC's J.R. Reed contributed to this report.