The Senate unanimously voted to impeach Pena after the lower house had also unanimously backed his removal, according to footage broadcast on Paraguayan television, the DPA reported.
Vice President Pedro Alliana was sworn in as the country's new leader shortly after Pena's removal. He is set to serve until August 2028.
Legal proceedings against Pena began almost immediately after Alliana was sworn in.
Under the prosecutor's order, Pena must remain under house arrest until he appears before a specialist unit for economic crime and corruption. According to local media, this is expected to happen on Saturday.
Footage broadcast by the Paraguayan news portal ABC showed police officers outside Pena's home.
An arrest warrant was also issued for central bank Chief Carlos Carvallo.
During the impeachment proceedings, senators agreed that Pena had committed misconduct in office.
Pena did not attend the Senate hearing, instead sending his lawyers to represent him. He had earlier unsuccessfully requested a 72-hour postponement.
His lawyers argued that the impeachment proceedings were no longer valid because Pena had already submitted his resignation. However, the Senate and lower house had refused to accept it.
Had Congress accepted his resignation, Pena would have become a senator for life as a former president, granting him parliamentary immunity.