How do I remove it?
Since you need to view the page to edit it and delete the web part it's tricky when you add a web part to your page that causes an exception every time it's loaded. Here's two simple options:
1) retract the solution in Visual Studio
2) use the ?Contents=1 parameter on the web page to see the list of web parts without loading them. for example: http://intranet.contoso.com/SitePages/Home.aspx?Contents=1
Then fix your error or at least add exception handling