Networking – How to disable NAT and still have internet connection? (I have IPoE not PPPoE)

Basically NAT is being a huge pain in the ass for me because the tiny NAT table of my modem/router keeps filling up all the time and then I can't get any packets through for several minutes (this happens for example every time I ping a lot of servers).

Now, I only have one computer so I don't need NAT and I want to get rid of it. I have 1 public IP from my ISP.

I've done some research and it seems one way to do this is to have the modem act as a bridge and use Windows to establish a PPPoE connection. But one problem, PPPoE requires an username/password from the ISP. And I don't think my ISP even supports PPPoE since all the modems they hand out are configured to use IPoE (ENET ENCAP) that doesn't require any kind of credentials to make a connection.

I have tried to simply turn off NAT from my modem, but then my computer loses internet connection. I've done some research and apparently it is because I'm using my LAN IP to send packets to the ISP which then drops them (because my modem is no longer changing the source ip of the packets).

I've tried to send packets using my public IP as the source, but got no responses. Is my modem dropping the responses instead of broadcasting them to my computer in the LAN?

But anyway, my ultimate question is, is there any way to not use NAT and still have an internet connection? Only one computer, only one public IP. And if so, how