The issue is that the python garbage collector for the API isn't working correctly so it remembers previous messages as part of future responses. If you do know how to fix that I'd love a pr on the GitHub repo and I'll give you a shoutout on the readme.
but the prompt isn’t very weird, https://github.com/JoshAtticus/geminium/blob/4e36317767d59c9aa2c58e58a8a140554c4ae202/src/api.py#L176-L189