I also had this issue, and after banging my head against a wall for >1 hour, I realized that downgrading to Ruby 3.2.3 fixed it. But weirdly I had some issue when re-installing ruby with brew, and I ran in some strange errors in terminal when running bundle.
I decided to use chruby instead to manage ruby version, going ahead. I published a post on medium that go into more detail, if anyone’s curious. This is what ended up working for me.