Why Binary Not Found in PATH?

I've got a Python project which uses a virtual environment. In my project folder, there is a bin/ folder which contains all the executables needed by the project. To use these, you run source bin/activate to insert the path to that bin/ directory to your PATH variable.

Now, most of the time this works as I'd expect. I run the source command and when I run which python, it shows the one I'd expect from the virtual environment. Sometimes however, it doesn't list that one. It lists a different one on my PATH. Here's an example of what I see:

We can see we are in the project directory:

$ pwd
/Users/dalemy/Projects/my_project

We can also see that the python and python3 binaries (actually symlinks) are in the bin/ directory:

$ ls -al /Users/dalemy/Projects/my_project/bin/python*
lrwxr-xr-x  1 dalemy  staff    7 10 Feb 15:16 /Users/dalemy/Projects/my_project/bin/python -> python3
lrwxr-xr-x  1 dalemy  staff   44 10 Feb 15:16 /Users/dalemy/Projects/my_project/bin/python3 -> /usr/local/Cellar/python/3.7.2_1/bin/python3

Here we can see that this is the first entry in the PATH variable:

$ echo $PATH
/Users/dalemy/Projects/my_project/bin:/Users/dalemy/.poetry/bin:/usr/local/opt/python3/bin:/usr/local/opt/ruby/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/MacGPG2/bin:/opt/X11/bin:/Users/dalemy/Config/scripts:/Users/dalemy/.poetry/bin:/Users/dalemy/.local/bin

But when we ask about where Python is located, it doesn't find it:

$ which python
/usr/local/bin/python

$ which python3
/usr/local/opt/python3/bin/python3

Running type -a shows the same thing:

$ type -a python
python is /usr/local/bin/python
python is /usr/bin/python
python is /usr/local/bin/python
python is /usr/bin/python

I've got no idea why this is happening. The bin/activate command runs hash -r and I've run it manually to be sure that it's clearing out anything cached. If I remove every single entry on my PATH except for the first one, I just get that it can't be found.

Once it starts doing this, the only solution I've found is to remove the virtual environment completely and recreate it.

What could possibly be going wrong here?