I suspect you probably are using Mac OS 10.7 which is very memory hungry. As noted by kccqzy, the purge command may reclaim some inactive memory if you have XCode installed.
Something you should try is closing and reopening applications which are memory hungry or have a tendency to leak memory (Safari, Firefox). Alternatively, restarting your computer would also accomplish the same though take considerably longer.
To answer the question, are you really using 4GB of memory. Yes. Technically more if you take note of the "Swap used:" stat.
Your information is very good for RAM and CPU, but I don't see anything amiss and things look quite balanced. Yes, there is one program that's busy and not responding, so you could kill that to rule it out as something that's slowing down Photoshop.
My hunch is you will need to examine storage to find the slow down.
My reasoning is Photoshop does a good job of pre-allocating disk space for swap of it's own (so it doesn't even swap to the OS swap) as well as pre-allocating all the RAM it thinks it will need. Even without seeing the normal RAM and CPU charts you posted, it's unlikely your Mac is paging virtual memory to disk based on the stats you posted from a snapshot in time.
Open terminal app from /Applications/Utilities and run two commands to list the attached drives (so you know which is disk0, disk1 etc...) and another to monitor the actual input and output for the drive.
df | grep disk
iostat 30
The second command will run continuously until you press control+c or close the terminal app or window.
My guess is you will see that there is continuous load on one or more disks and quitting the programs that are causing the disk to be busy will make Photoshop more snappy. You've already ruled out high CPU load and memory contention (the pressure would be yellow or red if that were to slow down Photoshop) so the next likely candidate is that the system is waiting for the storage.
The other suggestions to try killing other applications are good ones, but you will be more successful if you can measure the exact slow item and then change things and re-measure later.
Here is a sample of what you might see on an idle system:
Mac:~ me$ iostat 30
disk0 disk1 cpu load average
KB/t tps MB/s KB/t tps MB/s us sy id 1m 5m 15m
36.80 20 0.71 24.99 0 0.00 6 3 91 1.34 1.61 1.65
32.00 0 0.01 0.00 0 0.00 1 1 97 1.32 1.60 1.65
0.00 0 0.00 0.00 0 0.00 1 2 97 1.37 1.61 1.65
4.00 0 0.00 0.00 0 0.00 1 2 97 1.31 1.58 1.64
28.57 1 0.04 0.00 0 0.00 4 3 93 1.48 1.61 1.65
9.85 3 0.02 0.00 0 0.00 2 1 97 1.44 1.60 1.65
Here is what first a minute of activity on the internal SSD then idle, then a minute of activity on the Time Machine drive and idle, and finally Time Machine running with both drives having equivalent activity (read from one = write to another):
Mac:~ me$ iostat 30
disk0 disk1 cpu load average
KB/t tps MB/s KB/t tps MB/s us sy id 1m 5m 15m
36.95 20 0.71 42.93 0 0.00 6 3 91 1.99 1.90 1.75
8.1 1266 10.09 8.00 0 0.00 8 12 81 1.94 1.89 1.75
10.18 774 7.69 0.00 0 0.00 22 9 69 1.64 1.82 1.73
18.42 125 2.25 0.00 0 0.00 11 4 86 1.50 1.77 1.71
32.27 21 0.67 0.00 0 0.00 6 2 92 1.40 1.73 1.70
12.29 48 0.57 6.78 223 1.47 23 8 70 1.74 1.78 1.72
20.30 14 0.28 9.61 53 0.50 2 3 95 1.39 1.69 1.69
17.73 5 0.08 14.75 0 0.01 11 2 87 1.23 1.62 1.66
32.30 23 0.73 0.00 0 0.00 4 2 94 1.50 1.67 1.68
13.14 3 0.04 0.00 0 0.00 6 1 92 1.45 1.64 1.66
21.07 24 0.50 6.12 0 0.00 9 3 88 2.23 1.80 1.72
40.77 197 7.86 37.93 202 7.47 13 10 77 2.27 1.83 1.73
Best Answer
Yes - your Mac is perfectly normal and here's how to tell precisely how much extra RAM you can allocate to reach pre-designed limits that Apple has established for macOS.
You’ve provided very excellent details and thinking in this analysis. Overall, everything looks to be functioning perfectly, well allocating memory to all the programs you have, zero memory pressure, the compression routine is starting to kick in - you have a very balanced amount of wired to application-specific memory.
I wouldn’t worry a bit and this looks exactly like a normal system running in balance with excess RAM available for you to add significant more programs or data processing.
All UNIX type operating systems are designed to keep in RAM all the data it can and minimize the amount of free memory since it can release the allocations instantly when a new use is higher priority than the old cached data that can is there just in case:
Basically, the memory system has many layers of physical and logical abstractions that let it commit what looks to be a poor allocation on first glance that in practice is efficient, speedy and very well tested in practice.
When you would start to worry about not enough memory, the system will draw the pressure chart in yellow and red. Even running there can be just fine, it only means you can’t add more memory hungry processes, and not that you’re actually being slowed due to pressure.
From the above link, you can convince yourself how much extra memory you have ready to go by artificially running a command to over-allocate memory until you reach the warning level (or the critical level). Here are screen shots of my MacBook that runs every bit as fast when over-allocated to critical - everything is as snappy in Safari / Photos / whatever as it is with zero memory pressure.
The above image is everything going back to green after running the critical command and pressing control+C (or closing the terminal window running the memory_pressure diagnostic.
Here are the images after warning and critical - you can see how much allocation was needed to push the system to these thresholds:
So, on my Mac that only has 8 GB of RAM, I needed to allocate another 4.65 GB of memory to reach the "warning" state and another 6.46 GB of memory to reach the "critical" state where there's no longer an abundance of free memory for the next use. Even then, the system was not slowed any perceptible amount. Pretty amazing for an m3 chip running 1.1 GHz or so that was build 5 years ago.