I set up an SMB server on my Mac by going to System Preferences > Settings > Sharing and enabling File Sharing. Afterwards, I tried to access it from my Arch Linux laptop. It asked for my password, as I expected, and I enter my Mac's login password. The dialog keeps showing however.
Using the terminal and entering smbclient -L 192.168.0.64 yeilds the following output on both my laptop connected to the network, as well as in an Arch Linux virtual machine running on my Mac:
mike@ArchLinux > ~ > smbclient -L 192.168.0.64
smbclient: Can't load /etc/samba/smb.conf - run testparm to debug it
Enter mike's password: <I enter my Mac's login password here>
session setup failed: NT_STATUS_LOGON_FAILURE
Using PCManFM as my Linux file manager, going into network:/// shows "Windows Network". Going into there links me to smb:///, which then shows "Failed to retrieve share list from server: No such file or directory"
Best Answer
I had this behaviour with my Mac Mini running MacOs 10.13 after activating File sharing, but NOT authorizing specific users. So my guess is that you simply forgot to enable login for your account.