The concepts of quorum, and owners are separate topics. Just because a member of the WSFC doesn't get a vote in quorum does not mean it can't own a resource. Additionally, SQL Server doesn't really play a role at all--the same concepts apply regardless of what type of clustered resource you're dealing with.
Quorum:
The quorum is the number of votes necessary to transact business on your WSFC. Depending on your WSFC configuration, voters can be nodes (servers), a drive, or a file share. You need more than 50% of your votes in order for the WSFC to be online. If you lose 50% or more of your voters, then the WSFC and all clustered services (including your FCI) will go offline and not come back until you have (or force) quorum.
In your configuration, you have a File Share Witness (hopefully in an impartial location reachable by both primary & DR sites), and you've also changed the NodeWeight to 0 for your DR servers. Rather than thinking of NodeWeight as "The DR servers don't get to vote," you should think of it as "The DR servers get to vote, but their vote doesn't count." They are still there, they're still part of the WSFC, it's just that the WSFC doesn't listen.
Even though the FSW isn't hosted on the cluster (or, perhaps, more accurately, because the FSW isn't hosted on the cluster!), it still has to be "present" to vote. Which brings us to the concept of ownership...
Cluster Owner/Host Server:
Your WSFC has a network name and an IP address. That name & IP has to be tied to a machine that is part of your cluster. More specifically, it can be tied to any one machine in your cluster. This is part of your WSFC.
In your scenario, your DR servers have no vote for quorum, but they still must be possible owners of the WSFC Host Server. If you manually fail over to your DR site (which will involve forcing quorum because you have zero voters in DR), then one of your DR servers must host the cluster name & IP. If it doesn't, then your cluster can't come online.
Your FSW is "owned" by the same server that owns/hosts the Cluster itself. Therefore it must have all servers, including DR, as possible owners. When you do force quorum and come online in your DR site, the WSFC is going to want to continue talking to the FSW.
Your original question:
...what would be the effect if I limited the possible owners to just
those nodes at the primary site?
I suspect that you would have problems when you tried to force quorum and bring your WSFC online in the DR site--though, that's just a guess.
From a SQL Server perspective, you just care that you have quorum and the WSFC is reliably up. If you're having issues in that area, I'd look at the specific issues to your reliability. In reality, you probably don't really care which server is hosts your WSFC (and FSW).
If I understand your situation correctly, SQL Server isn't even in the picture right now and you're stuck at configuring Windows Server Failover Clustering?
If that's the case, then the best place to look for further clues (as this is quite the open ended question that could take a person many hours to investigate and depend upon settings in your specific environment) would be the cluster log. One of the easiest ways to gather the cluster log is to use the powershell command Get-ClusterLog it can take multiple inputs and I'd recommend using at least -UseLocalTime and -Directory to change the log from UTC to local and the directory that you'd like the logs to be output in, respectively.
Since this, currently, has no involvement with SQL Server the question may be closed to migrated (for future reference).
Best Answer
Add a quorum configuration in the failover cluster manager. In order for the cluster to continue to be online a majority of nodes need to be online, you have only two nodes. If one goes down, only one is online and that’s not majority. If you add a witness like shared drive which needs to be stored off the nodes, then if one node goes down, majority two are still online so cluster will remain online.