Git – How to Delete All Untracked Files

For files you might want to change your command line slightly to the suggested command-line on the git-rm page

git ls-files -z | xargs -0 rm -f

This is much safer with more complex file paths. For directories you could try a similar strategy:

git ls-tree --name-only -d -r -z HEAD | sort -rz | xargs -0 rmdir 

It does however depend on how you would like to treat directories that contain files (often gitignored) that are untracked. The command line above should leave these files and their directories.

But it would be easy to change this to delete the directory, whatever the contents.

You can easily remove unwanted files from local git repositories:
Just remove them with
git rm file
or
git rm -r directory (if you add the --cached-flag the file doesn't get removed from your filesystem). Then commit (or commit --amend) to remove the file from the last commit (it stays in in the history, though). See also here

If you want to remove it from a remote repo follow the same steps and push afterwards.