File Management – How to Delete an Indelible Directory?

directoryfilesnfsrm

I untarred a corrupt tar file, and managed to end up with some directory
that I can not delete,
If I try to delete it, it seems like it can not be found, but ls shows it's present, both with bash and with python I get similar behaviour, except right after I try to delete it with rm -rf, ls complains it can't find it, then it lists it (see below after rm -rf). The find command shows the file is present,
but still I can't think of a way to delete it.
Here are my attempts:

Here you see both ls and find agree we have a directory,

rl]$ ls
mikeaâ??cnt
rl]$ find -maxdepth 1 -type d -empty -print0  
./mikeaâcnt

But I can't delete it:

rl]$ find -maxdepth 1 -type d -empty -print0 |  xargs -0 rm -f -v 
rm: cannot remove `./mikeaâ\302\201\302\204cnt': Is a directory
rl]$ ls
mikeaâ??cnt

I can cd to it though and it's empty:

rl]$ cd mikeaâ^Á^Äcnt/
mikeaâ^Á^Äcnt]$ ls
mikeaâ^Á^Äcnt]$ pwd
.../rl/mikeaâcnt


mikeaâ^Á^Äcnt]$ cd ../
rl]$ ls
mikeaâ??cnt

see below that is not a simple file but a directory, plus ls behaves funny after the rm -rf it says it can't find the file then lists it straight after:

rl]$ rm mikeaâ^Á^Äcnt/
rm: cannot remove `mikeaâ\302\201\302\204cnt/': Is a directory
rl]$ rm -rf  mikeaâ^Á^Äcnt/
rl]$ ls
ls: cannot access mikeaâcnt: No such file or directory
mikeaâ??cnt
rl]$

So this is the attempt with python, the file is found, but the name is not
usable as a name that can be deleted:

rl]$ python 
Python 2.6.6 (r266:84292, Jul 10 2013, 22:48:45) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import shutil
>>> os.listdir('.')
['mikea\xc3\xa2\xc2\x81\xc2\x84cnt']
>>> shutil.rmtree(os.listdir('.')[0] )
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.6/shutil.py", line 204, in rmtree
    onerror(os.listdir, path, sys.exc_info())
  File "/usr/lib64/python2.6/shutil.py", line 202, in rmtree
    names = os.listdir(path)
OSError: [Errno 2] No such file or directory: 'mikea\xc3\xa2\xc2\x81\xc2\x84cnt'

even when I use tab completion the name it picks up is no usable:

rl]$ rm -rf mikeaâ^Á^Äcnt 
rl]$ ls
ls: cannot access mikeaâcnt: No such file or directory
mikeaâ??cnt

using the name that python shows with bash I get this:

rl]$ rm -rf "mikea\xc3\xa2\xc2\x81\xc2\x84cnt"
rl]$ ls
ls: cannot access mikeaâcnt: No such file or directory
mikeaâ??cnt

Is there anything I can do to get rid of this corrupt dir?
The underlying filesystem (NFS) seems functional and no other problems are reported, and I have had no such problems until the corrupt tar file.

EDIT:
Here is using find's own -exec option to call rm

rl]$ find -maxdepth 1 -type d -empty -exec rm -f {} \;
find: `./mikeaâ\302\201\302\204cnt': No such file or directory
rl]$ ls
ls: cannot access mikeaâcnt: No such file or directory
mikeaâ??cnt
rl]$

but the file is still there, (ls complains it can't find it, but then shows it anyway)

2nd EDIT:

rl]$ find -maxdepth 1 -type d -empty -exec rm -rf {} \;
find: `./mikeaâ\302\201\302\204cnt': No such file or directory
rl]$ ls
ls: cannot access mikeaâcnt: No such file or directory
mikeaâ??cnt

The behaviour is still unchanged, the file still present

3rd EDIT: 

rl]$ ls
mikeaâ??cnt
rl]$ find -maxdepth 1 -type d -empty -exec rm -rf {} + 
rl]$ ls
ls: cannot access mikeaâcnt: No such file or directory
mikeaâ??cnt

There seems to be more to the name than mikeaâcnt from looking at the output of the python attempt mikea\xc3\xa2\xc2\x81\xc2\x84cnt, and this screenshot:

ls output

4th EDIT:
This is the attempt with a wild card:

rl]$ echo * 
mikeaâcnt
rl]$ echo mike* 
mikeaâcnt
rl]$ rm -rf mike*
rl]$ ls
ls: cannot access mikeaâcnt: No such file or directory
mikeaâ??cnt

and my locale: 

rl]$  locale
LANG=en_US.utf8
LC_CTYPE="en_US.utf8"
LC_NUMERIC="en_US.utf8"
LC_TIME="en_US.utf8"
LC_COLLATE="en_US.utf8"
LC_MONETARY="en_US.utf8"
LC_MESSAGES="en_US.utf8"
LC_PAPER="en_US.utf8"
LC_NAME="en_US.utf8"
LC_ADDRESS="en_US.utf8"
LC_TELEPHONE="en_US.utf8"
LC_MEASUREMENT="en_US.utf8"
LC_IDENTIFICATION="en_US.utf8"
LC_ALL=

5th Edit:

rl]$ ls -i 
ls: cannot access mikeaâcnt: No such file or directory
? mikeaâ??cnt

but also the behaviour has changed, now ls and cd do this:

rl]$ ls
ls: cannot access mikeaâcnt: No such file or directory
mikeaâ??cnt
rl]$ cd mikeaâ^Á^Äcnt 
mikeaâcnt: No such file or directory.

This has happened after the attempts to delete, I'm thinking that it might be NFS issues as suggested in one of the answers here by vinc17.

6th EDIT:
This is the output of lsof and ls -a


rl]$ /usr/sbin/lsof mikeaâ^Á^Äcnt
lsof: status error on mikeaâ\xc2\x81\xc2\x84cnt: No such file or directory

above is wrong, here is the correct lsof invocation:(rl is the parent directory)

rl]$ /usr/sbin/lsof | grep mike | grep rl 
tcsh      11926   mike  cwd       DIR   0,33     4096 19569249 /home/mike/mish/rl
lsof      14733   mike  cwd       DIR   0,33     4096 19569249 /home/mike/mish/rl
grep      14734   mike  cwd       DIR   0,33     4096 19569249 /home/mike/mish/rl
grep      14735   mike  cwd       DIR   0,33     4096 19569249 /home/mike/mish/rl
lsof      14736   mike  cwd       DIR   0,33     4096 19569249 /home/mike/mish/rl
rl]$ 

rl]$ ls -a
ls: cannot access mikeaâcnt: No such file or directory
.  ..  mikeaâ??cnt

7th Edit:
move won't work, (I tried it before all this, but I did not save the output), but it has the same problem as ls and rm with the file.

8th EDIT:
this is using the hex chars as suggested:

 rl]$ ls --show-control-chars | xxd
0000000: 6d69 6b65 61c3 a2c2 81c2 8463 6e74 0a    mikea......cnt.
rl]$ rmdir $'mikea\6d69\6b65\61c3\a2c2\81c2\8463\6e74\0acnt' 
rmdir: failed to remove `mikea\006d69\006b651c3\a2c2\\81c2\\8463\006e74': No such file or directory
rl]$ ls
ls: cannot access mikeaâcnt: No such file or directory
mikeaâ??cnt
rl]$

9th Edit:
for the stat command:

 rl]$ stat  mikeaâ^Á^Äcnt 
stat: cannot stat `mikeaâ\302\201\302\204cnt': No such file or directory
 rl]$

Its seems even more likely from all the output, there is a bug or other NFS misbehaviour as suggested in the comments.

Edit 10:
This is strace output in a gist since its so large,
its the output or these two commands:

strace -xx rmdir ./* | grep -e '-1 E'`
strace -xx -e trace=file ls -li`

https://gist.github.com/mikeatm/e07fa600747a4285e460

Edit 11:
So before the above rmdir I noticed that I could cd into the directory,
but after the rmdir I could not cd again, similar to yesterday. The . and .. files were present:

rl]$ ls
mikeaâ??cnt
rl]$ cd mikeaâ^Á^Äcnt/
mikeaâ^Á^Äcnt]$ ls
mikeaâ^Á^Äcnt]$ ls  -a
.  ..
mikeaâ^Á^Äcnt]$ cd ../

Final Edit:
I saw a local admin over this and it was dealt with by logging on to the server itself and deleting from there. The explanation from them is that it could be a problem with character sets in the name being inappropriate.

Best Answer

The following excerpt from this essay potentially explains why that directory refuses to be deleted:

NFSv4 requires that all filenames be exchanged using UTF-8 over the wire. The NFSv4 specification, RFC 3530, says that filenames should be UTF-8 encoded in section 1.4.3: “In a slight departure, file and directory names are encoded with UTF-8 to deal with the basics of internationalization.” The same text is also found in the newer NFS 4.1 RFC (RFC 5661) section 1.7.3. The current Linux NFS client simply passes filenames straight through, without any conversion from the current locale to and from UTF-8. Using non-UTF-8 filenames could be a real problem on a system using a remote NFSv4 system; any NFS server that follows the NFS specification is supposed to reject non-UTF-8 filenames. So if you want to ensure that your files can actually be stored from a Linux client to an NFS server, you must currently use UTF-8 filenames. In other words, although some people think that Linux doesn’t force a particular character encoding on filenames, in practice it already requires UTF-8 encoding for filenames in certain cases.

UTF-8 is a longer-term approach. Systems have to support UTF-8 as well as the many older encodings, giving people time to switch to UTF-8. To use “UTF-8 everywhere”, all tools need to be updated to support UTF-8. Years ago, this was a big problem, but as of 2011 this is essentially a solved problem, and I think the trajectory is very clear for those few trailing systems.

Not all byte sequences are legal UTF-8, and you don’t want to have to figure out how to display them. If the kernel enforces these restrictions, ensuring that only UTF-8 filenames are allowed, then there’s no problem... all the filenames will be legal UTF-8. Markus Kuhn’s utf8_check C function can quickly determine if a sequence is valid UTF-8.

The filesystem should be requiring that filenames meet some standard, not because of some evil need to control people, but simply so that the names can always be displayed correctly at a later time. The lack of standards makes things harder for users, not easier. Yet the filesystem doesn’t force filenames to be UTF-8, so it can easily have garbage.

Related Question