The file descriptor 1 translates to the stdout FILE structure in the Kernel's Open Files Table.
This is a misunderstanding. The kernel's file table has nothing whatsoever to do with user-space file structures.
In any event, the kernel has two levels of indirection. There is the internal structure that represents the file itself, which is reference counted. There is an "open file description" that is reference counted. And then there is the file handle, which is not reference counted. The file structure points the way to the inode itself. The open file description contains things like the open mode and file pointer.
When you call close, you always close the file handle. When a file handle is closed, the reference count on its open file description is decremented. If it goes to zero, the open file description is also released and the reference count on the file itself is decremented. Only if that goes to zero is the kernel's file structure freed.
There is no chance for one process to release a resource another process is using because shared resources are reference counted.
Fiddling with a process with gdb
is almost never safe though may be
necessary if there's some emergency and the process needs to stay open
and all the risks and code involved is understood.
Most often I would simply terminate the process, though some cases may
be different and could depend on the environment, who owns the
relevant systems and process involved, what the process is doing,
whether there is documentation on "okay to kill it" or "no, contact
so-and-so first", etc. These details may need to be worked out in a
post-mortem meeting once the dust settles. If there is a planned
migration it would be good in advance to check whether any processes
have problematic file descriptors open so those can be dealt with in a
non-emergency setting (cron jobs or other scheduled tasks that run
only in the wee hours when migrations may be done are easily missed if
you check only during daytime hours).
Write-only versus Read versus Read-Write
Your idea to reopen the file descriptor O_WRONLY
is problematic as not
all file descriptors are write-only. John Viega and Matt Messier take a
more nuanced approach in the "Secure Programming Cookbook for C and C++"
book and handle standard input differently than standard out and
standard error (p. 25, "Managing File Descriptors Safely"):
static int open_devnull(int fd) {
FILE *f = 0;
if (!fd) f = freopen(_PATH_DEVNULL, "rb", stdin);
else if (fd == 1) f = freopen(_PATH_DEVNULL, "wb", stdout);
else if (fd == 2) f = freopen(_PATH_DEVNULL, "wb", stderr);
return (f && fileno(f) == fd);
}
In the gdb
case the descriptor (or also FILE *
handle) would need to
be checked whether it is read-only or read-write or write-only and an
appropriate replacement opened on /dev/null
. If not, a once read-only
handle that is now write-only will cause needless errors should the
process attempt to read from that.
What Could Go Wrong?
How exactly a process behaves when its file descriptors (and likely also
FILE *
handles) are fiddled behind the scenes will depend on the
process and will vary from "no big deal" should that descriptor never be
used to "nightmare mode" where there is now a corrupt file somewhere due
to unflushed data, no file-was-properly-closed indicator, or some other
unanticipated problem.
For FILE *
handles the addition of a fflush(3)
call before closing
the handle may help, or may cause double buffering or some other issue;
this is one of the several hazards of making random calls in gdb
without knowing exactly what the source code does and expects. Software
may also have additional layers of complexity built on top of fd
descriptors or the FILE *
handles that may also need to be dealt with.
Monkey patching the code could turn into a monkey wrench easily enough.
Summary
Sending a process a standard terminate signal should give it a chance
to properly close out resources, same as when a system shuts down
normally. Fiddling with a process with gdb
will likely not properly
close things out, and could make the situation very much worse.
Best Answer
When a program opens a file, that file ends up on a file descriptor that's free at the time. By opening a file before the program starts, you're only making one more file descriptor busy, so the file you're interested in might end up on a different descriptor. If you want the program to open a different file, you'll need to modify the open operation when it takes place, or intervene afterwards.
One way to modify the operation is to wedge some code between the program and the system library, by preloading a small piece of code. This assumes that the program is a dynamically linked binary, or a script executed by a dynamically linked binary (i.e. it isn't statically linked). Write the following code to a file
override_fopen.c
:Compile with the following command (that's for Linux, other Unix variants may require different options). Note the quotes around the path you want to override.
Run the program as follows (on OSX, use
DYLD_PRELOAD
instead ofLD_PRELOAD
):This only works if the program is calling the
fopen
oropen
library function. If it calls some other function, you'll need to override that one. You can useltrace
to see what library calls the program makes.