Windows – “Unable to open the self-extracting executable file. The file is locked or in use by another process.” error while installing anything on windows XP

To get rid of the shutdown dialog

(perhaps unsafely)

1. Open Task Manager
2. Find the messagebox dialog in the applications list. If you're not sure you found it, you can test by right clicking on what you think it is and selecting Bring to Front.
3. Right click on it and select Go to Process. You'll now be in the Processes tab with a process highlighted on the list.
4. Right click on the process and select Open File Location.

If it's an AVG-specific file (i.e. if it's in an AVG folder in Appdata or Program Files): Move it to another directory. It won't be able to autorun, but you'll still have it around in case it turns out you need it.

If it's not AVG specific (e.g. it's a DLL) that's a different can of worms.

To remove the installation

I'm not sure what to do about half-baked installs. If it were me I'd move the content out of Program Files (keep a copy in case you figure out you can, and really should, do something about it - a copy you can move back, etc) then redownload and reinstall it.

I would say it most definitly is related to the rootkit that was found. It is quite common for viruses to corrupt things in the registry and cause them to malfunction when they are removed.

I would try to run malwarebytes

If that doesnt fix the issues I would go ahead and reinstall windows. In fact I might even do this first.

If you need to do any data recovery Ive had great success with GetDataBack