Finally I've got a working solution, based on this tweet:

1. Disable all antivirus tools (I've had no external tools beside the Windows-built-in, so nothing to do here for me).
2. Follow the instructions in the KB article.

The KB article shows a script to run, after installing a MSI.

I've had to adjust the script since the Windows user group account administrators is named administratoren in my German Windows 8.1.

After running the adjusted script (for about 10-20 minutes), I've launched the Windows 10 setup again and it ran k.

I'm now writing this from my Windows 10 and seem to be happy.

Update

Since the Microsoft KB article seems to be removed, I've opened the Google Cache version of the KB article and copying the relevant information here.

From the KB article:

Error 0x80070005 in Windows Update when you try to install updates

Resolution

Error code 0x80070005 is also known as "ACCESS DENIED." It usually occurs when you lack file or registry permissions that are required to install the update. (Because file or registry permissions may be changed by malware, we recommend that you scan for malware after you repair the permissions.)

To resolve the issue, make sure that you are logged on as an administrator, and then install the updates again. If the problem continues, use the SubInACL tool to repair file and registry permissions. To do this, follow these steps:

1. Download the SubInACL tool (Mirror), and install it to the default path that is offered by the installer. The default path will be one of the following, depending on your version of Windows:

   • %ProgramFiles%\Windows Resource Kits\Tools (32-bit versions of Windows)
   • %ProgramFiles(x86)%\Windows Resource Kits\Tools (64-bit versions of Windows)

2. Start Notepad, and then copy and paste the following commands:

   @echo off
   
   setlocal
   
   echo.
   echo Determine whether we are on an 32 or 64 bit machine
   echo.
   
   if "%PROCESSOR_ARCHITECTURE%"=="x86" if "%PROCESSOR_ARCHITEW6432%"=="" goto x86
   
   set ProgramFilesPath=%ProgramFiles(x86)%
   
   goto startResetting
   
   :x86
   
   set ProgramFilesPath=%ProgramFiles%
   
   :startResetting
   
   echo.
   
   if exist "%ProgramFilesPath%\Windows Resource Kits\Tools\subinacl.exe" goto filesExist
   
   echo ***ERROR*** - Could not find file %ProgramFilesPath%\Windows Resource Kits\Tools\subinacl.exe. Double-check that SubInAcl is correctly installed and re-run this script.
   goto END
   
   :filesExist
   
   pushd "%ProgramFilesPath%\Windows Resource Kits\Tools"
   
   subinacl.exe /subkeyreg HKEY_LOCAL_MACHINE /grant=administrators=f  /grant=system=f
   subinacl.exe /subkeyreg HKEY_CURRENT_USER /grant=administrators=f  /grant=system=f
   subinacl.exe /subkeyreg HKEY_CLASSES_ROOT /grant=administrators=f  /grant=system=f
   subinacl.exe /subdirectories %windir% /grant=administrators=f /grant=system=f
   
   echo FINISHED.
   echo. 
   echo Press any key to exit . . .
   pause >NUL
   
   popd
   
   :END
   
   endlocal
   

3. Save the file as Reset.cmd. (Super User note: English download, German download)

4. Right-click the Reset.cmd file in File Explorer (in Windows 8.1 and Windows 8) or Windows Explorer (in Windows 7 and Windows Vista), and then click Run as administrator. In Windows XP, double-click the file in Windows Explorer.
5. Wait until the execution finishes, and then try to install the update again.

Note If you receive errors when you run SubInAcl, see the following Microsoft blog: Notes about a couple of possible issues while using the SubInAcl tool.

Solution 1:

Step 1: You can go to PC settings.
Step 2: Under Update and Recovery options, Select "Refresh This PC".

This refresh will delete your installed programs. This might work.

After refresh it might ask for couple of Windows Update install them

Solution 2: Run media creation tool and do a clean install of Windows 10

Hope this helps.