I've installed Microsoft SQL Server 2019 (CTP2.1) – 15.0.1100.94 (X64) on my VirtualBox Ubuntu 16.04.5 LTS. Everything seemed to work properly until I've powered off the machine. Since then SQL Server fails to start. Interestingly, rebooting doesn't affect SQL Server ability to restart.
Quick logs:
x@xxx:~$ journalctl -u mssql-server.service -b
Nov 30 14:53:25 xxx systemd[1]: Started Microsoft SQL Server Database Engine.
Nov 30 14:53:28 xxx sqlservr[1061]: /opt/mssql/bin/sqlservr: The file archive [/opt/mssql/lib/sqlservr.sfp] is invalid. Please resolve and try again.
Nov 30 14:53:28 xxx systemd[1]: mssql-server.service: Main process exited, code=exited, status=1/FAILURE
Nov 30 14:53:28 xxx systemd[1]: mssql-server.service: Unit entered failed state.
Nov 30 14:53:28 xxx systemd[1]: mssql-server.service: Failed with result 'exit-code'.
Nov 30 14:53:28 xxx systemd[1]: mssql-server.service: Service hold-off time over, scheduling restart.
Nov 30 14:53:28 xxx systemd[1]: Stopped Microsoft SQL Server Database Engine.
The file archive [/opt/mssql/lib/sqlservr.sfp] is invalid. Please resolve and try again.
Long story:
Failed on booting:
x@xxx:~$ systemctl status mssql-server
● mssql-server.service - Microsoft SQL Server Database Engine
Loaded: loaded (/lib/systemd/system/mssql-server.service; enabled; vendor preset: enabled)
Active: failed (Result: start-limit-hit) since Fri 2018-11-30 14:53:29 CET; 1min 44s ago
Docs: https://docs.microsoft.com/en-us/sql/linux
Process: 1234 ExecStart=/opt/mssql/bin/sqlservr (code=exited, status=1/FAILURE)
Main PID: 1234 (code=exited, status=1/FAILURE)
Nov 30 14:53:29 xxx systemd[1]: mssql-server.service: Unit entered failed state.
Nov 30 14:53:29 xxx systemd[1]: mssql-server.service: Failed with result 'exit-code'.
Nov 30 14:53:29 xxx systemd[1]: mssql-server.service: Service hold-off time over, scheduling restart.
Nov 30 14:53:29 xxx systemd[1]: Stopped Microsoft SQL Server Database Engine.
Nov 30 14:53:29 xxx systemd[1]: mssql-server.service: Start request repeated too quickly.
Nov 30 14:53:29 xxx systemd[1]: Failed to start Microsoft SQL Server Database Engine.
Nov 30 14:53:29 xxx systemd[1]: mssql-server.service: Unit entered failed state.
Nov 30 14:53:29 xxx systemd[1]: mssql-server.service: Failed with result 'start-limit-hit'.
Reinstall fixes problem:
x@xxx:~$ sudo apt-get install --reinstall mssql-server
Reading package lists... Done
Building dependency tree
Reading state information... Done
0 to upgrade, 0 to newly install, 1 reinstalled, 0 to remove and 4 not to upgrade.
Need to get 0 B/231 MB of archives.
After this operation, 0 B of additional disk space will be used.
Preconfiguring packages ...
(Reading database ... 66231 files and directories currently installed.)
Preparing to unpack .../mssql-server_15.0.1100.94-1_amd64.deb ...
Unpacking mssql-server (15.0.1100.94-1) over (15.0.1100.94-1) ...
Processing triggers for libc-bin (2.23-0ubuntu10) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up mssql-server (15.0.1100.94-1) ...
Processing triggers for libc-bin (2.23-0ubuntu10) ...
x@xxx:~$ sudo systemctl status mssql-server
● mssql-server.service - Microsoft SQL Server Database Engine
Loaded: loaded (/lib/systemd/system/mssql-server.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2018-11-30 15:05:37 CET; 53s ago
Docs: https://docs.microsoft.com/en-us/sql/linux
Main PID: 1695 (sqlservr)
Tasks: 124
Memory: 640.5M
CPU: 10.188s
CGroup: /system.slice/mssql-server.service
├─1695 /opt/mssql/bin/sqlservr
└─1724 /opt/mssql/bin/sqlservr
copied fishy file to ~ directory, reboot, no differences:
x@xxx:~$ sudo cp /opt/mssql/lib/sqlservr.sfp sqlservr.sfp
x@xxx:~$ sudo reboot
(...) // SSH session restart
login as: x
x@127.1.1.1's password:
Welcome to Ubuntu 16.04.5 LTS (GNU/Linux 4.4.0-131-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
9 packages can be updated.
7 updates are security updates.
New release '18.04.1 LTS' available.
Run 'do-release-upgrade' to upgrade to it.
Last login: Fri Nov 30 14:53:32 2018 from 10.0.2.2
x@xxx:~$ diff /opt/mssql/lib/sqlservr.sfp sqlservr.sfp
x@xxx:~$ sudo systemctl status mssql-server
[sudo] password for x:
● mssql-server.service - Microsoft SQL Server Database Engine
Loaded: loaded (/lib/systemd/system/mssql-server.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2018-11-30 15:38:07 CET; 1min 10s ago
Docs: https://docs.microsoft.com/en-us/sql/linux
Main PID: 1060 (sqlservr)
Tasks: 127
Memory: 1.7G
CPU: 10.386s
CGroup: /system.slice/mssql-server.service
├─1060 /opt/mssql/bin/sqlservr
└─1235 /opt/mssql/bin/sqlservr
Poweroff, SQL Server start fails, difference:
x@xxx:~$ sudo poweroff
(...) // SSH session restart
[sudo] password for x:
login as: x
x@127.1.1.1's password:
Welcome to Ubuntu 16.04.5 LTS (GNU/Linux 4.4.0-131-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
9 packages can be updated.
7 updates are security updates.
New release '18.04.1 LTS' available.
Run 'do-release-upgrade' to upgrade to it.
Last login: Fri Nov 30 15:38:55 2018 from 10.0.2.2
x@xxx:~$ sudo systemctl status mssql-server
[sudo] password for x:
● mssql-server.service - Microsoft SQL Server Database Engine
Loaded: loaded (/lib/systemd/system/mssql-server.service; enabled; vendor preset: enabled)
Active: failed (Result: start-limit-hit) since Fri 2018-11-30 16:12:49 CET; 27s ago
Docs: https://docs.microsoft.com/en-us/sql/linux
Process: 1240 ExecStart=/opt/mssql/bin/sqlservr (code=exited, status=1/FAILURE)
Main PID: 1240 (code=exited, status=1/FAILURE)
Nov 30 16:12:49 xxx systemd[1]: mssql-server.service: Unit entered failed state.
Nov 30 16:12:49 xxx systemd[1]: mssql-server.service: Failed with result 'exit-code'.
Nov 30 16:12:49 xxx systemd[1]: mssql-server.service: Service hold-off time over, scheduling restart.
Nov 30 16:12:49 xxx systemd[1]: Stopped Microsoft SQL Server Database Engine.
Nov 30 16:12:49 xxx systemd[1]: mssql-server.service: Start request repeated too quickly.
Nov 30 16:12:49 xxx systemd[1]: Failed to start Microsoft SQL Server Database Engine.
Nov 30 16:12:49 xxx systemd[1]: mssql-server.service: Unit entered failed state.
Nov 30 16:12:49 xxx systemd[1]: mssql-server.service: Failed with result 'start-limit-hit'.
x@xxx:~$ diff /opt/mssql/lib/sqlservr.sfp sqlservr.sfp
Binary files /opt/mssql/lib/sqlservr.sfp and sqlservr.sfp differ
Can someone explain this situation and suggest how to fix it?
Best Answer
sqlservr.sfp and other .sfp files installed by SQL Server are file archive files (like a tar.qz or .zip) that contain the binaries and other files necessary for things like SQL Server to run inside of the SQL Platform Abstraction Layer (SQL PAL). When sqlservr starts up either by directly running sqlservr or by starting the mssql-server service, the sqlservr process looks for the existence of these .sfp files and tries to load them into the SQL PAL. As a security measure, sqlservr checks these .sfp files against an allowed list of .sfps. If the .sfp is not one of the recognized .sfp files, it will not load it. The sqlservr.sfp is the main .sfp for running the actual SQL Server engine. For some reason the .sfp file check in this situation is failing. Maybe there is some process on your OS that is going through and modifying files or attributes of files like a virus scanner or disk cleaner. This is likely why it works until you reboot and then if you reinstall it works again. I recommend checking the checksums of the original file and the file after you reboot. It looks like based on your diff the file is different. Hopefully, that at least explains what is happening a bit better.