Mysql – Azure Pipelines: MySQL Exiting and No Longer Available with –initialize or –initialize-insecure (No Clear Errors)

azureMySQLmysql-5.7

Background (Aka the "Why")

I'm attempting to automate a build of a static website based on a WordPress site utilizing Azure DevOps Pipelines. As part of that process I need to get MySQL running, load a SQL File, and then run WP2Static. The Azure Pipeline build machines appear to already have MySQL 5.7.21 installed, so this seemed like it should be easy, but it looks like its not quite that simple.

When the Build Machine is created / booted up, I currently have it running the following script. Figuring that maybe Microsoft was blocking TCP connections, I moved to named-pipes. However, I really don't care how I connect to it as long as it works.

Here is my initial script:

SET BASEDIR=%~dp0
mkdir %BASEDIR%data
echo [mysqld] > %BASEDIR%\my.cnf
echo named-pipe >> %BASEDIR%\my.cnf 
echo enable-named-pipe >> %BASEDIR%\my.cnf 
echo datadir = %BASEDIR%data >> %BASEDIR%\my.cnf

echo MySQL is trying to start...

call c:\mysql-5.7.21-winx64\bin\mysqld --defaults-file="%BASEDIR%\my.cnf" --initialize-insecure --console --log-error-verbosity=3 --enable-named-pipe

echo Attempting to connect
mysql -h . -e "SHOW DATABASES;"

ps -a -l

I'm sure I must be missing something, but based on the following output from the script it looks like to me that MySQL spits out some warnings and then seems to shut-down . The reason why it does so is not clear to me. As a result the named pipe connection fails; and the ps command shows that MySQL is not currently running:

2020-07-07T18:32:34.1565180Z ##[section]Starting: Command Line Script
2020-07-07T18:32:34.1730992Z ==============================================================================
2020-07-07T18:32:34.1731308Z Task         : Command line
2020-07-07T18:32:34.1731613Z Description  : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
2020-07-07T18:32:34.1731944Z Version      : 2.164.2
2020-07-07T18:32:34.1732158Z Author       : Microsoft Corporation
2020-07-07T18:32:34.1732475Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
2020-07-07T18:32:34.1732871Z ==============================================================================
2020-07-07T18:32:35.7325602Z Generating script.
2020-07-07T18:32:35.7773030Z ========================== Starting Command Output ===========================
2020-07-07T18:32:35.8066922Z ##[command]"C:\windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "D:\a\_temp\61b70ad0-80ee-44ae-8a8c-10e058f99ae0.cmd""
2020-07-07T18:32:35.8286149Z MySQL is trying to start
2020-07-07T18:32:37.6592717Z 2020-07-07T18:32:36.599048Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2020-07-07T18:32:37.6595406Z 2020-07-07T18:32:36.613597Z 0 [Note] Ignoring --secure-file-priv value as server is running with --initialize(-insecure) or --bootstrap.
2020-07-07T18:32:37.6596483Z 2020-07-07T18:32:36.989216Z 0 [Note] c:\mysql-5.7.21-winx64\bin\mysqld (mysqld 5.7.21) starting as process 6028 ...
2020-07-07T18:32:37.6598283Z 2020-07-07T18:32:37.618813Z 0 [Note] --initialize specifed on an existing data directory.
2020-07-07T18:32:38.2916948Z 2020-07-07T18:32:38.291289Z 0 [Note] InnoDB: Mutexes and rw_locks use Windows interlocked functions
2020-07-07T18:32:38.2918103Z 2020-07-07T18:32:38.291592Z 0 [Note] InnoDB: Uses event mutexes
2020-07-07T18:32:38.2918864Z 2020-07-07T18:32:38.291762Z 0 [Note] InnoDB: _mm_lfence() and _mm_sfence() are used for memory barrier
2020-07-07T18:32:38.2921185Z 2020-07-07T18:32:38.291921Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.3
2020-07-07T18:32:38.3303282Z 2020-07-07T18:32:38.330044Z 0 [Note] InnoDB: Number of pools: 1
2020-07-07T18:32:38.7068224Z 2020-07-07T18:32:38.706521Z 0 [Note] InnoDB: Not using CPU crc32 instructions
2020-07-07T18:32:38.7117052Z 2020-07-07T18:32:38.711487Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2020-07-07T18:32:38.7301140Z 2020-07-07T18:32:38.729868Z 0 [Note] InnoDB: Completed initialization of buffer pool
2020-07-07T18:32:38.7456808Z 2020-07-07T18:32:38.745367Z 0 [Note] InnoDB: The first innodb_system data file 'ibdata1' did not exist. A new tablespace will be created!
2020-07-07T18:32:38.7463578Z 2020-07-07T18:32:38.746197Z 0 [Note] InnoDB: Setting file '.\ibdata1' size to 12 MB. Physically writing the file full; Please wait ...
2020-07-07T18:32:38.7694691Z 2020-07-07T18:32:38.768972Z 0 [Note] InnoDB: File '.\ibdata1' size is now 12 MB.
2020-07-07T18:32:38.7723103Z 2020-07-07T18:32:38.772061Z 0 [Note] InnoDB: Setting log file .\ib_logfile101 size to 48 MB
2020-07-07T18:32:38.8232642Z 2020-07-07T18:32:38.823009Z 0 [Note] InnoDB: Setting log file .\ib_logfile1 size to 48 MB
2020-07-07T18:32:39.0446649Z 2020-07-07T18:32:39.044411Z 0 [Note] InnoDB: Renaming log file .\ib_logfile101 to .\ib_logfile0
2020-07-07T18:32:39.0450858Z 2020-07-07T18:32:39.044887Z 0 [Warning] InnoDB: New log files created, LSN=45790
2020-07-07T18:32:39.0451481Z 2020-07-07T18:32:39.045061Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2020-07-07T18:32:39.0455784Z 2020-07-07T18:32:39.045369Z 0 [Note] InnoDB: Setting file '.\ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2020-07-07T18:32:39.0659142Z 2020-07-07T18:32:39.065654Z 0 [Note] InnoDB: File '.\ibtmp1' size is now 12 MB.
2020-07-07T18:32:39.0667595Z 2020-07-07T18:32:39.066539Z 0 [Note] InnoDB: Doublewrite buffer not found: creating new
2020-07-07T18:32:39.3343536Z 2020-07-07T18:32:39.334107Z 0 [Note] InnoDB: Doublewrite buffer created
2020-07-07T18:32:39.3554343Z 2020-07-07T18:32:39.355197Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2020-07-07T18:32:39.3555973Z 2020-07-07T18:32:39.355420Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2020-07-07T18:32:39.3654956Z 2020-07-07T18:32:39.365267Z 0 [Warning] InnoDB: Creating foreign key constraint system tables.
2020-07-07T18:32:39.4160929Z 2020-07-07T18:32:39.415712Z 0 [Note] InnoDB: Foreign key constraint system tables created
2020-07-07T18:32:39.4170518Z 2020-07-07T18:32:39.416813Z 0 [Note] InnoDB: Creating tablespace and datafile system tables.
2020-07-07T18:32:39.4217915Z 2020-07-07T18:32:39.421390Z 0 [Note] InnoDB: Tablespace and datafile system tables created.
2020-07-07T18:32:39.4218757Z 2020-07-07T18:32:39.421559Z 0 [Note] InnoDB: Creating sys_virtual system tables.
2020-07-07T18:32:39.4376309Z 2020-07-07T18:32:39.437378Z 0 [Note] InnoDB: sys_virtual table created
2020-07-07T18:32:39.4380632Z 2020-07-07T18:32:39.437882Z 0 [Note] InnoDB: Waiting for purge to start
2020-07-07T18:32:39.4889855Z 2020-07-07T18:32:39.488735Z 0 [Note] InnoDB: 5.7.21 started; log sequence number 0
2020-07-07T18:32:39.4978146Z 2020-07-07T18:32:39.497305Z 0 [Note] Plugin 'FEDERATED' is disabled.
2020-07-07T18:32:39.5842087Z 2020-07-07T18:32:39.583904Z 0 [Note] Salting uuid generator variables, current_pid: 6028, server_start_time: 1594146756, bytes_sent: 0, 
2020-07-07T18:32:39.5929145Z 2020-07-07T18:32:39.592692Z 0 [Note] Generated uuid: '3cc22021-c080-11ea-99da-00155d00041a', server_start_time: 1594152784, bytes_sent: 1783360884640
2020-07-07T18:32:39.5932003Z 2020-07-07T18:32:39.592950Z 0 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 3cc22021-c080-11ea-99da-00155d00041a.
2020-07-07T18:32:39.6570161Z 2020-07-07T18:32:39.656643Z 0 [Warning] Gtid table is not ready to be used. Table 'mysql.gtid_executed' cannot be opened.
2020-07-07T18:32:39.6812765Z 2020-07-07T18:32:39.681011Z 1 [Note] Creating the system database
2020-07-07T18:32:39.6815216Z 2020-07-07T18:32:39.681267Z 1 [Warning] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
2020-07-07T18:32:39.7556584Z 2020-07-07T18:32:39.749274Z 1 [Note] Creating the system tables
2020-07-07T18:32:45.6899339Z 2020-07-07T18:32:45.689665Z 1 [Note] Filling in the system tables, part 1
2020-07-07T18:32:45.7052230Z 2020-07-07T18:32:45.704870Z 1 [Note] Filling in the system tables, part 2
2020-07-07T18:32:45.7053427Z 2020-07-07T18:32:45.705095Z 1 [Note] Filling in the mysql.help table
2020-07-07T18:32:47.2451101Z 2020-07-07T18:32:47.244864Z 1 [Note] Creating user for internal session service
2020-07-07T18:32:47.2460472Z 2020-07-07T18:32:47.245837Z 1 [Note] Creating the sys schema
2020-07-07T18:32:49.7657232Z 2020-07-07T18:32:49.765435Z 1 [Note] Bootstrapping complete
2020-07-07T18:32:49.7664555Z 2020-07-07T18:32:49.766227Z 0 [Note] Binlog end
2020-07-07T18:32:49.7680554Z 2020-07-07T18:32:49.767626Z 0 [Note] InnoDB: FTS optimize thread exiting.
2020-07-07T18:32:49.7683289Z 2020-07-07T18:32:49.768153Z 0 [Note] InnoDB: Starting shutdown...
2020-07-07T18:32:49.8705165Z 2020-07-07T18:32:49.870023Z 0 [Note] InnoDB: Dumping buffer pool(s) to D:\a\_temp\data\ib_buffer_pool
2020-07-07T18:32:49.8708453Z 2020-07-07T18:32:49.870642Z 0 [Note] InnoDB: Buffer pool(s) dump completed at 200707 18:32:49
2020-07-07T18:32:51.4529123Z 2020-07-07T18:32:51.452533Z 0 [Note] InnoDB: Shutdown completed; log sequence number 2551701
2020-07-07T18:32:51.4531426Z 2020-07-07T18:32:51.453019Z 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2020-07-07T18:32:51.4930578Z Attempting to connect
2020-07-07T18:32:51.7665460Z ERROR 2017 (HY000): Can't open named pipe to host: .  pipe: MySQL (2)
2020-07-07T18:32:51.9306039Z       PID    PPID    PGID     WINPID   TTY         UID    STIME COMMAND
2020-07-07T18:32:51.9307825Z      1349       1    1349        800  ?         197108 18:32:51 /usr/bin/ps
2020-07-07T18:32:51.9710768Z ##[section]Finishing: Command Line Script

Does anyone have any ideas on how to further troubleshoot this; or maybe even have an answer on what I'm doing wrong?

Best Answer

As usual, after spending hours and hours working with this; I figured it out within 20 minutes of posting a question on it. As usual, it was also something dead simple.

Calling mysqld with the flag --initialize-insecure JUST initializes the database and then it exits. This is not an indication of a problem, this is the intended behavior. My expectation that it would then start MySQL was WRONG.

I needed to call mysqld once to INITIALIZE the database, and then a 2nd time without the --initialize-insecure in order to RUN the database.

Now that I've found the answer it seems so simple / obvious. Oh well, hopefully this will help someone else.