SQL Server Availability Groups – Problem Adding Secondary Replicas

Microsoft Azure completely ignores IPs of any internal network on Azure, if you want any computers to connect to a listener in an AlwaysOn availability group make sure 100% sure you specify the VIP, Virtual IP, of the Cloud Service and not the internal IP of the cluster. You will not be able to connect to the cluster or to the listener.

When on the Dashboard view, note the address shown under Public Virtual IP (VIP) Address. Completely ignore the internal IP Address for listener configuration.

It can connect from SQL 1 server, which is the primary for AG.

By "connect", do you mean it can ping AG-LISTENER from SQL1?

It sounds like what your problem might be is with the port number you chose for your listener. By choosing 5525, you are selecting a non-default port (1433 would be the default).

So when you try to connect to the listener, what does your connection string look like? I'm guessing it looks something like this:

data source = ag-listener; initial catalog = ...

You have two options here. You can either be explicit with your listener's port number:

data source = ag-listener,5525; initial catalog = ...

Likewise, if you're testing this out with SQL Server Management Studio (SSMS), then for the Connect to Server dialog box, instead of putting in ag-listener for the Server name text box, put in ag-listener,5525.

Or you can change the port that your listener is listening on to 1433 (read the below BOL reference before considering this change):

alter availability group YourAvailabilityGroupName
modify listener 'AG-LISTENER'
(
    port = 1433
);

It is worth noting when you can use the default port (1433). Take a look at this reference on BOL explaining when you can and can't use 1433 for the listener (excerpt copy/pasted below for reference):

You can configure the default port to 1433 in order to allow for simplicity of the client connection strings. If using 1433, you do not need to designate a port number in a connection string. Also, since each availability group listener will have a separate virtual network name, each availability group listener configured on a single WSFC can be configured to reference the same default port of 1433.

(portions omitted for brevity)

If you use the default port of 1433 for availability group listener VNNs, you will still need to ensure that no other services on the cluster node are using this port; otherwise this would cause a port conflict.

EDIT: If the above isn't your problem (as seen from your comment below) then, after you look through the logs, I'd say the next best course of action is to start looking at the network traffic to see what is (and isn't) happening. You can use a network monitoring tool like netmon to accomplish this.

Another thing I'd do, and I realize you said the firewall isn't a problem, but I'd see if the port is actually listening (my favorite tool for this is portqry).