To connect to a SQL Server using a custom port number instead of the default '1433' you need to use a comma ',' between the IP address or hostname and the port number as shown below:
5.0.1.555,10080
Enter the above with your own IP/Hostname and Port number into the "Server name" field if you are using SQL Management Studio and "Server" field if you are using Azure Data Studio.
SQL Management Studio (SSMS) | Azure Data Studio |