SBA users unable to IM using Lync 2013 mobility

I've found an issue where Lync 2013 SBA users could not use mobility due to the undocumented UCWA port, TCP 5088, being filtered by a firewall. The Lync 2013 mobile client would sign in but would time-out during any attempt to use IM.

Please refer to my buddy, flinchböt, who wrote the findings and solution on his blog, http://flinchbot.wordpress.com/2014/07/01/port-5088-missing-from-lync-2013-documentation/

CMS Replication issues

While installing a Lync Server 2010, I ran into an issue where one front-end would not replicate to the Central Management Store. I could ping and telnet to port 445 so that wasn't the issue.  Running netstat from the problem server and filtering on port 445, I received the below errors:

A quick internet search on search on CMS replication issues turned up an excellent post titled Troubleshooting CMS Replication from The Lync Guy's Blog (Fromerly OCSGuy). The Lync Guy ran into this exact issue where access was denied to the CMS database while trying to write to the RTCReplicaRoot folder on the problem server. In his case, a security script had broken NTFS permissions and a complete rebuild (including the OS) was warranted.

My case was somewhat different; I wasn't using a security script and I was trying to avoid re-installing the OS primarily because I had done so previously due to another issue in which the roles and features in Windows Server 2008 R2 displayed an error message in service manager.

Additionally, roles and features could not be changed using Power Shell. I followed the steps here to no avail and rebuilt the OS but I digress. 

I initially tried the steps listed on Louis UC Blog to repair the share permissions to the RTCReplicaRoot folder but this did not work in my case.

So I removed the problem server from the Lync topolopy, performed a bootstrap /scorch (to remove Lync) and deleted the RTCReplicaRoot folder (to ensure that the folder would be recreated by Lync setup with the correct permissions), added the server back to the topology, and re-ran Lync Setup. 

This solved my CMS replication issue!

One Case of Dialed Number not in Service

A trouble ticket came in where a user could not dial a specific number. The client normalized the number but would not dial it. Additionally, the number could be dialed via a PBX or PSTN phone. Since the number normalized and the CSCP test voice route showed the number routing to the gateway, it was easy to guess that the AudioCodes gateways did not have the 630 prefix defined (as shown below).

Note: If this had not been the case I would have looked at the gateway logs to determine if the number was being manipulated correctly and if so, engage the switch room team.

After the prefix was added, Lync clients could dial it successfully. 

Why did the PBX dialing from our organization work? Because the PBXs were updated with the new prefix. The issue was that there was no internal process to update the gateways whenever the PBXs were updated with new prefixes. This issue has been corrected. 

Polycom CX Phone reboot issue - Resolved

During Lync Pilot, users complained that the CX600 Lync Phones would completely reboot after being awaken from sleep mode. After engaging Cisco, it was determined that this was an issue with the 3750-X series switches that were in use.

Here are the notes from the Cisco Engineer that assisted with troubleshooting this issue,

"You have class-2 PoE IP telephones that are using approximately 2.6 -

3.0 Watts when in sleep mode.  When resuming operation, the devices

request a power value of 65 via LLDP.  During this transition, the

phones are being reset by the Cisco switch.  Debugging this issue on the

switch we found the following:

014173: May 16 16:21:32.550: ilpower_powerman_request_from_lldp: rx lldp

power class tlv from port GigabitEthernet0/19

014174: May 16 16:21:32.550: Shut down interface (Gi0/19) since

consumption power 6500 is greater than allocation 3000

Power value of 65 allows utilization between 3.84 and 6.49 W.  The

problem isn't that the device is requesting more power than allowed by

the class, rather that it is attempting to use LLDP to request higher

power.  According to the ANSI standard, this type of request requires a

reload of the device.

###################

http://www.tiaonline.org/standards/technology/voip/documents/ANSI-TIA-10 57_final_for_publication.pdf

10.2.5.4 Power value

The Power Value field shall represent the maximum power consumption of

an Endpoint Device or the minimum power available from a PSE port of a

Network Connectivity Device when the LLDPDU is transmitted. It shall not

be used to indicate instantaneous variations of power required or

offered at the time of transmission of the LLDPDU.

A PD Endpoint Device that requires more power than previously

advertised, due to a hardware reconfiguration, shall not advertise a

higher power value than is currently advertised by the PSE Network

Connectivity Device it is connected to on that port.

If the PD Endpoint Device requires power greater than either the power

value currently advertised by the PSE Network Connectivity Device or the

upper power limit of the corresponding IEEE 802.3af power class, then it

must renegotiate this new power limit by resetting the port (i.e. by

triggering renegotiation of power at the IEEE 802.3af level).

################### "

Therefore, the reboot is expected behavior.  As a workaround to the

issue, the power TLV was disabled from the LLDP field as the Cisco Engineer suggested. 

RESOLVED with Lync Server CU4 (Novemeber) patch!!!

One case of immediate fast busy

This is an issue that occurred early in Lync Pilot stage but was one inherited from the OCS 2007 R2 infrastructure.  Symptom: When a user makes a MOC/Lync call it sometimes goes to an immediate fast busy.  Common experiences from users are below:

·         Users were able to make the call on a typical desk phone immediately after failure.

·         Peers were able to call the number successfully immediately after a failure.

·         Call fails with 5 digit or 10 digit dialing.

·         Intermittent issue in all cases and  difficult or impossible to duplicate.

·         Users with OCS desk phones and headsets experienced the issue.

All client ucc logs contained the highlighted text in this excerpt:

Aug 12 14:56:53 10.1.137.98 (122-124-138) CC -APP|ACU_CLEAR_IN   |   0:61:   1#68:hungup,cause 41h=65=Bearer capability not implemented 00 00 00 00 00 00 00 00 68 41 00 00 00 00 00 00 00 ff 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff 00 ff ff 00 00 00 ff 00 00 00 00 00 00 00 00 00 00 00

Aug 12 14:56:53 10.1.137.98 (   lgr_psbrdex)(6136914   )  pstn recv <-- CALL_DISCONNECTED Trunk:0 Conn:1 RetCause:104 NetCause:65 [Time: 19:56:53]

Aug 12 14:56:53 10.1.137.98 (121-122-138) DIT-PHD|PH_IT_XMIT_IN  |   0: 0:   0|4: 00 01 01 c6

Aug 12 14:56:53 10.1.137.98 (   lgr_psbrdif)(6136915   )  Abnormal Disconnect cause:65#GWAPP_BC_NOT_IMPLEMENTED Trunk:0 Conn:1 [Time: 19:56:53]

Bearer capability not implemented can be caused by one of the following occurrences:

•You need to change the PCM Type value to the setting appropriate for your country. This is the most common cause, especially in countries where G.711 A-law companding is the standard. If your gateway is configured for µ-law and the service provider or PBX is expecting A-law, you will see calls disconnected with this cause code.

•The central office (CO) does not understand an information element in the setup message.

•You are connected to a PBX and you are sending out a network type number when the switch accepts only Unknown or National.

•You are selecting European PRI and you have the progress indicators turned on when they should be off. 

None of the above were applicable in our case.

The AudioCodes M2K gateway logs showed the following: 

Good call setup message notice the mu-law>

0:   1|40:               Q.931:  PD=8 CR=30228(0x7614,Orig) SETUP(05):                                               Bc(04):  3.1kHz-Audio/64k/mu-law                90 90 a2                                                ChanId(18):        B10/excl(PRI)     a9 83 8a                                                CallingNb(6c):                type:natl,plan:ISDN "6153340986"            a1 36 31 35 33 34 33 30 39 38 36                                   CalledNb(70):                type:unknown,plan:unknown "99475224"            80 39 39 34 37 35 32 32 34...          Hex dump: 08 02 76 14 05 04 03 90 90 a2 18 03 a9 83 8a 6c 0b a1 36 31 35 33 34 33 30 39 38 36 70 09 80 39 39 34 37 35 32 32 34 a1

Nov  1 05:40:29 10.1.137.98 (122-123-138) DLD-PHD|PH_DA_RQ       |   0: 0:   0|44:I[52,122]p0 tei=0  02 01 68 f4 08 02 76 14 05 04 03 90 90 a2 18 03 a9 83 8a 6c 0b a1 36 31 35 33 34 33 30 39 38 36 70 09 80 39 39 34 37 35 32 32 34 a1

Nov  1 05:40:29 10.1.137.98 (122-123-138) NS -MNS|MNS_EVENT_IN   |   0: 0

------------------------------------------------------------------------------------------------------------

Bad call with A-law>

0:   1|40:               Q.931:  PD=8 CR=17691(0x451b,Orig) SETUP(05):                                               Bc(04):  3.1kHz-Audio/64k/A-law                90 90 a3                                                ChanId(18):        B15/excl(PRI)     a9 83 8f                                                 CallingNb(6c):                type:natl,plan:ISDN "6158579738"            a1 36 31 35 38 37 35 39 37 33 38                                   CalledNb(70):                type:unknown,plan:unknown "92922426"            80 39 32 39 32 32 34 32 36...          Hex dump: 08 02 45 1b 05 04 03 90 90 a3 18 03 a9 83 8f 6c 0b a1 36 31 35 38 37 35 39 37 33 38 70 09 80 39 32 39 32 32 34 32 36 a1

Cause(08):          cause 41h=65=Bearer capability not implemented            81 c1 04                Hex dump: 08 02 c5 1b 45 08 03 81 c1 04

----------------------------------------------------------------------------------------------------------

Using the A-law causes the bearer capability problem and the call with fail.

Normal calls should have been using mu-law (used for North America and Japan) but on the failed immediate fast busy calls they were trying to use A-law (European coding). 

After changing a bit in the AudioCodes gateway on the outbound call to use mu-law, the failed behavior continued. 

AudiCodes Support was engaged and a firmware patch was developed for this case. 

The "Bearer capability not implemented" error has not returned since the firmware upgrade.

Script to compare an Enterprise Voice number to the assigned one in AD

Import-Module activedirectory
Import-Module lync

Get-csUser | where  {$_.EnterpriseVoiceEnabled -eq 'true'} | foreach{
$u = get-aduser -Properties telephoneNumber -Identity $_.Identity.tostring()
If ($u -ne $null)
{
#strip out all non numerical values
$rawdigits = $u.telephonenumber -replace '\.',''
$rawdigits = $rawdigits -replace '\s',''
$rawdigits = $rawdigits -replace '\(',''
$rawdigits = $rawdigits -replace '\)',''
$rawdigits = $rawdigits -replace '-',''
$rawdigits = $rawdigits.trim()

$LyncLineuri = $_.lineuri

$LyncLineuri = $LyncLineuri -replace 'tel:\+1',''

If($rawdigits -eq $LyncLineuri)
{
write-host $u.distinguishedname  " telephoneNumber attribute - "  $rawdigits  " matches the LineUri "  $LyncLineuri
}
else
{
write-host $u.distinguishedname  " telephoneNumber attribute - "  $rawdigits  " does not match the LineUri " $LyncLineuri
}
}
else
{
write-host $_.Identity.tostring() " no user match found"

}
}

Troubleshooting Certificates in Lync with CAPI2 logging

Certificate issues in Lync 2010 can sometimes be complex to troubleshoot.  In my experience, sometimes the Lync Server Event logs are not always helpful enough to solve the issue. Enter the CAPI2 log (which is not enabled by default).

First Enable the log:  

Let the CAPI2 events collect:

Now investigate:

Other certificate troubleshooting can involve the following:

Enabling the root certificate for all purposes:

Verifying Enhanced Key Usage (depending on Server role check here):

Validating Lync Server access to root Certificate Revocation lists. This can be easily done by copying the CRL Distribution Point Url into a web browser and being prompted for a download :