Lync 2013 Edge - Beeps when making PSTN calls

Hello,

I have recently started deploying a Lync 2013 topology alongside an existing Lync 2010 topology.  The Lync 2010 Edge Server was removed from the topology and a new Lync 2013 Edge server was added to the topology.  

The Topology has

1 x Lync 2010 Enterprise Server

1 x Lync 2010 Monitoring and Archiving Server

2 x Lync 2013 Standard Edition Servers (Pool Pair)

1 x Lync 2013 Edge Server

1 x Reverse Proxy

1 x PSTN Gateway

2 x SBA/Gateways (Branch offices)

All Edge services are working fine but I have a weird problem.

When making PSTN phone calls Lync Phone Edition beeps 8 times before I hear the ringing.  Dialling from Lync Clients also beeps.  If I restart the Lync 2013 Edge server, then the beeps go away for roughly 1 day, then return.  At one point it was ok for 4 days but then the beeps/delay returned.

I have checked the performance of the server, it is using hardly any CPU or Memory.  The hypervisor the Edge server is homed on also has no obvious performance problems.

The Topology in Lync has been verified and I have re-ran the Lync Deployment Wizard, installed certificates on all Lync servers just to make sure everything is okay.

The TCP binding order of the NICs on the Edge server has been configured so that the Internal NIC is top of the list.  I have verified that the network settings are correct, the External NIC has the edge IP addresses, default gateway and internal DNS servers.  The Internal NIC has the internal IP address, mask, no DNS or gateway.  There is also a static route on the server back to the internal networks (10.0.0.0 255.0.0.0).  Trace routes and pings verify everything is routing the right way.  I have also checked DNS and verified the entry for the Edge server is correct.

The Windows Event Logs of the Edge server show no errors or warnings when the problem occurs.

Any ideas?  It is leading me to believe this is more of a bug than a configuration problem as everything works fine then randomly just stops.  The only workaround I have is to restart the Edge server.

Just to add - This impacts users who are homed on the Lync 2010 and Lync 2013 pools.  Only when the Edge server is restarted, everything returns to normal.

OS of all servers is Windows Server 2008 R2

  • Edited by UC Chris Tuesday, May 07, 2013 2:46 PM
May 2nd, 2013 11:20am

Hi

The  Beep before a connection is usually due to a network/call setup delay. It is a tone to tell you that it is trying. 

As you said the impact for your homed users only Internal to Internal should not got through the Edge. 

Free Windows Admin Tool Kit Click here and download it now
May 2nd, 2013 12:59pm

Hello,

Thank you for the reply.  Correct, this is only impacting Internal > PSTN calls.  

A restart of the Edge server resolves the problem.  It is causing frustration with end users so I need to fix it.

I have also tried to disable partner federation discovery (as I read this could cause this problem) but it did not help.

I am just looking for more avenues to check as I seem to have hit a brick wall.

Regards

May 2nd, 2013 1:13pm

Hi 

Why your outbound calls are going via Egde it should not be the case. Would you please run Lync logging tool on Lync server(select Mediation Server, SIP stack and S4 ) 

http://blog.schertz.name/2011/06/using-the-lync-logging-tool/

Free Windows Admin Tool Kit Click here and download it now
May 2nd, 2013 1:43pm

Hi,

It is very strange as the issue only happen for Internal>PSTN calls but it can be fixed by restarting Edge. I would like to confirm if it is convenient for you to remove Edge association from topology tempoary to check whether the issue is fully related to Edge Server.

Does the issue happen when internal calls to a external lync user?

You can also try to use Microsoft Network Monitor with port and IP filters to check connexions.

Here is another simlar case for your reference. You can follow pslager's reply to check your DNS entries.

http://social.technet.microsoft.com/Forums/en-US/ocsmanagement/thread/1d90eb61-4a00-4e3b-b847-26bbab3f61c8/

May 3rd, 2013 11:22am

Hi Kent,

If I remove the Edge association from Topology Builder the beeps go away.

I have already seen the post above and confirmed that I only have 1 DNS entry in internal DNS for the internal IP of the Edge server.

I have not been able to test a Lync Internal to Lync External call.

I am just waiting for the problem to reoccur so that I can run the Lync Logging Tool.

What is interesting is if I run the Lync Logging Tool now (whilst everything is ok), make a PSTN call, open the log file and search for the Edge servers hostname there are no references to it.....  Very confusing!

Regards

Free Windows Admin Tool Kit Click here and download it now
May 3rd, 2013 12:18pm

Hello,

Here is the output of the Lync Logging Tool when making a PSTN call and the "beeps" are occurring.

Lync Logging

HTV440134 is the Lync 2013 Edge.

Regards

May 7th, 2013 12:28pm

Make sure your edge server can reach the front-end server via its FQDN, and you've got your routes setup properly (ie.  default gateway set only on external interface, and persistent routes setup to the internal network).  If your edge can't resolve the FQDN, add a HOSTS entry if necessary.  I've seen cases where seemingly unrelated edge configuration issues cause problems like the one you're seeing.

Free Windows Admin Tool Kit Click here and download it now
May 7th, 2013 4:34pm

Hi 

Check if you are able to Ping Edge server from the client.

May 7th, 2013 5:24pm

Hi,

Ping from client to edge server is fine.

Ping FQDN of front end servers from edge server is ok (and vice versa)

The Edge server has 2 NICS

Internal NIC has internal IP address,  no DNS servers, no gateway

External NIC has external IP addresses, internal DNS servers and default gateway

Persistent static routes are configured for the Internal networks

The TCP binding order is configured so that the Internal NIC is top of the list, as follows:

Local Area Connection (Internal network)
DMZ (External Network)
[Remote Access connections]

Pings confirm that everything is routing the correct way.  Checked DNS to make sure that the edge servers DNS entries are correct (no duplicates etc).

Free Windows Admin Tool Kit Click here and download it now
May 7th, 2013 5:45pm

What IPs are your external edge services using?  Are you using NAT?  When you ping the FQDN for the AV service, does it return the private or public IP?
May 7th, 2013 6:21pm

Hi,

The external edge services are using 3 public IP addresses, not using NAT.

If I ping the fqdn of the AV service it returns the public IP address.

Regards


  • Edited by UC Chris Tuesday, May 07, 2013 3:30 PM
Free Windows Admin Tool Kit Click here and download it now
May 7th, 2013 6:29pm

Can you confirm if UDP port 3478 is allowed from any internal IP address to the internal IP of the edge?  I'm assuming there's a firewall between your internal edge interface and the internal network.
May 7th, 2013 6:43pm


There isn't a firewall between the internal edge interface and internal network.

The internal NIC is on the private LAN and the external NIC is in the public DMZ.

Regards

Free Windows Admin Tool Kit Click here and download it now
May 7th, 2013 6:49pm

Just noticed something interesting.  When the problem occurs if I try to stop the following service on the Edge:

Lync Server Audio/Video Edge

It fails and just says "Stopping"

If I try to kill MediaRelaySvc.exe in task manager nothing happens.

I have also noticed that when restarting the Edge server the beeps still occur (it is always 8 beeps).  As soon as the Lync Server Audio/Video Edge service starts, the beeps stop.  Just tested PSTN call whilst this service was starting and got the beeps, as soon as the service started the beeps went away.

Also after a restart I can start and stop the Lync Audio/Video Edge service ok.

May 7th, 2013 6:59pm

Does the Lync event logs give any clue at all?  Look in Services - Applications and Services Logs - Lync Server (not the Windows application logs)
Free Windows Admin Tool Kit Click here and download it now
May 7th, 2013 7:07pm

Nothing, the only errors are "In the past 1 minutes the server received 1 invalid incoming certificates" etc.  These are all from the same IP address and happen randomly throughout the day.

The only thing I haven't tried/looked at yet is rebuilding the Edge server.  It was originally built from a fresh template, all windows updates installed etc.  The OS is Windows Server 2008 R2, so I don't know if installing Server 2012 will make a difference.  I was hoping I could find a fix though.....


  • Proposed as answer by Kent-Huang Monday, May 27, 2013 12:10 PM
  • Marked as answer by Kent-Huang Tuesday, May 28, 2013 3:44 PM
  • Unmarked as answer by UC Chris Monday, June 03, 2013 8:46 AM
  • Unproposed as answer by UC Chris Monday, June 03, 2013 8:50 AM
May 7th, 2013 7:50pm

Is that on the edge?  Or the front-end?

I'm sure its something very small that's been overlooked. Maybe try putting an internal DNS server on the internal-facing NIC (you don't have any right now, right?), and putting an external DNS on the external facing one.

Free Windows Admin Tool Kit Click here and download it now
May 9th, 2013 10:10pm

I tried various combinations of DNS servers but that did not help.

I have rebuilt the server on Windows Server 2012.  Same name, same IP address, same NIC settings (DNS, GW, static routes), same hypervisor, same cpu's and memory, same SAN LUN etc.  No beeps so far!

  • Proposed as answer by Raju_rajuMVP Tuesday, May 28, 2013 2:29 PM
  • Marked as answer by Kent-Huang Tuesday, May 28, 2013 3:44 PM
  • Unmarked as answer by UC Chris Monday, June 03, 2013 8:46 AM
  • Unproposed as answer by UC Chris Monday, June 03, 2013 8:50 AM
May 28th, 2013 5:28pm

Installed Lync 2013 CU1 and beeps are back......

Think this might be a call to support.


  • Edited by UC Chris Monday, June 03, 2013 8:45 AM Changed "Lync CU1" to "Lync 2013 CU1"
Free Windows Admin Tool Kit Click here and download it now
June 3rd, 2013 11:45am

Hi
Any updates on this issue= I seem to have the same problem.

Regards
Jonatan

August 13th, 2013 11:59am

Hi,

I have since moved jobs so no longer work on that Lync deployment.  It was logged with Microsoft support but from what my old colleagues are saying it still isn't fixed.  Everything works fine then the Lync AV Edge service has a handle leak.  When the beeps start the handles on the AV Edge service go up to over 100,000.  Antivirus software was completely disabled, it happend on Server 2008 R2 and 2012, CPU, and Memory usage is low.  Everything just points to a handle leak / bug with Lync. 

Last update I heard MS support asked for a default gateway to be put on to the internal Edge interface so I am guessing they are clitching at straws.  I am amazed MS have still not resolved this.

I had to configure a scheduled task to reboot the Edge server every night as a workaround.

Free Windows Admin Tool Kit Click here and download it now
August 13th, 2013 1:02pm

I had the same issue. When I try to call somebody internal I receive 4 bleeps before the free line tone comes in.

Make sure you can ping from the vlan of your phone to the Edge server and also from the Edge server to your phone.

I have being testing this senario. I have added the phone in our Default VLAN from this VLAN we can reach the EDGE Server and from our Phone Vlan we are not able to reach the EDGE. Need to check the routing this will solve your problem.

October 24th, 2013 5:04pm

I had this problem and found that it is related to the Edge server's DNS.

To resolve:

- Go to the Network properties on the External/DMZ card. Disable DNS registration. This will be the card that has the three ip addresses on it.

- Open your local DNS and remove the LynceEdge DNS addresses that are pointing to the DMZ.

- Open a command prompt on the Edge Server and type ipconfig /flushdns. Repeat the same on the FrontEnd and Mediation servers just to be even.

- Wait 2 minutes and try a call. The beeping will be reduced to 1 maybe 2 tops.


  • Edited by JerseyCatt Tuesday, June 17, 2014 12:58 PM
  • Proposed as answer by Rossknightdaw Friday, August 22, 2014 12:40 PM
  • Unproposed as answer by Rossknightdaw Friday, August 22, 2014 12:43 PM
Free Windows Admin Tool Kit Click here and download it now
June 17th, 2014 7:27am

Hi there,

I don't suppose this issue was identified and with a known solution listed?  We have the same issue with Lync 2013 and CU2?

Adding Edge to the Deployment - is causing IP phones to take 8 rings before dialing out - a restart of the Edge server sorted the issue for a day - now the issue is back!

Any response would be appreciated - thanks.
August 22nd, 2014 3:48pm

Sorry I'm no longer working on that Lync environment.  I believe this was eventually resolved by changing the following SRV record:

_sipinternaltls._tcp

It was pointing to the Lync 2010 pool, apparently changing it to the Lync 2013 pool resolved the Edge Beeps issue (Both pools were both online as it was during a migration).  Personally I can't see how this DNS record which is used for Pool discovery would cause this......  Something wasn't right in Lync and there was definitely a handle leak on the Edge server.  However I have deployed many Lync environments since exactly the same way and never seen the issue.  Your hypervisor doesn't happen to be VMware?  That is the only difference for me....

Free Windows Admin Tool Kit Click here and download it now
August 22nd, 2014 4:02pm

Thanks Chris for the prompt response.

Our environment is running on VMware, however is a pure 2013 environment. I agree on the 'not sure how this DNS entry could resolve the issue', but have checked the entry anyway.

I suspect something changed in the service over time. Not sure if you have any other sites running CU2 with similar issues.

Will revert for interest sake if we and identify the cause of the issue.

August 22nd, 2014 4:13pm

Things that I've seen that have caused this behaviour in the past:

  1. External edge NIC publishing the external NIC IP address as the internal edge server FQDN in DNS, causing internal communications to attempt to talk to the external NIC instead of the internal NIC.
  2. Internal DNS A record for the edge server has both internal and external IPs listed (caused by #1 above)
  3. Improper firewall configuration
  4. Default gateway set on both internal and external edge NIC (should only be set on external NIC, with static routes setup for the internal NIC)
Free Windows Admin Tool Kit Click here and download it now
August 22nd, 2014 4:32pm

Hi Carey,

Very interesting, the environment I had problems with was hosted on VMware ESX 5.  The Lync topology was Lync 2013 standard, collocated med server, single edge in a Pool Pair with another SE server (used purely for failover).  I have since only deployed Lync 2013 (Standard and Enterprise) on Hyper-V and not seen this particular issue again.

Have you tried installing Lync 2013 CU5 (August updates)?

Ken,

I have just read through the history of this post and I checked all 4 points mentioned.  Carey it is worth following the points in Ken has made to verify Edge config (I'm positive I did this).

Cheers

Chris


August 22nd, 2014 4:44pm

Hi Chris, have you given these steps related the Edge Server DNS settings a try?

- Go to the Network properties on the External/DMZ card. Disable DNS registration. This will be the card that has the three ip addresses on it.

- Open your local DNS and remove the LynceEdge DNS addresses that are pointing to the DMZ.

- Open a command prompt on the Edge Server and type ipconfig /flushdns. Repeat the same on the FrontEnd and Mediation servers just to be even.

- Wait 2 minutes and try a call. The beeping will be reduced to 1 maybe 2 tops.


  • Edited by JerseyCatt Friday, August 22, 2014 2:52 PM
Free Windows Admin Tool Kit Click here and download it now
August 22nd, 2014 5:51pm

Hi All,

We have tried the items suggested.

- External Card DNS, gateways on on nic, static routes, etc

We have tried to stop the av edge service and encountered the same as above - 'stopping'.

Even with a 'taskkill /f /pid xxxx' this still is in an unstoppable state. we are awaiting a reboot and then refocusing attention on the following

1. Ensure all other configs on the edge are correct in-case an external firewall port is causing the issue? (there is no internal firewall)

2. CU deployment - just requires a bit of planning as there are numerous other pools and sites in the environment.

Thanks to all so far - I will post results if they resolution is evident.

August 22nd, 2014 5:59pm

Hi All,

Something else to check is the Edge Server and verify the it doesn't have the IIS role installed. This will cause a port conflict with the Lync Server Audio/Edge service since it uses ports 80/443. A conflict with these ports will prevent the service from starting. Resulting in the delay for inbound and outbound calls.

Best case scenario is to remove the IIS role or stop and disable World Wide Web Publishing Service.

After perfoming this it solved the beep delay issue I was having since i just migrated our environment from Lync Server 2010 to 2013 literally yesterday evening.

In my previous post I was in the 2010 environment and my problems were related to Edge DNS records that had to be removed. Disabling "Register this connection's addresses in DNS" on the external NIC and deleting the records took care of it. 

Bottom line: any beep delay issues usually originate from a problem with the Edge server.

Thanks.

Free Windows Admin Tool Kit Click here and download it now
September 19th, 2014 12:30am

Hi,

did you find a solution so far?

thanks.

November 8th, 2014 11:50am

In the past this issue was related to DNS records for me.

The solution for me in the most recent scenario what ensuring that service was running as expected. Note my previous post below.

---------

Something else to check is the Edge Server and verify the it doesn't have the IIS role installed. This will cause a port conflict with the Lync Server Audio/Edge service since it uses ports 80/443. A conflict with these ports will prevent the service from starting. Resulting in the delay for inbound and outbound calls.

Best case scenario is to remove the IIS role or stop and disable World Wide Web Publishing Service.

After perfoming this it solved the beep delay issue I was having since i just migrated our environment from Lync Server 2010 to 2013 literally yesterday evening.

In my previous post I was in the 2010 environment and my problems were related to Edge DNS records that had to be removed. Disabling "Register this connection's addresses in DNS" on the external NIC and deleting the records took care of it.

Bottom line: any beep delay issues usually originate from a problem with the Edge server.

Free Windows Admin Tool Kit Click here and download it now
November 10th, 2014 9:15pm

Adding a persistent route to all networks containing clients solved this for me.

Ref: http://technet.microsoft.com/en-us/library/gg412787.aspx

  • Proposed as answer by Christer C Wednesday, December 03, 2014 9:06 AM
December 3rd, 2014 11:06am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics