Hi,On one of my servers,the perf counter 'RPC Averaged Latency' often goes above 100. Users are complaining about the Outlook "bubble". Here's what the other counters look like:The disk latency countes sec/read sec/write never go above 20ms. The RPC Operations/sec seem high for the user count - 350-500 - 1500 active users - above .15 Ops/sec/userRPC Requests never go above 30 - hover around 10.Since it doesn't appear to be disk latency or server latency, could this be realted to the global catalog server? If so, what counters could I look at? Also, if the RPC Operations/sec is high, shouldn't RPC Requests also reflect that?Thank you.

Do you have any virus scanning engine for Exchange store? If yes, \MSExchangeIS\Virus Scan Queue Length might be a counter of interest. If the Virus Scan Queue Length counter increases with RPC requests for MSExchangeIS, then a potential bottleneck could be the Antivirus software.What version of Exchange are you running? Did you run ExBPA and ExTRA on the server? I recommend trying them as well for troubleshooting.Exchange Best Practices Analyzer (ExBPA)- http://www.microsoft.com/downloads/details.aspx?familyid=DBAB201F-4BEE-4943-AC22-E2DDBD258DF3&displaylang=enExchange Troubleshooting Assistant (ExTRA) - http://www.microsoft.com/downloads/details.aspx?familyid=4BDC1D6B-DE34-4F1C-AEBA-FED1256CAF9A&displaylang=enThanks.Bhargav Shukla - MSFT --------------------- Exchange Reference: Exchange Server 2007 TechCenter - http://technet.microsoft.com/en-us/library/bb124558.aspx; Exchange Server 2003 TechCenter - http://technet.microsoft.com/en-us/library/bb123872(EXCHG.65).aspx; Microsoft Exchange Team Blog - http://msexchangeteam.com/default.aspx

Thanks for your reply. The ExTRA said that the Operations/sec were above .015. I'm now monitoring the AV counter and will post the results when I have some significant data.

The RPC Averaged Latency counter was highest today at 10:00 a.m.Virus Scan Queue Length: 0RPC Averaged Latency: 209RPC Operations/sec: 262RPC Requests: 2Disk Counters are all below 20 ms

What version of exchange are you running? Did you run ExBPA? Can you send me the results of ExBPA? An increase in latency may be caused by the following: An increase in RPC load A bottleneck in one or more server resources Furthermore, a bottleneck may be caused by the following: High RPC load (or any other load on the server)Hardware malfunction or configuration error KB article 839863 discusses the issue and troubleshooting steps - http://support.microsoft.com/default.aspx/kb/839862 Here are two very helpful links to understand different troubleshooting scenarions and what performance counters to use for each. Exchange 2007 - http://technet.microsoft.com/en-us/library/bb201720.aspxExchange 2003 - http://technet.microsoft.com/en-us/library/bb124328(EXCHG.65).aspx

Exchange 2003What has me confused is that I see spikes inRPC Averagedlatency without a corresponding spike in RPC Operations/sec or RPC Requests. Yes, I'll send you the ExBPA. Could this be network latency? If so, is there a way to tell what client is experiencing the latency at that time so I can troubleshoot clients specific network connections? ExMon, Avg. Client Latency, maybe?Teo

RPC Latency represents how long it takes from the time the Store.exe process received the packet to the time it returned the packet. Which clearly means the server is hurting on resources. Either it is not sized properly or needs to be tuned properly. Exmon can help you single out clients that can be putting extra load on the server which is reflected by ExTRA when it flagged 0.15 for operations/sec.Bhargav Shukla - MSFT - Exchange Reference: Exchange Server 2007 TechCenter - http://technet.microsoft.com/en-us/library/bb124558.aspx; Exchange Server 2003 TechCenter - http://technet.microsoft.com/en-us/library/bb123872(EXCHG.65).aspx; Microsoft Exchange Team Blog - http://msexchangeteam.com/default.aspx

What is the server hardware config and what is the disk config (i.e. Local, SAN, NAS). Which disk counters are you viewing? If you havea backup solution is the backup happening during the RPC spike?Sr. Exchange Engineer - Constellation Energy

Areyou are able to analyze the switch (network infrastructure) to see if it is reporting any performance issues?Andrew Sword, MVP

I am having the exact same issue. Random people getting Outlook timeout bubbles. I have uninstalled all Symantec software, disabled TOE, Killed the Blackberry server, I can't see it being a hardware issue.Exchange 2003 EntOne store - 114GI do have alot of user over 2 Gig mailboxes.95% Outlook 2003 ClientsWindows 2003 Server R2Some new things in the last month:10 iPhones7 BlackberrysOutlook plugin on 5 peopleRemote site DC was shutoff without DCPROMO.- Have cleaned ADS though since thenAny help or recommendations are welcome.I came from a company that had stores in the 300G and same setup so I'm at a loss. I don't see anything crazy on the network. Havechanged switch ports and NICs.

Haveyou run ExBPA and ExTRA?I always recommend it as it points out obvious configuration issues. Also make sure you monitor and analyze important Exchange performance counters to determin if the issue is your server hardware/software or is it the network and other factors.Here is a good article about maximizing client performance. http://technet.microsoft.com/en-us/library/bb124670.aspx.Isolating cause of degraded performance is another good reference. http://technet.microsoft.com/en-us/library/bb124328(EXCHG.65).aspxBhargav Shukla - MSFT - Exchange Reference: Exchange Server 2007 TechCenter - http://technet.microsoft.com/en-us/library/bb124558.aspx; Exchange Server 2003 TechCenter - http://technet.microsoft.com/en-us/library/bb123872(EXCHG.65).aspx; Microsoft Exchange Team Blog - http://msexchangeteam.com/default.aspx

I have run those tools andthey bascially said there is high RPC activity without giving me any direction. I do like the look of the links you posted and I'll go over them. My servers are on their own switch and VLAN and one thing I noticed is that when I sniff on that switch on any port I see a good bit ofChecksum errors.I've seen thison 4 serverson that VLAN. I'm not an expert on packets butdoes anyone know if that is normal?I looked up the error and a link on Wireshark said could be "due to high network traffic, bad NIC, etc..", I am planning on swapping the NIC out this weekend but I saw this in the other servers so I'm not sure.Has anyone heardif this could be a virus or maybe an MS patch?-J