"While restoring it disconnets users from replica database" It eplains that you are secondary is stand by and you are using it for read only purpose.
Log restore job requires Exclussive access, So it will disconnect all users and starts the restore job. After finishing restore job it will allow connections.
The reason for your job to stall could be
1) it takes some time to get exclusive access
2) YOur transaction log is too big so it take some time to restore.
3) depends on how often you are running the restore job.
4) if you stop the logshipping and resumes all pending logs will be restored using log sequence number.
- What is the size of transaction log (LDF) file?
- How many rows do you see when you run DBCC LOGINFO command in the same database on primary server.
- When restore is running, have you checked the percent_complete column in sys.dm_exec_requests