According to your post, my understanding is that you wanted to restrict permission for a subsite.
What do you mean system had asked me to create groups ,and I created it for that subsite . But still I am able to see sitecollection groups in subsite?
Did you mean create a group in subsite?
In my opinion, if you dont want to access a subsite for some people, you should first check the users permission in the subsite.
As you had said, you dont want people access one subsite in the site collection, you can create a group in the top site, then add the users into it.
In the particular subsite, you should remove the inherit permissions, then delete the group which created in the top site.
Now the users in the group would not access the subsite.
Thanks & Regards,Jason