In SharePoint, I have a request to setup one task list but limit the visibility of the task based on who created the task or who it is assigned to.  Also, there should be a way to denote a task as visible to all within the same task list. 

So, if a task is created by you or assigned to you, you should be able to see it and edit it, but if it is assigned to someone else or created by someone else, it should be hidden from you.  Also, if the task is flagged as visible to all, you should also be able to see it and edit it.

Is there any way to create this effect in a task list?  I have tried researching this, but I cant seem to find any good explanation on whether or not this can be done. 

A lot of what I have read deals with creating a page where the content query web part can be used in conjunction with audiences to create this effect, but the problem is that a user still can access the complete task list directly.  Also, any time you view the task details and close the view, SharePoint takes you back to the complete task list, not the page with the content query web part.

So it seems that to create this effect, I have to use audiences, create a page which uses the content query web part, and modify the default navigation so that working with the tasks, (creating them, viewing them, or updating them) sends you back to the custom page instead of the task list.

Any thoughts on whether or not this is possible or if it even makes sense?

In List Settings | Advanced Settings, you can set Item Level Permissions for item creators. However, that would not give you your second part, so I would suggest creating a SharePoint Designer workflow, adding an Impersonation step and setting the item level permissions to the item Creator and the person Assigned To the item. You could fire the workflow on item creation and item update in case the Assignee changes. This is the secure way of doing this.