A healthy skepticism keeps our belief-system lean, but an unhealthy skepticism makes us starve.
A good rational account of anything should be as simple and uncomplicated as possible. Sometimes reality is complicated, of course, but one should start off (in accord with Occam) by looking for simple explanations before favouring complex one. Anyone can believe anything if they make up ad hoc justifications for their belief.
The hope of a unified and comprehensive belief system is that a small set of basic principles might be explanatory of all phenomena. This is what physics is all about!
Suppose we encounter some new unexplained observation X in the universe. A purely empirical world-view makes it clear how we should approach such observations: make hypotheses, invent experiments to test them, and refine. Notice, however, that this is about the correct response to a material observation which does not seem to fit in with one's current explanatory system - or "rational account" - of the material universe.
On the other hand, it can work the other way too. Sometimes a theoretician notices something a little ad-hoc about a theory (such as Newton being aware of the unsatisfactory nature of both his "action at a distance" gravity and of his "absolute space-time" mechanics) and then proposing a new theory which is less "ad hoc" in this case it was Einstein who proposed Special and then General Relativity. When this is done certain phenomena that had previously just been minor annoyances (such as the deviation of the orbit of Mercury from Newtonian predictions) can become of central importance and indicative of the truth of the new theory.
If we were to allow our epistemology to become cluttered with extras and exceptions (like "we can believe in Y or Z without evidence") then anything would go; yet why would any-one ever do that? It seems to me more that different people disagree about what various evidences can reasonably take to indicate, it seems to me. Even people with very weird beliefs (even schizophrenics) will propose evidence supportive of their delusions.
The metaphysics and epistemology which one is committed to profoundly affects the way in which one evaluates evidence. This can become a closed vicious circle. So someone absolutely committed to a religious scheme [eg Dawkin Atheism] can get themselves into the mind-frame that any evidence which is contrary to their world-view is either inadmissible; or else must be explicable in some way that accords with their world view even if they cannot identify this explication at present.
So a person who is committed to the unreality of ghosts will dismiss all anecdotal evidence of hauntings on the grounds that, because ghosts are unreal, all evidence of ghosts must be false evidence and so should be dismissed in favour of some other as yet unknown "logical/scientific/rational/meterialist/more probable" explanation. I use this as an example. I am not arguing in favour of the reality of ghosts.
Epistemology is the study of truth and knowledge and opinion and how these are relation and how any one can come to have any of them. An "epistemology" is a particular theory of how we come to gain knowledge/truth/opinion.
You are I never have any choice but to deal with experience and observation and similar in accord with our current understanding as to how we gain knowledge. Sometimes, of course, one can have reason to revisit such a theory and attempt to modify it to make it more "fit for purpose" in some way.
I believe, as a matter of faith, that one should never give anything which one wishes to account for or speak of an exemption from rationality - which, I think, is what you are hinting at. I am a committed "rationalist" in the sense that I believe that logic and human reason is applicable to reality and that understanding a thing, even or process largely consists of being able to give a coherent and comprehensive account (not simply a "just so story") of that thing, event or process.