Portfolio lenders also can serve as "niche" lenders because certain things are more important to them than meeting the more standardized underwriting guidelines of a mortgage banker. An example would be a savings & loan which is more concerned with an individual's savings history than being able to fully document income.
If you apply for a loan with a portfolio lender and you are declined, that's it. You're done. If you still think you should be able to qualify for a loan, you have to start over somewhere else.