Examining the syndicate loans market for publicly traded U.S. firms I show that tax avoidance is positively related to loan spreads. Importantly, however, tax-specific premiums disappear for loans with large number of co-leads, which facilitate credit risk diversification, for loans with performance pricing provisions, which facilitate borrower-lender incentive alignment, and for borrowers with CDS contracts, which facilitate credit risk transfer. Moreover, non-bank institutional investors demand higher risk premiums to compensate for their high-risk investment strategies that also account for tax-specific risks and do not have particular focus on tax-specific risk taking. Finally, I show that simultaneous access to private and public debt financing, which reflects greater firm-level financial flexibility and fewer hold-up problems, largely mitigates agency risks associated with all forms of tax avoidance. These syndicate-level risk-mitigating measures work jointly well and are more effective, ex-ante, at moderating tax-specific risks in comparison to maintenance-based covenant structures alone. These results help identify channels through which firms can mitigate non-tax costs associated with tax avoidance and, hence, effectively pursue strategies that persistently reduce their corporate tax liabilities without incurring material agency costs.