While tax-free bonds make the interest tax free, tax saving bonds give you tax benefits for the investment amount. Remember, in case of tax-saving bonds the interest that is paid is fully taxable in the hands of the investor. There are broadly 2 types of tax-saving bonds..
1.  Tax Saving Bonds under Section 80CCF of the Income Tax Act
This was a special addition to Section 80C that provided an additional benefit of Rs.20,000 in the form of exemption in case of investment in infrastructure bonds. Assuming that you are in the peak bracket of 31.2% tax, an investment of Rs.20,000 in these Section 80CCF bonds will entitle you to a tax rebate of Rs.6240 (31.2% of Rs.20,000). Therefore your effective investment in the first year will be only Rs.13,760 and this will substantially enhance your effective yield. However, these Section 80CCF bonds were discontinued effective the AY 2012-13 and are not issued any longer.
2.  Capital Gains exemption bonds under Section 54EC
Under Section 54EC, your long term capital gains arising from assets other than from shares and securities can be reinvested in specified Section 54EC bonds issued by infrastructure companies. If your capital gains are entirely invested in these Section 54EC bonds then the entire capital gains becomes tax free in your hands. However, it needs to be remembered that these investments in Section 54EC bonds will be subject to a lock in of 3 years and any breach of this lock-in will mean that you lose your capital gains tax exemption already claimed. Such investment in Section 54EC bonds will have to be made within 6 months of the generation of capital gains.
So while tax-free bonds offer tax-free interest, tax saving bonds offers you special exemptions for investment!