An extensive consultation saw the vast majority of those who responded say they were against the use of the additives.
But the Bank said the alternative of using palm oil raised environmental concerns and would cost the Treasury an extra £16.5m over the next ten years.
The Government told the Bank it did not believe a switch would achieve value for money for taxpayers.
Campaigners launched a petition last autumn after it emerged that the new plastic £5 note contained traces of an animal product derivative.
BBC News.