To change what information is displayed on printed invoices and receipts (including invoices/receipts from the past), go to Account > billing activity and click on Change contact information on invoices.

There is also an Additional info field on this page, which can be used to include your VAT number or tax information for your country.

Or, you can press the Use payment details instead button to just use the contact information that was provided when you entered your payment information.