@Marke You can do it with UPS, You would however need to become an account shipper first. I kind of gave up on this as its not easy for people who want to just do a one off shipment but what you are saying you would like to do may make it worthwhile by setting up an account, you would complete all the paperwork up front and open an account by contacting the call center, however its ground service only and you would have to go to a UPS customer center (essentially a delivery depot) rather than a high street Access point (a shop with a sign in the window) each time you wanted to drop the battery off. Turn up at the counter with the package, select standard service and the charges would be billed to your account that you would pay when the bill came.
Come to think of it as an account customer you could probably just book a pick up on line in which ever country you were staying in and the driver would come to your address and collect, you would need to be able to print a label mind you.
Remember it's a ground service so it moves via road and it would be ok through mainland Europe but not strictly speaking if it has to go via ferry to say the Mediterranean islands. I know they do move that way but they should not as that's covered by maritime shipping regulations and all the large carriers don't strictly speaking have the right agreements, however ADR (that's dangerous goods via road) would be able to go from the UK to Sweden. For an example just a couple of days UK-Sweden as it goes through the tunnel, in this case it goes UK-NL-DE then in to Sweden in 2 days.
Its a bit of work for sure but if you were planning multiple stops on a round Europe trip it might be worth it.
Of course this is only worth it if you are flying with your bike to the different destinations, as if you were driving none of this would even be needed, you would just leave the battery on the bike.
If it helps I can get you a contact email for the UPS DG shipping coordinator for your country if you let me know where you are based, the 2 guys for the UK are Geoff and Roy I know them both so hooking you up via email not a problem.
Good luck