Tried to install a F5 VE on ESXi 6.5 U1 but it failed because of the error in the topic. Searched around and tried a bunch of things. In the end updating the UI is what solved the issue.
Instructions here (don't use in production!):
https://labs.vmware.com/flings/esxi-embedded-host-client#instructions
Also found another solution from a japanese technician which I did not try:
http://en-designetwork.hatenablog.com/entry/2017/05/07/big-ip-ve-esxi65
/Patrik