I have 3 dashcams set up and one uses the car's USB port, but not all car's supply enough power from the USB I understand. It does not explain why he gets powered up correctly when using a GPS and not when directly connected to the A119 presumable using the same cable.
Strange because if the unit is working with an external power supply and through a GPS it should work from the car's power. The only other though is that it a dirty or bad connection in the unit. I would try Viofo directly and submit a ticket at this link;
Try another start up delay perhaps 10-15 seconds instead of the default 5 seconds. You will find it in the settings if you have one of the recent firmwares. The 5 sec delay is a default so there will always be a slight delay in starting up,