Federation Commander

If an Orion doubles its engines, it needs to take an engine damage point at the end of the turn, per doubled engine.

What happens if one or more of the warp engines has no more boxes remaining due to combat damage; what happens to the 'doubling' damage point?

If the engine that is supposed to take the damage from doubling has no boxes left, then the damage point from doubling is "lost". Small consolation, as you are likely dead at this point, but there you are.

What happens if one or more of the warp engines has no more boxes remaining due to combat damage; what happens to the 'doubling' damage point?

I would suspect you just failed When Not to Double Engines 101